Three-dimensional model generation method based on interactive editing, computer medium and equipment

Abstract

The invention discloses a three-dimensional model generation method based on interactive editing, a computer medium and equipment. The method comprises the following steps: acquiring an initial three-dimensional model; modification parameters of the user for the initial three-dimensional model are obtained, the modification parameters are parameters of simulation modification performed by the user under target rendering characterization, the target rendering characterization comprises at least one of dominant rendering characterization and invisible rendering characterization, and the user can perform simulation modification under appropriate rendering characterization; the modification parameters are converted into modification constraints of the target three-dimensional model generation process; the modification constraint is used for guiding the modification direction of the initial three-dimensional model; and finally, based on the modification constraint and the initial three-dimensional model, generating a target three-dimensional model expected by the user. According to the method, the modification effect of the initial three-dimensional model can be restrained to conform to the expectation of a user, unreasonable modification in simulation modification can be corrected, the detail modification degree of the user can be reduced, and the accuracy of three-dimensional model generation can be improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of three-dimensional model generation, and in particular, relates to a three-dimensional model generation method based on interactive editing, a computer medium and a device. Background Art

[0002] 3D model generation refers to the use of deep learning and other technologies to generate corresponding digital 3D models based on user input text or images. 3D models contain a variety of digital representations, including polygonal meshes, signed distance fields, and rendering parameters. These representations can be converted to each other through algorithms. 3D model generation focuses on using neural networks to generate data in one of these representations.

[0003] In the related art, some solutions focus on generating specific categories to achieve specific field applications. Such solutions can achieve expert generation in specific categories, but are limited by training data and cannot achieve high-quality generation of arbitrary three-dimensional models. There are also some solutions that can achieve the generation of unlimited text or images based on unlimited categories, but there is a problem that the generated results do not meet user expectations. In addition, during the generation process, even if it is found that the intermediate results deviate from user expectations, or the user has new inspiration based on the intermediate results, the existing solutions do not allow the user to intervene midway, that is, it is impossible to promptly feedback the generated intermediate results to the user and accept the user's modification feedback.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present application, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] The purpose of the present invention is to provide a three-dimensional model generation method, computer medium and equipment based on interactive editing, so as to solve the problem in related technologies that it is difficult to generate a three-dimensional model that meets expectations.

[0006] According to one aspect of an embodiment of the present application, a method for generating a three-dimensional model based on interactive editing is provided, the method comprising:

[0007] Obtaining an initial three-dimensional model;

[0008] Acquiring a modification parameter of the user for the initial three-dimensional model, where the modification parameter is a parameter simulated and modified by the user under a target rendering representation, where the target rendering representation includes at least one of an explicit rendering representation and an invisible rendering representation;

[0009] The modification parameters are converted into modification constraints in the target three-dimensional model generation process; the modification constraints are used to guide the modification direction of the initial three-dimensional model;

[0010] Based on the modification constraints and the initial 3D model, a target 3D model desired by the user is generated.

[0011] In some embodiments, obtaining the modification parameters of the user for the initial three-dimensional model includes: when the target rendering representation is an explicit rendering representation, obtaining the modification area and the modified explicit rendering parameters; the modification area is the area selected by the user when performing simulated modification on the initial three-dimensional model; when the target rendering representation is an implicit rendering representation, obtaining the modification area and the interactive text information input by the user for the initial three-dimensional model.

[0012] In some embodiments, the modification parameters are converted into modification constraints of the target three-dimensional model generation process, including: when the target rendering representation is an explicit rendering representation, determining the regularization function and regularization factor of this simulation modification according to the type of modification operation used by the user; the modification operation types include: conventional geometric operations, rigid dragging, deformation dragging, combination and deletion; determining the modification constraints based on the initial explicit rendering parameters of the initial three-dimensional model, the modified explicit rendering parameters, the regularization function and the regularization factor.

[0013] In some embodiments, the modification parameters are converted into modification constraints of the target three-dimensional model generation process, including: when the target rendering representation is an implicit rendering representation, obtaining the initial implicit rendering parameters of the initial three-dimensional model; using a large language model to convert the interactive text information into prompt words; the prompt words refer to the vocabulary that can be understood by the three-dimensional model optimization network; based on the initial implicit rendering parameters, the prompt words and the modification area, determining the modification constraints.

[0014] In some embodiments, based on the initial implicit rendering parameters, the prompt word and the modification area, the modification constraints are determined, including: based on the prompt word and the modification area, determining the first implicit rendering parameter corresponding to the modification area; obtaining the second implicit rendering parameter of the maintenance area; the maintenance area is the area in the initial three-dimensional model excluding the modification area; based on the initial implicit rendering parameters, the first implicit rendering parameters and the second implicit rendering parameters, determining the first modification constraint of the modification area and the second modification constraint of the maintenance area.

[0015] In some embodiments, the method further includes: obtaining an influence proportional factor of the maintenance area, the influence proportional factor being used to characterize the ratio of the influence degree of the prompt word on the maintenance area and the influence degree on the modification area; and determining a third modification constraint of the maintenance area based on the second modification constraint and the influence proportional factor.

[0016] In some embodiments, based on the modification constraints and the initial three-dimensional model, generating a target three-dimensional model desired by the user includes: obtaining a guidance degree factor corresponding to the modification constraints; and obtaining the target three-dimensional model based on the guidance degree factor, the modification constraints and the initial three-dimensional model.

[0017] In some embodiments, obtaining the initial three-dimensional model includes: obtaining initial input information from a user; and constraining the initial three-dimensional model generation process using a specified optimization algorithm to generate an initial three-dimensional model corresponding to the initial input information.

[0018] According to one aspect of an embodiment of the present application, a computer medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the three-dimensional model generation method based on interactive editing provided by any embodiment of the present application is implemented.

[0019] According to one aspect of an embodiment of the present application, an electronic device is provided, comprising: a processor; a memory for storing executable instructions of the processor; the processor executes the executable instructions to enable the electronic device to implement the three-dimensional model generation method based on interactive editing provided in any embodiment of the present application.

[0020] In the technical solution of the present application, an initial three-dimensional model is obtained; the modification parameters of the user for the initial three-dimensional model are obtained, and the modification parameters are the parameters that the user simulates and modifies under the target rendering representation, and the target rendering representation includes at least one of the explicit rendering representation and the invisible rendering representation, so that the user can simulate and modify the three-dimensional model in a variety of ways; and then the modification parameters are converted into modification constraints for the target three-dimensional model generation process; the modification constraints are used to guide the modification direction of the initial three-dimensional model, so as to constrain the modification effect of the initial three-dimensional model to meet the user's expectations, and to correct unreasonable modifications in the simulated modification; finally, based on the modification constraints and the initial three-dimensional model, the target three-dimensional model expected by the user is generated. Through the above method, not only the generated target three-dimensional model is closer to the user's expectations, but also through the user's brief simulated modification, the corresponding modification constraints can be automatically formed, and the modification constraints can automatically adjust the initial three-dimensional model, which can reduce the degree of detail modification by the user and improve the accuracy of the three-dimensional model generation.

[0021] It should be understood that the above general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the specification are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 The flowchart of the method for generating a three-dimensional model based on interactive editing provided in one embodiment of the present application is schematically shown.

[0024] Figure 2 The schematic diagrams schematically show the conventional geometric operation before and after modification provided by an embodiment of the present application.

[0025] Figure 3 The diagram schematically shows the rigid drag operation before and after modification provided by an embodiment of the present application.

[0026] Figure 4 The schematic diagram schematically shows the deformation drag operation before and after modification provided by an embodiment of the present application.

[0027] Figure 5 The structural block diagram of an electronic device provided in one embodiment of the present application is schematically shown.

[0028] Figure 6 The computer system structure block diagram schematically shows the electronic device used to implement the embodiment of the present application. DETAILED DESCRIPTION

[0029] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be more comprehensive and complete and fully convey the concept of the example embodiments to those skilled in the art.

[0030] In addition, the features, structures or characteristics described in the present application may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, realizations or operations are not shown or described in detail to avoid blurring the various aspects of the present application.

[0031] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0032] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0033] Three-dimensional models and 3D models both refer to digital representations of three-dimensional objects, such as polygonal meshes, symbolic distance fields, and rendering parameters, and these representations can be converted to each other through algorithms. Three-dimensional model generation focuses on using neural networks to generate data in one of these representations. In related technologies, some solutions focus on generating specific categories to achieve specific field applications. Such patents can achieve expert generation in specific categories, but are restricted by training data and cannot achieve high-quality generation of arbitrary three-dimensional models. There are also some solutions that can achieve the generation of unlimited text or images based on unlimited categories, but there is a problem that the generation results do not meet user expectations. For example, when the user inputs the text "a wizard", most solutions can usually generate high-quality results. However, when the user inputs the text "a wizard holding a sword in his left hand, a magic wand in his right hand, and a magic hat", the final generated result is likely to have several problems: (1) only a wizard is generated, and one or more of the elements such as "sword", "magic wand" and "magic hat" may be missing; (2) even if all elements are generated, it cannot be guaranteed that the "sword" is held in the "left hand" and the "magic wand" is held in the "right hand"; (3) the generated "sword", "magic wand", "magic hat" and "wizard" may not be what the user wants in terms of shape and texture details, such as the posture of the character, the style of the "magic wand" or the shape of the "magic hat". The main reason for the above problems (1) and (2) is that high-quality 3D training data is very scarce, and the ability of the 3D generation neural network itself is low. When the input is more complex or deviates from the distribution of the training data set, it is easy to generate inconsistent results, resulting in uneven quality of the results. In practice, especially when it comes to text descriptions of quantity and spatial position, neural networks usually have difficulty in correctly understanding the user's intentions. Regarding the above problem (3), due to the inherent complexity of the three-dimensional model itself, it is difficult for text input to accurately and completely describe the model that the user expects to generate, and overly complex text will also exceed the understanding ability of the neural network. Even if image input is used, if the user does not expect to generate a real photographic object, how to obtain an image that meets the expectations also faces the same problem, and the image itself is difficult to reflect the complete three-dimensional structure. In addition, during the generation process, even if it is found that the intermediate results deviate from the user's expectations, or the user has new inspiration based on the intermediate results, the relevant technology does not allow the user to intervene midway, that is, it is impossible to promptly feedback the generated intermediate results to the user and accept the user's modification feedback; instead, it can only be achieved through a large number of repeated generation and then the selection of relatively satisfactory results, or the collection of more text / image 3D model data pairs, and the improvement of the overall ability of the network through a large amount of training. Therefore, the present application provides a three-dimensional model generation method based on interactive editing, which guides the generation process of the target three-dimensional model by obtaining the simulated modifications made by the user to the initial three-dimensional model, such as Figure 1 As shown, the method includes S110 to S140, and the specific process is as follows.

[0034] S110: Obtain an initial three-dimensional model.

[0035] Specifically, the initial 3D model can be a preset 3D model, for example, when a user selects a model sample "house" in the generation interactive interface, the model sample is used as the initial 3D model. Alternatively, the initial 3D model can be automatically generated according to the text information input by the user, for example, when a user inputs "red brick house", a house made of red bricks is automatically generated.

[0036] In some embodiments, obtaining the initial three-dimensional model includes: obtaining initial input information from a user; and constraining the initial three-dimensional model generation process using a specified optimization algorithm to generate an initial three-dimensional model corresponding to the initial input information.

[0037] Specifically, the initial input information refers to the information input by the user for the first time during the generation process of the target three-dimensional model. The initial input information is the information of the user describing the expected three-dimensional model, and its form may include multiple forms, such as text, images, and three-dimensional models. The specified optimization algorithm may be score distillation sampling (SDS), which is a method based on deep learning iterative optimization, which can use a two-dimensional image generation network to optimize the rendering parameters of a three-dimensional model to be generated. The specific formula for the optimization gradient of the score distillation sampling can be the following formula 1:

[0038]

[0039] In the above formula 1, ε, z, y, t, ω and are the basic parameters of the diffusion model, where ε represents the Gaussian noise added to the data, z represents the noisy data, y represents the input conditional information, t represents the noise time step, ω represents the classifier-free guidance enhancement factor, and Represents the neural network parameters. Fractional distillation sampling calculates the predicted noise of the rendering result The difference between the actual noise ε, distilling the prior information from the pre-trained image diffusion model, and using the classifier-free guided enhancement, through the gradient Act on the rendering parameters. θ represents the explicit rendering parameters, and x=g(θ) corresponds to the rendering result calculated by the rendering function g under the explicit rendering parameters θ. Constraining the initial 3D model generation process can be understood as guiding the generation of an initial 3D model that conforms to the initial input information.

[0040] S120: Acquire modification parameters of the user for the initial three-dimensional model, where the modification parameters are parameters that the user simulates and modifies under a target rendering representation, and the target rendering representation includes at least one of an explicit rendering representation and an invisible rendering representation.

[0041] Specifically, the initial three-dimensional model is usually different from the expected result of the user. The present application can provide the user with an operation tool for simulated modification in the interactive interface so that the user can simulate and modify the initial three-dimensional model. Simulated modification can be understood as a modification that does not really act on the initial three-dimensional model, or a modification that does not directly act on the initial three-dimensional model. For example, the initial three-dimensional model can be copied to obtain a copied three-dimensional model, and the modifications made by the user are applied to the copied three-dimensional model to display the modification results to the user without changing the initial three-dimensional model. Explicit rendering representations include point clouds, voxels, triangular meshes, volume rendering, and Gaussian splatting. Gaussian splatting is a 3D data representation method that corresponds 3D model rendering parameters to a Gaussian ellipsoid in 3D space. Implicit rendering representations include signed distance functions (SDF), occupancy fields, and neural radiance fields (NeRF). Neural radiance field is a 3D data representation method that uses a neural network to fit the partial rendering results of a 3D model to store implicit rendering parameters in the neural network. For example, taking Gaussian splash as an example, the specific form of the explicit rendering parameter θ is as follows:

[0042]

[0043] In the above formula 2, c i , o i , μ i and∑ i They represent the color, opacity, position and covariance of Gaussian sphere i respectively, and N is the number of Gaussian spheres.

[0044] In some embodiments, obtaining the modification parameters of the user for the initial three-dimensional model includes: when the target rendering representation is an explicit rendering representation, obtaining the modification area and the modified explicit rendering parameters; the modification area is the area selected by the user when performing simulated modification on the initial three-dimensional model; when the target rendering representation is an implicit rendering representation, obtaining the modification area and the interactive text information input by the user for the initial three-dimensional model.

[0045] Specifically, when the target rendering representation is an explicit rendering representation, the user can change the position and direction of the Gaussian sphere of the selected area in the initial 3D model through the interactive tool. The selected area is the modified area S. Combining with formula 2, it can be known that the explicit rendering parameters modified by the user include the position μ i and covariance∑ i, where i∈S. When the target rendering representation is an implicit rendering representation, the user can input the content that he intends to modify through the interactive tool. For example, the user first selects the area to be modified in the initial 3D model, and then inputs the text "the color of the hat is red", where the area selected by the user is the modification area, and the text input by the user is the interactive text information. The modification area and the interactive text information are the modification parameters that need to be obtained.

[0046] S130, converting the modification parameters into modification constraints in the target three-dimensional model generation process; the modification constraints are used to guide the modification direction of the initial three-dimensional model.

[0047] Specifically, the modification constraints of the target three-dimensional model generation process can be represented by a loss function. It should be understood that the loss function is a function that calculates the predicted value and the true value. When the predicted value is infinitely close to the true value, the difference between the two can be minimized. Then when editing under the target rendering representation, the true value is the three-dimensional model obtained by the user's model modification, and the predicted value is the target three-dimensional model. If the user makes modifications under both the explicit rendering representation and the implicit rendering representation, the modifications under the two rendering representations are recorded separately, and the modification constraints corresponding to the explicit rendering representation and the modification constraints corresponding to the implicit rendering representation are converted respectively. When converting the modification parameters into modification constraints, the corresponding preset loss function is determined according to the type of model modification, and the modification parameters are substituted into the preset loss function to calculate the corresponding modification constraints.

[0048] In some embodiments, the modification parameters are converted into modification constraints of the target three-dimensional model generation process, including: when the target rendering representation is an explicit rendering representation, determining the regularization function and regularization factor of this simulation modification according to the type of modification operation used by the user; determining the modification constraints according to the initial explicit rendering parameters of the initial three-dimensional model, the modified explicit rendering parameters, the regularization function and the regularization factor.

[0049] Specifically, the modification operation types include: conventional geometric operations, rigid dragging, deformation dragging, combination and deletion. Different modification operation types may correspond to different regularization functions and regularization factors. Regularization functions and regularization factors may be used to constrain the structure of the three-dimensional model to maintain rationality. For example, the user wants the character model's left arm to be raised upward, but mistakenly stretches the left arm too long. The regularization function and regularization factor may correct the modification parameters of the process so that after the initial three-dimensional model is superimposed with the modification constraint, a reasonable result can be obtained. Exemplarily, when the target rendering representation is an explicit rendering representation, the regularization function R and the regularization factor λ are determined according to the modification operation type. The general form of the modification constraint is as follows: Formula 3:

[0050] L θ =λR(θ,θ′)+‖θ-θ′‖.

[0051] For example, conventional geometric operations include translation, rotation, scaling, and stretching, such as Figure 2 As shown in the figure, a model is translated and rotated. The black ellipse in the figure represents the Gaussian ball in the Gaussian splash representation. After performing conventional geometric operations, the position μ of the Gaussian ball in the modified area can be obtained. i ′ and covariance ∑ i ′, where i∈S, N is the number of Gaussian balls, and its modified constraint can be expressed as the following formula 4:

[0052]

[0053] Rigid dragging refers to changing the rigidity of joint objects, such as dragging a character's arm up and down. Figure 3 As shown, select the modification area S and drag the anchor point μ I and drag the end point μ I ′, the black ellipse in the figure represents the Gaussian ball in the Gaussian splash representation, and the black dotted line represents the modified area S. During the optimization process, the anchor point Gaussian ball will move toward the dragging end point, and the surrounding Gaussian balls will be pulled by the anchor point. At the same time, the loss function constrains the overall structure to remain unchanged, showing a rigid body property. The modification constraint can be expressed as the following formula 5:

[0054]

[0055] Deformation dragging means changing the shape of an object as a whole or in part when changing its position. Figure 4 As shown, select the modification area S and drag the anchor point μ I and drag the end point μ I ′, the black ellipse in the figure represents the Gaussian ball in the Gaussian splash representation, and the black dotted line represents the modified area S. During the optimization process, the anchor point Gaussian ball will move toward the dragging end point, and the surrounding Gaussian balls will be pulled by the anchor point, and the distance from the dragging anchor point μ I The farther the Gaussian ball is, the smaller the pulling degree is. The distance from the drag anchor point μ I The closer the Gaussian ball is, the greater the degree of traction is, causing the overall structure of the object to be deformed to a certain extent. The modification constraint can be expressed as the following formula 6:

[0056]

[0057] Among them, γ is a hyperparameter that controls the degree of deformation, and O is the set of all Gaussian balls whose radius is less than r from the anchor point, which is the following formula 7:

[0058] O={i|‖μ i -μ I ‖2≤r}.

[0059] As well as combination and deletion, it means combining multiple objects or deleting part of the area, and you can directly add the Gaussian sphere set of another object or delete the modified area S.

[0060] In some embodiments, the modification parameters are converted into modification constraints of the target three-dimensional model generation process, including: when the target rendering representation is an implicit rendering representation, obtaining the initial implicit rendering parameters of the initial three-dimensional model; using a large language model to convert the interactive text information into prompt words; the prompt words refer to the vocabulary that can be understood by the three-dimensional model optimization network; based on the initial implicit rendering parameters, the prompt words and the modification area, determining the modification constraints.

[0061] Specifically, a large language model (LLM) refers to a deep learning model trained with a large amount of text data. The model can generate natural language text or understand the meaning of language text. In this embodiment, a large language model is used to convert interactive text information into prompt words that can be understood by the three-dimensional model optimization network. For example, if the interactive text information input by the user is "hold a sword in the left hand", then the prompt words can be "left hand", "hold" and "sword", breaking down the interactive text information into short words or sentences to make it easier for the three-dimensional model optimization network to understand the user's modification intention. Combined with formula 1, the neural network parameters It is the initial implicit rendering parameter Since the input condition information y has changed, the input condition information after simulation modification should be the prompt word y′. In the modification area S, the initial implicit rendering parameter Substituting θ into Formula 1 and the prompt word y′ into y in Formula 1, we can get the optimization gradient under the implicit rendering representation.

[0062] In some embodiments, the target rendering representation includes multiple ones, and the user first performs simulation modification under the explicit rendering representation, and then performs modification under the implicit rendering representation. Therefore, before converting the modification parameters into modification constraints of the target 3D model generation process, it is necessary to first convert the modification constraints under the explicit rendering representation into modification constraints under the implicit rendering representation. The conversion process can be performed according to the following formula:

[0063] Formula 8:

[0064]

[0065] Where C(θ) is a set of predefined rendering perspectives, M is the size of the geometry, and f is the implicit rendering function. is an implicit rendering parameter, g is an explicit rendering function, and θ is an explicit rendering parameter. Through the above formula 8, the modification constraint under the explicit rendering representation can be transformed into the modification constraint under the implicit rendering representation.

[0066] In some embodiments, based on the initial implicit rendering parameters, the prompt word and the modification area, the modification constraints are determined, including: based on the prompt word and the modification area, determining the first implicit rendering parameter corresponding to the modification area; obtaining the second implicit rendering parameter of the maintenance area; the maintenance area is the area in the initial three-dimensional model excluding the modification area; based on the initial implicit rendering parameters, the first implicit rendering parameters and the second implicit rendering parameters, determining the first modification constraint of the modification area and the second modification constraint of the maintenance area.

[0067] Specifically, in the modified area S, the prompt word y′ is substituted into Formula 1 to determine the first implicit rendering parameter corresponding to the modified area S: The modification constraints include modification constraints of the modification area and modification constraints of the maintenance area. Then according to the initial implicit rendering parameters and the first implicit rendering parameter The second implicit rendering parameter of the maintenance area can be calculated Set the initial implicit rendering parameters First implicit rendering parameter and the second implicit rendering parameter Substituting the preset optimization gradient, the first modification constraint of the modified area and the second modification constraint of the maintained area are calculated, so that the modification constraints of the modified area and the maintained area can be adjusted independently, which can more accurately adjust the three-dimensional model results of different areas.

[0068] In some embodiments, the method further includes: obtaining an influence scale factor of the maintenance area; and determining a third modification constraint of the maintenance area based on the second modification constraint and the influence scale factor.

[0069] Specifically, the impact ratio factor is used to characterize the ratio of the impact of the prompt word on the maintenance area to the impact of the prompt word on the modification area. For example, if the impact of the prompt word on the maintenance area is a and the impact of the prompt word on the modification area is b, then the impact ratio factor is In some embodiments, the impact scale factor can be a preset fixed value or can be set by the user. The user can adjust the degree to which the maintenance area follows the modification area by setting different impact scale factors. For example, when the impact scale factor β is less than or equal to 1, while ensuring that the structures of the modification area and the maintenance area match, the shape, texture, and angle of the maintenance area are not changed as much as possible; if the impact scale factor β is 0, the maintenance area will be completely fixed. Through this embodiment, different impact scale factors can be set to control the degree to which the maintenance area is affected by the modification area, so that the three-dimensional model result of the maintenance area can also meet the user's expectations, saving the number of times or time for the user to manually adjust the details of the three-dimensional model.

[0070] S140: Generate a target three-dimensional model desired by the user based on the modification constraints and the initial three-dimensional model.

[0071] Specifically, the modification constraint can be represented by a loss function, and the optimization gradient of the initial 3D model can also be represented by a loss function. By superimposing the modification constraint and the optimization gradient of the initial 3D model, the optimization gradient of the target 3D model can be obtained, that is, the total loss function of the target 3D model can be obtained. According to the total loss function, the target 3D model can be constrained to be modified in the direction of the user's simulated modification, so as to obtain a target 3D model that meets the user's expectations.

[0072] In the technical solution of the present application, an initial three-dimensional model is obtained; the modification parameters of the user for the initial three-dimensional model are obtained, and the modification parameters are the parameters for the user to simulate the modification under the target rendering representation, and the target rendering representation includes at least one of the explicit rendering representation and the invisible rendering representation, so that the user can simulate the modification of the three-dimensional model in a variety of ways; and then the modification parameters are converted into modification constraints for the target three-dimensional model generation process; the modification constraints are used to guide the modification direction of the initial three-dimensional model, so as to constrain the modification effect of the initial three-dimensional model to meet the user's expectations, and to correct unreasonable modifications in the simulated modification; finally, based on the modification constraints and the initial three-dimensional model, the target three-dimensional model expected by the user is generated. Through the above method, not only the generated target three-dimensional model is closer to the user's expectations, but also the modification parameters for the user's simulated modification can be automatically converted into the corresponding modification constraints, which can automatically adjust the initial three-dimensional model, reduce the degree of detail modification by the user, and improve the accuracy of the three-dimensional model generation.

[0073] In some embodiments, based on the modification constraints and the initial three-dimensional model, generating a target three-dimensional model desired by the user includes: obtaining a guidance degree factor corresponding to the modification constraints; and obtaining the target three-dimensional model based on the guidance degree factor, the modification constraints and the initial three-dimensional model.

[0074] Specifically, the guidance degree factor indicates the degree to which the target three-dimensional model is affected by the modification constraint. If the guidance degree factor α is larger, the target three-dimensional model is affected by the modification constraint L. θ The greater the degree of influence; if the guidance degree factor α is smaller, the target 3D model is subject to the modification constraint L θ The smaller the impact, the smaller the degree of influence. For example, the total loss function of the target 3D model is L = L SDS +αL θ , where L SDS is the loss function of the initial 3D model, L θ It is the modification constraint corresponding to the simulation modification. The guidance degree factor α can adjust the influence of the modification constraint on the target three-dimensional model. The guidance degree factor can be set by the user, or the value range can be determined by collecting samples for analysis.

[0075] The electronic equipment of this application is introduced below, such as Figure 5 As shown, the present application provides an electronic device 500, which includes: a processor 510 and a memory 520, the memory 520 is used to store executable instructions of the processor 510; the processor 510 executes the executable instructions to enable the electronic device 500 to implement the three-dimensional model generation method based on interactive editing provided by any embodiment of the present application.

[0076] Specifically, the three-dimensional model generation method based on interactive editing provided by the present application is stored in the memory 520 of the electronic device, and the three-dimensional model generation method based on interactive editing is executed by the processor 510 to constrain the modification effect of the initial three-dimensional model to meet the user's expectations, and to correct unreasonable modifications in the simulated modification, so that the generated target three-dimensional model is not only closer to the user's expectations, but also obtains the modification parameters of the user's simulated modification, which can be automatically converted into corresponding modification constraints. The modification constraints can automatically adjust the initial three-dimensional model, reduce the degree of detail modification performed by the user, and improve the accuracy of the three-dimensional model generation.

[0077] It should be known that the specific implementation content of the electronic device in the present application has been explained in detail in the corresponding method embodiment and will not be repeated here.

[0078] Figure 6 The structure block diagram of a computer system for implementing an electronic device according to an embodiment of the present application is schematically shown.

[0079] It should be noted that Figure 6 The computer system 600 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0080] like Figure 6 As shown, the computer system 600 includes a processor 601, which can be a CPU (Central Processing Unit) or an MCU (Microcontroller Unit). The processor 601 can perform various appropriate actions and processes according to the program stored in the read-only memory 602 (Read-Only Memory, ROM) or the program loaded from the storage part 608 to the random access memory 603 (Random Access Memory, RAM). In the random access memory 603, various programs and data required for system operation are also stored. The processor 601, the read-only memory 602 and the random access memory 603 are connected to each other through a bus 604. The input / output interface 605 (Input / Output interface, i.e., I / O interface) is also connected to the bus 604.

[0081] The following components are connected to the input / output interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a local area network card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed so that a computer program read therefrom is installed into the storage section 608 as needed.

[0082] In particular, according to an embodiment of the present application, the process described in each method flow chart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer readable medium, and the computer program contains a program code for executing the method shown in the flow chart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the processor 601, various functions defined in the system of the present application are executed.

[0083] It should be noted that the computer-readable medium shown in the embodiment of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by an instruction execution system, device or device or used in combination with it. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than computer readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0084] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the above-mentioned module, program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0085] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0086] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation method of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the implementation method according to the present application.

[0087] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary technical means in the art that are not disclosed in the present application.

[0088] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A three-dimensional model generation method based on interactive editing, characterized in that: include: Obtaining an initial three-dimensional model; Acquiring a modification parameter of the initial three-dimensional model by a user, wherein the modification parameter is a parameter simulated and modified by the user under a target rendering representation, wherein the target rendering representation includes at least one of an explicit rendering representation and an invisible rendering representation; Converting the modification parameters into modification constraints for a target three-dimensional model generation process; the modification constraints are used to guide the modification direction of the initial three-dimensional model; Based on the modification constraints and the initial three-dimensional model, a target three-dimensional model desired by a user is generated.

2. The method for generating a three-dimensional model based on interactive editing according to claim 1, characterized in that: The obtaining of the modification parameters of the initial three-dimensional model by the user includes: When the target rendering representation is an explicit rendering representation, obtaining a modified area and modified explicit rendering parameters; the modified area is an area selected by the user when performing simulation modification on the initial three-dimensional model; When the target rendering representation is an implicit rendering representation, the modification area and interactive text information input by the user for the initial three-dimensional model are obtained.

3. The method for generating a three-dimensional model based on interactive editing according to claim 2, characterized in that: The step of converting the modification parameters into modification constraints of a target three-dimensional model generation process includes: When the target rendering representation is an explicit rendering representation, a regularization function and a regularization factor for the current simulation modification are determined according to the modification operation type used by the user; the modification operation type includes: conventional geometric operation, rigid drag, deformation drag, combination and deletion; The modification constraint is determined according to the initial explicit rendering parameters of the initial three-dimensional model, the modified explicit rendering parameters, the regularization function and the regularization factor.

4. The method for generating a three-dimensional model based on interactive editing according to claim 2, characterized in that: The step of converting the modification parameters into modification constraints of a target three-dimensional model generation process includes: When the target rendering representation is an implicit rendering representation, obtaining initial implicit rendering parameters of the initial three-dimensional model; The interactive text information is converted into prompt words by using a large language model; the prompt words refer to words that can be understood by the three-dimensional model optimization network; The modification constraint is determined based on the initial implicit rendering parameter, the prompt word, and the modification area.

5. The method for generating a three-dimensional model based on interactive editing according to claim 4, characterized in that: The determining the modification constraint based on the initial implicit rendering parameter, the prompt word and the modification area includes: Based on the prompt word and the modified area, determining a first implicit rendering parameter corresponding to the modified area; Acquire a second implicit rendering parameter of a maintenance area; the maintenance area is an area in the initial three-dimensional model excluding the modification area; Based on the initial implicit rendering parameter, the first implicit rendering parameter, and the second implicit rendering parameter, a first modification constraint of the modification area and a second modification constraint of the maintenance area are determined.

6. The method for generating a three-dimensional model based on interactive editing according to claim 4, characterized in that: Also includes: Acquire an influence proportional factor of the maintenance area, where the influence proportional factor is used to represent the ratio of the influence degree of the prompt word on the maintenance area to the influence degree of the prompt word on the modification area; Based on the second modification constraint and the impact scaling factor, a third modification constraint of the maintenance region is determined.

7. The method for generating a three-dimensional model based on interactive editing according to claim 1, characterized in that: The step of generating a target three-dimensional model desired by a user based on the modification constraint and the initial three-dimensional model comprises: Obtaining a guidance degree factor corresponding to the modification constraint; The target three-dimensional model is obtained based on the guidance degree factor, the modification constraint and the initial three-dimensional model.

8. The method for generating a three-dimensional model based on interactive editing according to claim 1, characterized in that: The obtaining of the initial three-dimensional model comprises: Get the user's initial input information; The initial three-dimensional model generation process is constrained using a specified optimization algorithm to generate the initial three-dimensional model corresponding to the initial input information.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the three-dimensional model generation method based on interactive editing described in any one of claims 1 to 8 is implemented.

10. An electronic device, characterized in that: include: processor; A memory, configured to store executable instructions of the processor; The processor executes the executable instructions to enable the electronic device to implement the three-dimensional model generation method based on interactive editing as described in any one of claims 1 to 8 above.

Images