A natural language-based intelligent generation method and system for three-dimensional models

Abstract

The present invention discloses a method and system for intelligently generating three-dimensional models based on natural language. The method comprises: receiving and analyzing a natural language description input by a user in real time, extracting key information data from the natural language description based on a three-dimensional model data structure, a denoising symbol set, and a stop word list, and mapping the data into a three-dimensional model parameter set; calling a three-dimensional model generation network corresponding to the scene type described in the natural language description and the three-dimensional model parameter set to generate an initial three-dimensional model; receiving and parsing a model optimization instruction input by the user in real time, and updating the geometric structure, detailed features, and physical properties of the initial three-dimensional model based on the parsing results, generating a dynamic interactive model, and outputting a three-dimensional model file of the dynamic interactive model. By applying the method and system provided by the present invention, the efficiency, accuracy, flexibility, ease of use, and user experience of three-dimensional modeling are improved.

Description

Technical Field

[0001] The present application relates to the fields of computer graphics and image processing and natural language data processing, and more specifically, to a method and system for intelligently generating three-dimensional models based on natural language. Background Art

[0002] Three-dimensional models are widely used in fields such as virtual reality (VR), augmented reality (AR), game development, industrial design, architectural design, and medical simulation. However, traditional 3D modeling methods often rely on specialized software, require specialized modeling skills, and are time-consuming. Furthermore, existing methods for generating 3D models based on 3D scanning or parametric modeling suffer from operational complexity and limited flexibility.

[0003] In recent years, with the rapid development of deep learning and natural language processing (NLP) technologies, generating 3D models from natural language descriptions has become increasingly feasible. However, due to the inability to understand the semantics and context of natural language, 3D model generation relies on specialized technicians. As a result, existing technologies still have limitations in terms of generated model quality, efficiency, and flexibility in natural language understanding. These include missing details in generated 3D models, insufficient understanding of language instructions, and difficulty in dynamically adjusting model characteristics.

[0004] Based on this, it is necessary to introduce a new method and system, which can not only generate high-quality three-dimensional models through simple language descriptions, but also support real-time dynamic adjustment and diversified expression of three-dimensional models according to the user's personalized needs, so as to solve the technical problems existing in the existing technology, such as the inability to understand the semantics and contextual relationship of natural language, the reliance on professional technicians for three-dimensional model generation, the lack of three-dimensional model details, insufficient ability to understand language instructions, and difficulty in dynamically adjusting the model, thereby lowering the threshold for three-dimensional model generation and use, and improving the efficiency, accuracy, flexibility and ease of use of three-dimensional model modeling, as well as user experience. Summary of the Invention

[0005] In response to the technical problems mentioned above, the present invention provides a natural language-based intelligent generation method and system for three-dimensional models. By performing semantic parsing on the natural language description received from the user, an initial three-dimensional model is generated based on a customized three-dimensional model data structure and a three-dimensional model generation network trained according to different scene types, and the initial three-dimensional model is rendered and visualized based on an interactive window. At the same time, based on the interactive window, the geometric structure, detailed features, and physical properties of the initial three-dimensional model are updated in real time according to the model optimization instructions input by the user, a dynamic interactive model is generated, and a three-dimensional model file of the dynamic interactive model is output. This not only realizes the generation of high-quality three-dimensional models through simple language descriptions, but also realizes the real-time dynamic adjustment and diversified expression of the three-dimensional model according to the user's personalized needs. It solves the technical problems existing in the prior art, such as the inability to understand the semantics and contextual relationships of natural language, the reliance on professional technicians for three-dimensional model generation, the lack of three-dimensional model details, insufficient ability to understand language instructions, and difficulty in dynamically adjusting the model. It lowers the threshold for three-dimensional model generation and use, while improving the efficiency, accuracy, flexibility, ease of use, and user experience of three-dimensional model modeling.

[0006] The present invention provides a method for intelligently generating a three-dimensional model based on natural language, the method comprising: S1, defining a three-dimensional model data structure and a denoising symbol set, defining and constructing a stop word list and a common sense knowledge base according to a scene type; S2, receiving and parsing a natural language description input by a user in real time, parsing the natural language description based on the three-dimensional model data structure, the denoising symbol set and the stop word list, extracting key information data for generating a three-dimensional model based on the parsing result, and mapping the key information data into a three-dimensional model parameter set; S3, training and generating a three-dimensional model generation network based on the scene type and the data structure of the three-dimensional model network, calling the three-dimensional model generation network corresponding to the scene type described in the natural language according to the scene type described in the natural language and the three-dimensional model parameter set to generate an initial three-dimensional model, and rendering and visually displaying the initial three-dimensional model in real time based on a three-dimensional model rendering engine; S4, receiving and parsing a model optimization instruction input by the user in real time based on the initial three-dimensional model, updating the geometric structure, detailed features and physical properties of the initial three-dimensional model according to the parsing result, generating a dynamic interactive model, and generating and outputting a three-dimensional model file of the dynamic interactive model according to a user request;

[0007] Among them, the noise symbol set is used to store punctuation marks in natural language, the stop word list is used to store the segmented words that constitute natural language in the form of strings, and the segmented words include pronouns, conjunctions, prepositions, quantifiers, modal particles, high-frequency conjunctions and common words. The common sense knowledge base is used to store the default rules of the spatial position relationship between entities in each scene; the three-dimensional model generation network is a network structure of a three-dimensional model that is pre-defined and trained according to the scene type, as well as data storage space and data transmission channels, which are used to store and transmit key information data of the three-dimensional model; the data structure of the three-dimensional model network includes scene type, entity object ID, name of entity object, network hierarchy, geometric data, texture data and material data, and the network hierarchy corresponds to the entity relationship of each entity object.

[0008] Preferably, in step S1, the three-dimensional model data structure includes scene type, entity object, entity relationship, attribute parameters, detail features, and physical properties; wherein the entity relationship includes entity level and topological structure; the attribute parameters include spatial position, shape parameters, size parameters, material parameters, color, height, length, width, height and thickness; the detail features include texture, material, smoothness and concave-convex features; and the physical properties include light reflection characteristics, hardness characteristics and transparency.

[0009] Preferably, in step S2, the step of parsing the natural language description based on the three-dimensional model data structure, the denoising symbol set and the stop word list, and extracting and generating key information data of the three-dimensional model according to the parsing result further includes: S21, obtaining text: parsing the user's natural language description, determining the type of the natural language description, and performing text conversion processing on the natural language description based on the parsing result and the type of the natural language description based on the three-dimensional model data structure to obtain the text of the natural language description; S22, text preprocessing: performing word segmentation and denoising on the text of the natural language description based on the denoising symbol set and the stop word list Processing, removing punctuation marks and word segments in the text to obtain a preprocessed text; S23, syntax parsing: constructing a syntax tree according to the preprocessed text, and determining the entity objects, scene trunks and components of the natural language description based on the constructed syntax tree and the preprocessed text, and marking the parts of speech constituting the natural language description; S24, semantic parsing: performing semantic extraction based on the entity objects, scene trunks and components of the natural language description, as well as the syntax tree and the preprocessed text, identifying and determining the scene type of the natural language description, attribute parameters of the entity objects, and entity relationships between the entity objects, generating the key information data, and mapping it into a three-dimensional model parameter set;

[0010] Among them, the types of natural language descriptions include text and voice; the text includes words, punctuation marks, conjunctions and repeated words; the key information data includes: the scene type, entity objects, attribute parameters of entity objects, and entity relationships between entity objects described in the natural language.

[0011] Preferably, the step S21 also includes a speech-to-text conversion and analysis step, specifically: if the type of the natural language description is text, then according to the analysis result of the natural language description, the text of the natural language description is obtained; if the type of the natural language description is speech, then according to the analysis result of the natural language description, the speech corresponding to the natural language description is converted to obtain the text of the natural language description.

[0012] Preferably, in step S22, the text preprocessing step further includes: S22-1, denoising processing: decomposing the text of the natural language description into words, phrases, and punctuation marks, and converting them into strings respectively, traversing each converted string based on the denoising symbol set, if there is a string in the converted string that matches the punctuation mark in the denoising symbol set, then deleting the matching string from the string corresponding to the natural language description to generate a denoised string; S22-2, stop word filtering: based on the stop word list, traversing the denoised string, if there is a string in the denoised string that matches the word in the stop word list, then deleting the string that matches the word in the stop word list from the denoised string to obtain the preprocessed text.

[0013] Preferably, in step S23, the grammatical parsing step also includes: S23-1, constructing a grammatical tree: constructing a grammatical tree according to the preprocessed text, and marking the parts of speech of words and phrases in the preprocessed text based on the grammatical tree, determining the scene trunk of the natural language description, and the corresponding components of each scene trunk, wherein the parts of speech include: core verbs and object nouns; S23-2, entity recognition: identifying the scene type corresponding to the natural language description according to the object nouns in the preprocessed text, and determining the entity object according to the scene trunk of the natural language description and the corresponding components of each scene trunk, and marking the parts of speech constituting the natural language description.

[0014] Preferably, in step S24, the semantic parsing step further includes: S24-1, attribute extraction: extracting attribute data of each entity object corresponding to the natural language description according to the name of the entity object, the attribute data including attribute parameters, detailed features and physical properties; S24-2, relationship identification: determining the entity relationship between each entity object according to the attribute parameters of each entity object corresponding to the natural language description, if there is a null value in the attribute parameters of the entity object, then calling the common sense knowledge base corresponding to the natural language description according to the scene type corresponding to the natural language description and the scene type of the common sense knowledge base, determining each entity object corresponding to the natural language description. Entity relationships between entity objects; S24-3, parameter generation: generating the key information data according to the scene type, entity objects, attribute data of entity objects, and entity relationships between entity objects described in the natural language, and mapping the key information data to the three-dimensional model parameter set according to the scene type, name and attributes of the entity objects; wherein the three-dimensional model parameter set is an intermediate parameter set for automatically converting the key information data described in the natural language into a three-dimensional model, and based on the scene type, name and attributes of the entity objects, the entity relationships and attribute data between the entity objects are hierarchically mapped in the order of spatial position, shape, material and color.

[0015] Preferably, in step S3, the step of training and generating a three-dimensional model generation network according to the scene type and the data structure of the three-dimensional model network further includes: S311, data processing: acquiring a data set of natural language descriptions corresponding to each scene type, and performing network standard conversion and feature extraction according to each scene type based on the acquired data set to obtain three-dimensional model generation network data and network feature data in a unified data format, as well as three-dimensional model network features; S312, network model training: generating an initial generation network model based on the three-dimensional model generation network data and the three-dimensional model network features; S313, network model evaluation: optimizing the initial generation network model based on a loss function and the network feature data to generate the three-dimensional model generation network;

[0016] The formula of the loss function is:

[0017]

[0018] Where L is the difference between the predicted result of the generated network model and the actual result, n is the number of network features of the 3D model, n is an integer greater than 1, and x i represents the network features of the i-th real 3D model, Represents the i-th 3D model network feature predicted by the generative network model, It represents the square of the L2 norm of the vector, which is used to calculate the square of the Euclidean distance between the true value and the predicted value of the generative network model. λ is a hyperparameter that controls the strength of regularization, and R(θ) is the regularization term with respect to the model parameter θ.

[0019] Preferably, in step S3, the step of calling a three-dimensional model generation network corresponding to the scene type described in the natural language and the three-dimensional model parameter set to generate an initial three-dimensional model further includes: S321, global structure generation: inputting the three-dimensional model parameter set into the three-dimensional model generation network corresponding to the scene type described in the natural language, and determining the coordinates and geometric shape of the spatial position of each entity object according to the entity level of the entity object corresponding to the natural language description and the entity relationship between the entity objects in accordance with the entity level from high to low, the name of the entity object and the attribute parameters, to generate the global geometric structure of the initial three-dimensional model; S322, local detail generation: based on the global structure of the initial three-dimensional model, using the detail features in the three-dimensional model parameter set to add the texture, material, smoothness and concave-convex features of each entity object corresponding to the natural language description, to generate a local enhanced geometric structure of the initial three-dimensional model; S322, physical property enhancement: based on the local detail enhancement structure of the initial three-dimensional model, using the physical properties in the three-dimensional model parameter set to add the illumination reflection characteristics, hardness characteristics and transparency of each entity object corresponding to the natural language description, to generate the initial three-dimensional model.

[0020] Accordingly, the present invention also provides a natural language-based intelligent generation system for three-dimensional models, the system comprising an initialization module, a data processing module, an initial model construction module, and a dynamic model interaction module;

[0021] The initialization module is used to define a 3D model data structure and a denoising symbol set, and to define and construct a stop word list and a common sense knowledge base according to the scene type. The data processing module is used to receive and parse the natural language description input by the user in real time, and to parse the natural language description based on the 3D model data structure, the denoising symbol set and the stop word list, extract key information data for generating the 3D model based on the parsing results, and map the key information data into a 3D model parameter set. The initial model construction module is used to train and generate a 3D model generation network based on the scene type and the data structure of the 3D model network, call the 3D model generation network corresponding to the scene type described in the natural language and the 3D model parameter set to generate an initial 3D model based on the scene type described in the natural language and the 3D model parameter set, and render and visualize the initial 3D model in real time based on the 3D model rendering engine. The dynamic model interaction module is used to receive and parse the model optimization instructions input by the user in real time based on the initial 3D model, and update the geometric structure, detailed features and physical properties of the initial 3D model based on the parsing results to generate a dynamic interaction model, and generate and output a 3D model file of the dynamic interaction model based on the user request.

[0022] Among them, the noise symbol set is used to store punctuation marks in natural language, the stop word list is used to store the segmented words that constitute natural language in the form of strings, and the segmented words include pronouns, conjunctions, prepositions, quantifiers, modal particles, high-frequency conjunctions and common words. The common sense knowledge base is used to store the default rules of the spatial position relationship between entities in each scene; the three-dimensional model generation network is a network structure of a three-dimensional model that is pre-defined and trained according to the scene type, as well as data storage space and data transmission channels, which are used to store and transmit key information data of the three-dimensional model; the data structure of the three-dimensional model network includes scene type, entity object ID, name of entity object, network hierarchy, geometric data, texture data and material data, and the network hierarchy corresponds to the entity relationship of each entity object.

[0023] The present invention applies the above technical solution, performs semantic parsing on the natural language description received from the user input, generates an initial three-dimensional model based on a customized three-dimensional model data structure and a three-dimensional model generation network trained according to different scene types, and renders and visualizes the initial three-dimensional model based on an interactive window. At the same time, based on the interactive window, the geometric structure, detailed features and physical properties of the initial three-dimensional model are updated in real time according to the model optimization instructions input by the user, a dynamic interactive model is generated, and a three-dimensional model file of the dynamic interactive model is output. This not only realizes the generation of high-quality three-dimensional models through simple language descriptions, but also realizes the dynamic adjustment and diversified expression of the three-dimensional model in real time according to the user's personalized needs. It solves the technical problems existing in the prior art of the inability to understand the semantics and contextual relationship of natural language, the reliance on professional technicians for three-dimensional model generation, the lack of three-dimensional model details, insufficient ability to understand language instructions, and difficulty in dynamically adjusting the model, thereby lowering the threshold for three-dimensional model generation and use, while improving the efficiency, accuracy, flexibility, ease of use and user experience of three-dimensional model modeling. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 A schematic diagram showing a flow chart of a natural language-based intelligent generation method for a three-dimensional model proposed in an embodiment of the present invention;

[0026] Figure 2 A schematic structural diagram of a natural language-based intelligent generation system for three-dimensional models proposed in an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0028] The present invention provides a method for intelligently generating a three-dimensional model based on natural language. Figure 1 As shown, the method includes the following steps:

[0029] S1, defines the 3D model data structure and the denoising symbol set, and defines and builds a stop word list and common sense knowledge base according to the scene type.

[0030] Among them, the noise symbol set is used to store punctuation marks in natural language, the stop word list is used to store the segmented words that constitute natural language in the form of strings, and the segmented words include pronouns, conjunctions, prepositions, quantifiers, modal particles, high-frequency conjunctions and common words; the common sense knowledge base is used to store the default rules of the spatial position relationship between entities in each scene.

[0031] In this embodiment, in step S1, the three-dimensional model data structure includes scene type, entity object, entity relationship, attribute parameters, detail features, and physical attributes;

[0032] in,

[0033] The entity relationship includes entity level and topology;

[0034] The attribute parameters include spatial position, shape parameters, size parameters, material parameters, color, height, length, width, height and thickness;

[0035] The detailed features include texture, material, smoothness and concave-convex features;

[0036] The physical properties include light reflection properties, hardness properties and transparency.

[0037] S2, receiving and analyzing the natural language description input by the user in real time, and parsing the natural language description based on the three-dimensional model data structure, the denoising symbol set and the stop word list, extracting and generating key information data of the three-dimensional model according to the parsing results, and mapping the key information data into a three-dimensional model parameter set.

[0038] In this embodiment, in step S2, the step of parsing the natural language description based on the three-dimensional model data structure, the denoising symbol set, and the stop word list, and extracting and generating key information data of the three-dimensional model based on the parsing result further includes:

[0039] S21, obtaining text: parsing the user's natural language description to determine the type of the natural language description, and performing text conversion processing on the natural language description based on the parsing result and the type of the natural language description based on the three-dimensional model data structure to obtain the text of the natural language description;

[0040] S22, text preprocessing: performing word segmentation and denoising processing on the text described in the natural language based on the denoising symbol set and the stop word list, removing punctuation marks and word segmentation in the text, and obtaining a preprocessed text;

[0041] S23, syntax parsing: constructing a syntax tree according to the preprocessed text, and determining the entity objects, scene trunks, and components of the natural language description based on the constructed syntax tree and the preprocessed text, and marking the parts of speech constituting the natural language description;

[0042] S24, semantic parsing: performing semantic extraction based on the entity objects, scene backbones, and components described in the natural language, as well as the syntax tree and the preprocessed text, identifying and determining the scene type described in the natural language, attribute parameters of the entity objects, and entity relationships between the entity objects, generating the key information data, and mapping it into a set of 3D model parameters;

[0043] in,

[0044] The type of the natural language description, including text and voice;

[0045] The text includes words, punctuation marks, conjunctions and repeated words;

[0046] The key information data includes: the scene type described in the natural language, the entity object, the attribute parameters of the entity object, and the entity relationship between the entity objects.

[0047] In this embodiment, the step S21 further includes the step of speech-to-text conversion and analysis, specifically:

[0048] If the type of the natural language description is text, obtaining the text of the natural language description according to the parsing result of the natural language description;

[0049] If the type of the natural language description is speech, the speech corresponding to the natural language description is converted according to the parsing result of the natural language description to obtain the text of the natural language description.

[0050] In this embodiment, in step S22, the text preprocessing step further includes:

[0051] S22-1, denoising: decomposing the text of the natural language description into words, phrases, and punctuation marks, and converting each of the words, phrases, and punctuation marks into character strings, traversing each of the converted character strings based on the denoising symbol set, and if any of the converted character strings contains a character string that matches a punctuation mark in the denoising symbol set, deleting the matching character string from the character string corresponding to the natural language description, thereby generating a denoised character string;

[0052] This step removes punctuation marks from the text described in the natural language;

[0053] S22-2, stop word filtering: Based on the stop word list, the denoised character string is traversed. If there is a character string in the denoised character string that matches the word in the stop word list, the character string that matches the word in the stop word list is deleted from the denoised character string to obtain the preprocessed text.

[0054] This step removes the segmented words in the text of the natural language description, including pronouns, conjunctions, prepositions, quantifiers, modal particles, high-frequency conjunctions and common words.

[0055] In this embodiment, in step S23, the syntax parsing step further includes:

[0056] S23-1, constructing a syntax tree: constructing a syntax tree based on the preprocessed text, and annotating the parts of speech of words and phrases in the preprocessed text based on the syntax tree, determining the scene trunk of the natural language description, and the components corresponding to each scene trunk,

[0057] Wherein, the parts of speech include: core verbs and object nouns;

[0058] S23-2, entity recognition: Identify the scene type corresponding to the natural language description based on the object noun in the preprocessed text, and determine the entity object based on the scene trunk of the natural language description and the components corresponding to each scene trunk, and mark the parts of speech that constitute the natural language description.

[0059] In this embodiment, in step S24, the semantic parsing step further includes:

[0060] S24-1, attribute extraction: extracting attribute data of each entity object corresponding to the natural language description based on the name of the entity object, the attribute data including attribute parameters, detailed features and physical properties;

[0061] S24-2, Relationship Identification: Determine the entity relationships between the entity objects based on the attribute parameters of the entity objects corresponding to the natural language description. If any of the attribute parameters of the entity objects contain null values, call the common sense knowledge base corresponding to the natural language description based on the scene type corresponding to the natural language description and the scene type of the common sense knowledge base to determine the entity relationships between the entity objects corresponding to the natural language description.

[0062] S24-3, parameter generation: generating the key information data based on the scene type, entity objects, attribute data of the entity objects, and entity relationships between the entity objects described in the natural language, and mapping the key information data into the three-dimensional model parameter set based on the scene type, name and attribute of the entity objects;

[0063] in,

[0064] The three-dimensional model parameter set automatically converts key information data described in natural language into an intermediate parameter set for generating a three-dimensional model. Based on the scene type, the name and attributes of the entity object, the entity relationship and attribute data between each entity object are hierarchically mapped in the order of spatial position, shape, material and color.

[0065] It should be noted that this step provided by the present invention can not only parse the natural language description input by the user, but also extract the key information required for generating the three-dimensional model, such as shape, size, material, texture, detail characteristics, etc.

[0066] It can also support multi-language input and complex description parsing, such as "generate a round dining table with wood grain, about 1 meter high and 80 centimeters in diameter."

[0067] S3, training and generating a three-dimensional model generation network according to the scene type and the data structure of the three-dimensional model network, calling the three-dimensional model generation network corresponding to the scene type described in the natural language according to the scene type described in the natural language and the three-dimensional model parameter set to generate an initial three-dimensional model, and performing real-time rendering and visualization of the initial three-dimensional model based on the three-dimensional model rendering engine.

[0068] in,

[0069] The three-dimensional model generation network is a network structure of a three-dimensional model that is pre-defined and trained according to the scene type, as well as a data storage space and a data transmission channel for storing and transmitting key information data of the three-dimensional model;

[0070] The data structure of the three-dimensional model network includes scene type, entity object ID, entity object name, network hierarchy, geometry data, texture data and material data, and the network hierarchy corresponds to the entity relationship of each entity object.

[0071] In this embodiment, in step S3, the step of training and generating a 3D model generation network according to the scene type and the data structure of the 3D model network further includes:

[0072] S311, data processing: obtaining a dataset of natural language descriptions corresponding to each scene type, and performing network standard conversion and feature extraction based on the obtained dataset according to each scene type to obtain 3D model generation network data and network feature data in a unified data format, as well as 3D model network features;

[0073] S312, network model training: generating an initial generation network model based on the three-dimensional model generation network data and the three-dimensional model network features;

[0074] S313, network model evaluation: optimizing the initial generation network model based on the loss function and the network feature data to generate the three-dimensional model generation network;

[0075] The formula of the loss function is:

[0076]

[0077] Where L is the difference between the predicted result of the generated network model and the actual result, n is the number of network features of the 3D model, n is an integer greater than 1, and x i represents the network features of the i-th real 3D model, Represents the i-th 3D model network feature predicted by the generative network model, Represents the square of the L2 norm of the vector, which is used to calculate the square of the Euclidean distance between the true value and the predicted value of the generative network model. λ is a hyperparameter that controls the strength of regularization, and R(θ) is the regularization term about the model parameter θ.

[0078] The square of the L2 norm of the vector is used to calculate the square of the Euclidean distance between the true value and the predicted value of the generated network model to measure the degree of difference between the two.

[0079] In this embodiment, in step S3, the step of calling a 3D model generation network corresponding to the scene type described in the natural language and the 3D model parameter set to generate an initial 3D model further includes:

[0080] S321, global structure generation: Inputting the 3D model parameter set into the 3D model generation network corresponding to the scene type described in the natural language, determining the spatial coordinates and geometric shapes of each entity object according to the entity levels of the entity objects corresponding to the natural language description and the entity relationships between the entity objects, in descending order of entity levels, entity object names, and attribute parameters, to generate an initial global geometric structure of the 3D model;

[0081] S322, local detail generation: Based on the global structure of the initial three-dimensional model, the texture, material, smoothness, and concave-convex features of the physical objects corresponding to the natural language descriptions are added using detail features in the three-dimensional model parameter set to generate a local enhanced geometric structure of the initial three-dimensional model;

[0082] S322, physical property enhancement: Based on the local detail enhancement structure of the initial three-dimensional model, the physical properties in the three-dimensional model parameter set are used to add the lighting reflection characteristics, hardness characteristics and transparency of each entity object corresponding to the natural language description to generate an initial three-dimensional model.

[0083] S4, based on the initial three-dimensional model, receives and parses the model optimization instructions input by the user in real time, and updates the geometric structure, detail features and physical properties of the initial three-dimensional model according to the parsing results, generates a dynamic interactive model, and generates and outputs the three-dimensional model file of the dynamic interactive model according to the user's request.

[0084] In order to enable those skilled in the art to more accurately understand the above technical solution provided by the present invention, examples are now given to further supplement the above technical solution.

[0085] Step 1: The user's natural language description received is "generate a living room scene, including a sofa, a coffee table, and a carpet." At this time, the modeling process is started to record the user's input requirement description.

[0086] Step 2. Parse the natural language description, extract key information related to 3D model generation, and generate key information data;

[0087] The steps of natural language parsing include:

[0088] 2.1 Parse the natural language description and distinguish the input types

[0089] Text parsing: Directly processes the text description entered by the user, such as "generate a living room scene, including a sofa, coffee table, and carpet."

[0090] Speech analysis: Convert user speech into text through automatic speech recognition (ASR) technology for subsequent processing.

[0091] 2.2 Text Preprocessing

[0092] Denoising: Remove unnecessary symbols or repeated words from user input

[0093] Word segmentation: Break the input text into words or phrases. For example, ["generate", "a", "living room", "scene", ",", "include", "sofa", ",", "coffee table", "and", "carpet", "."]

[0094] Stop word filtering: remove irrelevant words. For example, ["generate", "living room", "scene", "sofa", "coffee table", "carpet"]

[0095] 2.3 Syntax parsing and extracting syntactic structure

[0096] Use dependency parsing tools (such as SpaCy or Stanford NLP) to build a grammatical tree of the input text, determine the backbone of the sentence, and determine the main structure of the sentence.

[0097] The grammar parsing result is:

[0098] Scene backbone: living room;

[0099] Components: sofa, coffee table, carpet.

[0100] 2.4 Perform semantic extraction to identify entities, attributes, and relationships related to 3D modeling;

[0101] Entity recognition: Identify scene types and objects ("living room", "sofa", "coffee table", "carpet").

[0102] Attribute extraction: If the entity object has specific attribute parameters, the attributes of the object are extracted or set by default.

[0103] Relationship identification: If entity relationships have been determined between the entity objects, the relationship between the objects or the default conventional layout is obtained and determined.

[0104] Construct a coordinate system based on the name, attributes and entity relationship of the entity object, and determine the specific position coordinates of each entity object in this coordinate system.

[0105] 2.5 Map the semantic information into the parameter set required for 3D model generation, including shape parameters, size parameters, material parameters, etc.

[0106] Semantic information includes the name of the object, descriptions of its attributes, descriptions of the relationships between its parts, and descriptions related to the scene. This information can comprehensively describe the characteristics and details of an object or scene, providing a basis for subsequent 3D model generation.

[0107] The mapping of semantic information is based on the semantic parsing algorithm of natural language processing, combined with the pre-trained model generation network, and realizes the automatic conversion of natural language to three-dimensional modeling parameters through semantic layered mapping of parameters such as shape and material.

[0108] The parameter set required for 3D model generation is an intermediate value, which is designed to meet the technical requirements of 3D model generation. The key information related to 3D model generation is the progressive relationship from original information to specific data that can be used for model generation, providing direct input for the generation network.

[0109] That is:

[0110]

[0111]

[0112] Step 3. Based on the key information, the 3D model generation network is called to generate an initial 3D model;

[0113] The 3D model generation network is a deep learning-based generative model developed by combining diffusion models and generative adversarial networks (GANs). Its core function is to generate a consistent 3D geometric model from input semantic parameters. The pre-trained generative network is called through an API and fed with a parsed set of parameters (such as shape, size, and material). The generative network decodes the high-dimensional semantic embeddings layer by layer, mapping them to a 3D geometric space and ultimately outputting the initial model.

[0114] The generative network is based on a diffusion model or a generative adversarial network (GAN), and gradually generates a three-dimensional model through a multi-level generation strategy, including:

[0115] The multi-stage generation strategy is an important part of the 3D model generation network. Through step-by-step generation and optimization, it ensures that the generated 3D model has high quality and high level of detail. It is mainly divided into the following steps:

[0116] a. Global geometry generation stage

[0117] Establish the basic outline and spatial distribution framework of the 3D model. Generate the global geometric shape based on the input semantic information. Determine the basic parameters (entity name, attributes) and spatial location of the model.

[0118] This is achieved by using diffusion models or generative adversarial networks (GANs) to generate low-resolution global geometric structures from high-dimensional semantic embeddings.

[0119] The final output is a low-resolution 3D model that contains basic shapes but lacks details.

[0120] b. Local detail generation stage

[0121] Enrich your model's surface details to enhance realism. Add detailed features, including textures, bumps, and surface smoothness. Detail materials (such as wood grain and metallic sheen) and colors.

[0122] This is achieved using texture mapping and a deep learning-based detail generation module:

[0123] Texture generation: Generate high-resolution surface textures based on input parameters.

[0124] Local optimization: Adding detailed features to complex surfaces (such as carvings and decorations).

[0125] Finally, a medium-resolution 3D model with rich surface features is output.

[0126] c. Physical property enhancement stage

[0127] Give models physical properties to enhance their practical usability and realism. Add lighting reflection properties such as metallic gloss and transparency.

[0128] This is achieved by using physically based rendering (PBR) technology and deep learning methods to give the model material and lighting characteristics.

[0129] The final output is a high-resolution 3D model with real materials and physical properties.

[0130] d. Integrity verification phase

[0131] Ensure that the generated 3D model meets quality standards in terms of geometry and properties. Check geometric integrity. Verify the consistency of detailed features and physical parameters.

[0132] This can be achieved by using automated detection algorithms, geometry repair tools (such as Meshlab), or model verification networks (deep learning).

[0133] The final output is a complete 3D model that has passed quality checks and is ready for export or further adjustments.

[0134] Generate scene structure:

[0135] 3.1 Generate the global geometric structure of the 3D model based on semantic information;

[0136] Global layout generation: layout the spatial positions of sofas, coffee tables, and carpets.

[0137] 3.2 Adding detailed features, including local textures and complex surface properties;

[0138] Sofa details

[0139] Material: cloth or leather.

[0140] Texture: Fabric can be smooth or have a simple weave; leather has a smooth surface that may have fine pores or a faux leather texture.

[0141] Coffee table details

[0142] Material: wood or glass.

[0143] Grain: Wood has a natural grain; glass is completely smooth and may be transparent or translucent.

[0144] Carpet details

[0145] Material: wool or chemical fiber.

[0146] Texture: Wool carpets are generally fluffier and have a soft feel; synthetic carpets are smoother, easier to clean, and may mimic the look of other materials.

[0147] 3.3 Verify the integrity of the generated results.

[0148] Ensure that the generated model is complete and consistent in both geometric structure and visual appearance.

[0149] Generate the initial model kitchen_model = generate_3d_scene(parameters)

[0150] Generate a 3D scene based on the parsed semantic parameters:

[0151] The parameters are:

[0152]

[0153]

[0154]

[0155] Step 4. Post-process and optimize the initial 3D model and generate a dynamic interactive model. The optimization includes geometric repair, texture enhancement, and physical property assignment.

[0156] Technical details in post-processing optimization, the goal of geometry repair is to ensure that the model does not have errors during rendering or use.

[0157] Non-manifold geometry repair;

[0158] Repair of duplicate and broken surfaces.

[0159] Different from the global geometry in step 3, the model geometry is the core of the post-processing optimization phase. It focuses on the integrity and topology of the model and is the target of post-generation quality optimization. This ensures the functionality, rendering quality, and geometric accuracy of the generated model in subsequent use.

[0160] Global geometry is the basic outline of a 3D model, focusing on its overall shape, size, and spatial distribution. It is the rough stage of model generation and primarily provides a framework and reference for adding detailed features and physical properties later on.

[0161] The post-processing optimization step comprises:

[0162] 1) Topological optimization of the model geometry, including repairing non-manifold geometry, duplicate faces, and broken faces;

[0163] The main contents of topology optimization are the following three

[0164] Fixing non-manifold geometry:

[0165] Non-manifold geometry refers to geometric shapes with abnormal topology (such as those that share edges with more than two faces).

[0166] Non-manifold structures may cause rendering errors or prevent the physics simulation from being applied.

[0167] Fix duplicate faces:

[0168] Detect and delete redundant faces to avoid redundant calculation and data storage problems of the model.

[0169] Repair the broken surface:

[0170] Repair open areas in the model (such as holes or unclosed edges) to ensure that the geometry is completely enclosed.

[0171] This is achieved by using geometric analysis algorithms to detect non-manifold structures and broken surfaces, and then reconnecting the surfaces using topology-based automatic repair methods. Automated software can be used to detect and repair non-manifold geometry, or dedicated modeling tools can be used to manually repair broken surfaces.

[0172] 2) Optimize the smoothness of the model surface and reduce noise and defects;

[0173] By optimizing surface quality, the visual effect and application value of the model can be improved.

[0174] This process primarily removes unnecessary small bumps or sharp corners generated during the generation process, making the surface smoother. It also repairs surface irregularities caused by generation errors, ensuring that the surface appears continuous both visually and tactilely.

[0175] Implementation:

[0176] A curvature-optimized smoothing algorithm uses a deep learning model to avoid defect detection and repair.

[0177] Use 3D modeling tools to automatically or manually smooth surfaces.

[0178] 3) Add or adjust the physical properties of the model, including light reflection characteristics, hardness, and transparency.

[0179] Give your models realistic materials and optical properties, making them suitable for rendering, physics simulation, and other real-world applications.

[0180] Light reflection characteristics: simulate optical effects in the real world (such as specular reflection, diffuse reflection)

[0181] Hardness property: defines the rigidity or flexibility of a material, such as the brittleness of glass or the strength of metal.

[0182] Transparency: Simulates the visual effects of transparent or translucent materials, such as glass and liquid.

[0183] Implementation:

[0184] Physically Based Rendering (PBR) technology: defines optical properties such as reflectivity, roughness, metallicity, etc.

[0185] Material Mapping: Use texture maps (color maps, normal maps) to define the surface characteristics of a material.

[0186] Transparency adjustment: Set the material's alpha value (transparency parameter) to achieve transparent or semi-transparent effects.

[0187] Supports users to dynamically adjust the 3D model through multiple rounds of language input. The dynamic adjustment includes:

[0188] 1) Receive user input and parse language instructions into executable adjustment commands.

[0189] (1) User adds description: "Change the carpet to gray"

[0190] (2) User adds description: "Add a pot of green plants to the coffee table"

[0191] 2) Adjust the shape, size, color and texture of the model, and modify the model's geometric and material properties in real time based on the analysis results.

[0192] "Change the carpet to gray" parses dynamic input and updates the carpet properties:

[0193]

[0194]

[0195] 3) Add or delete local structures of the model, support users to dynamically add or delete certain parts of the model, and optimize the structure of the model.

[0196] The system parses and executes the scene update for "adding a pot of green plants to the coffee table":

[0197]

[0198]

[0199] 4. Real-time rendering of model update results. Every time the user enters an adjustment command, the model changes are presented in a timely manner to enhance the interactive experience.

[0200] 5. Supports multiple rounds of interaction and iterative adjustment, allowing users to make multiple iterative modifications when adjusting the model until a model that meets the requirements is generated.

[0201] Step 5. Output the final 3D model file, allowing users to select the required file format and resolution.

[0202] The 3D model can be exported in multiple file formats, including but not limited to OBJ, STL, and FBX file formats, and supports the following functions:

[0203] 1. User-defined export options, such as whether to include materials and textures:

[0204] Format: OBJ, FBX;

[0205] Whether to include material: yes;

[0206] Resolution: Export high-resolution versions for rendering and low-resolution versions for real-time presentation.

[0207] 2. Generate different resolution versions of the model on demand, including high-resolution and lightweight low-polygon models:

[0208] High-resolution files:

[0209] living_room_scene_with_gray_carpet_and_plant_high_res.obj

[0210] Low-resolution files:

[0211] living_room_scene_with_gray_carpet_and_plant_low_poly.fbx

[0212] 3. Export scene model:

[0213] export_model(kitchen_model,format="OBJ",resolution="high",filename="living_room_scene_with_gray_carpet_and_plant_high_res.obj")

[0214] export_model(kitchen_model,format="FBX",resolution="low",filename="living_room_scene_with_gray_carpet_and_plant_low_poly.fbx")

[0215] Export the 3D scene model to the specified format and resolution.

[0216]

[0217] Example 1: Generate a static model.

[0218] The user enters the description "Generate a red cylindrical vase with a height of 30 cm and a diameter of 10 cm." The system parses the natural language description, extracts model features, and uses the generative network to generate a 3D model of the vase. Once completed, it is exported to OBJ format for the user to download.

[0219] 1. User input: "Generate a red cylindrical vase with a height of 30 cm and a diameter of 10 cm"

[0220] 2. Extract parameters after parsing:

[0221] parameters={

[0222] "shape":"cylinder",

[0223] "dimensions":{"height":30,"diameter":10},

[0224] "material":{"color":"red"}

[0225] }

[0226] 3. Call the generated network:

[0227] model=generate_3d_model(parameters).

[0228] 4. Perform optimization processing:

[0229] optimized_model=optimize_model(model).

[0230] 5. Export the model:

[0231] export_model(optimized_model, format="OBJ", filename="red_vase.obj).

[0232] Example 2: Generating a dynamic interaction model.

[0233] The user enters the description "Create a blue sports car." After the initial model is generated, the user adds descriptions such as "Change the wheels to black" or "Add a sunroof." The model is updated in real time and exported in FBX format.

[0234] Step 1: Receive user input

[0235] Initial input: "Generate a blue sports car"

[0236] Record the initial description and start the modeling process.

[0237] Step 2: Parse the natural language description.

[0238] Syntax analysis:

[0239] Subject: sports car;

[0240] Attributes: blue;

[0241] Semantic Mapping:

[0242] Shape parameters: {"type":"car"};

[0243] Material parameters: {"color":"blue"};

[0244] Output parameter set:

[0245] parameters={

[0246] "type":"car",

[0247] "color":"blue"

[0248] }

[0249] Step 3: Call the 3D model generation network.

[0250] Initial generation:

[0251] Call a pre-trained model based on a generative adversarial network (GAN).

[0252] Global features: Build the geometry of the car body (overall appearance, proportions).

[0253] Detail generation: adding local features (wheels, windows).

[0254] Integrity verification: ensuring the geometric consistency of the vehicle body.

[0255] Output the initial model:

[0256] car_model=generate_3d_model(parameters)

[0257] Step 4: Dynamic interactive adjustment.

[0258] User added description: "Change the wheels to black".

[0259] Parse dynamic input and update the model:

[0260] updated_parameters={

[0261] "wheels":{"color":"black"}

[0262] }

[0263] car_model=update_3d_model(car_model,updated_parameters)

[0264] User added description: "Add a roof sunroof"

[0265] Parse and perform model updates:

[0266] updated_parameters={

[0267] "roof":{"feature":"sunroof"}

[0268] }

[0269] car_model=update_3d_model(car_model,updated_parameters)

[0270] Render the dynamic model update results in real time and display them to the user.

[0271] Step 5: Export the final model.

[0272] User selects export options:

[0273] Format: FBX;

[0274] Whether to include material: yes;

[0275] File name: "blue_car_with_black_wheels_and_sunroof.fbx".

[0276] Export the model:

[0277] export_model(car_model,format="FBX",filename="blue_car_with_black_wheels_and_sunroof.fbx").

[0278] Example 3: Complex scene modeling.

[0279] The user enters the description "Generate a living room scene, including a sofa, coffee table, and carpet." The system generates a complete scene based on the description and supports further refinement, such as "Change the carpet to gray" or "Add a potted plant on the coffee table."

[0280] Corresponding to the natural language-based intelligent generation method of a three-dimensional model in an embodiment of the present invention, the present invention also discloses a natural language-based intelligent generation system for a three-dimensional model, such as Figure 2 As shown, the system includes an initialization module, a data processing module, an initial model building module and a dynamic model interaction module;

[0281] in,

[0282] The initialization module is used to define the 3D model data structure and the denoising symbol set, and to define and construct a stop word list and a common sense knowledge base according to the scene type;

[0283] The data processing module is configured to receive in real time a natural language description input by a user, parse the natural language description based on the 3D model data structure, the denoising symbol set, and the stop word list, extract and generate key information data of the 3D model based on the parsing result, and map the key information data into a 3D model parameter set;

[0284] It should be noted that the data processing module provided by the present invention can not only parse the natural language description input by the user and extract the key information required for 3D model generation, such as shape, size, material, texture, and detailed characteristics, but also support multi-language input and the analysis of complex descriptions, such as "Generate a round dining table with wood grain, about 1 meter high and 80 centimeters in diameter."

[0285] The initial model construction module is used to train and generate a three-dimensional model generation network based on the scene type and the data structure of the three-dimensional model network, call the three-dimensional model generation network corresponding to the scene type described in natural language and the three-dimensional model parameter set to generate an initial three-dimensional model, and render and visualize the initial three-dimensional model in real time based on the three-dimensional model rendering engine.

[0286] It should be noted that the initial model building module provided by the present invention is also used for:

[0287] Based on the parsing results, a pre-trained 3D model generation network is called to generate an initial 3D model, wherein the 3D model generation network is, for example, a 3D generator based on a diffusion model or a generative adversarial network.

[0288] A multi-level mapping is performed on the input semantics to generate a 3D model hierarchically from global features (such as shape) to local details (such as texture).

[0289] Global feature extraction: Generate the basic geometric structure and global feature framework of the model based on the input semantic information.

[0290] Extract semantic information related to global shape, size, and scale.

[0291] Use geometry generation tools to generate basic frameworks, such as cylinders and cubes.

[0292] Local detail generation: Based on the global framework, surface texture, detail structure and characteristic parameters are added.

[0293] Extract local features (such as material, texture) and add them to the model surface.

[0294] Use texture mapping techniques to optimize detail and adjust lighting and material properties.

[0295] Multi-layer mapping integration: combining global features and local details to generate a complete 3D model.

[0296] Combine global features and local details to generate a complete 3D model.

[0297] Through iterative optimization and consistency verification, the model is ensured to conform to the semantic description.

[0298] The dynamic model interaction module is used to receive and parse the model optimization instructions input by the user in real time based on the initial three-dimensional model, and update the geometric structure, detailed features and physical properties of the initial three-dimensional model according to the parsing results to generate a dynamic interaction model, and generate and output the three-dimensional model file of the dynamic interaction model according to the user request.

[0299] in,

[0300] The noise symbol set is used to store punctuation marks in natural language, the stop word list is used to store the segmented words that constitute natural language in the form of strings, and the segmented words include pronouns, conjunctions, prepositions, quantifiers, modal particles, high-frequency conjunctions and common words. The common sense knowledge base is used to store the default rules of the spatial position relationship between entities in various scenarios;

[0301] The three-dimensional model generation network is a network structure of a three-dimensional model that is pre-defined and trained according to the scene type, as well as a data storage space and a data transmission channel for storing and transmitting key information data of the three-dimensional model;

[0302] The data structure of the three-dimensional model network includes scene type, entity object ID, entity object name, network hierarchy, geometry data, texture data and material data, and the network hierarchy corresponds to the entity relationship of each entity object.

[0303] It should be noted that the dynamic model interaction module is also used to adjust the initial 3D model or the dynamic interaction model in real time. Users can modify the generated 3D model by entering new natural language descriptions, such as "Change the color of the table to dark brown" or "Add matching chairs." This supports multiple rounds of dialogue with users to gradually optimize the model.

[0304] Real-time adjustments to the initial 3D model or dynamic interactive model also include topology correction, surface detail enhancement, and model compression. After real-time adjustments, models with different quality levels can be output, such as high-resolution and low-polygon models.

[0305] Among them, the goal of model compression is to reduce file size, improve rendering efficiency, and maintain visual quality.

[0306] The main steps are as follows:

[0307] 1) Geometric simplification:

[0308] Polygon reduction: Reduce model complexity through algorithms such as Edge Collapse and Vertex Clustering.

[0309] LOD generation: Create model versions with different resolutions to adapt to different viewing distances and device performance requirements.

[0310] 2) Texture optimization and compression:

[0311] Use ASTC and other formats to compress texture files, significantly reducing storage size.

[0312] Generate normal maps to preserve high-resolution details while reducing polygon count.

[0313] 3) Material and shader optimization:

[0314] Merge material instances to reduce rendering state switching.

[0315] Simplify shader logic to improve rendering efficiency on low-performance devices.

[0316] 4) Remove redundant information:

[0317] Delete unused materials, textures, and hidden faces.

[0318] Simplify animation curves and reduce the amount of keyframe data.

[0319] 5) File format conversion and optimization:

[0320] Convert to GLTF / GLB or FBX format to adapt to different platform requirements.

[0321] Choose the right management method for embedded resources and external links.

[0322] By applying the above technical solution, by semantically parsing the natural language description received from the user input, an initial three-dimensional model is generated based on a customized three-dimensional model data structure and a three-dimensional model generation network trained according to different scene types, and the initial three-dimensional model is rendered and visualized based on an interactive window. At the same time, based on the interactive window, the geometric structure, detailed features and physical properties of the initial three-dimensional model are updated in real time according to the model optimization instructions input by the user, a dynamic interactive model is generated, and a three-dimensional model file of the dynamic interactive model is output. This not only realizes the generation of high-quality three-dimensional models through simple language descriptions, but also realizes the dynamic adjustment and diversified expression of the three-dimensional model in real time according to the user's personalized needs. It solves the technical problems existing in the existing technology, such as the inability to understand the semantics and contextual relationship of natural language, the reliance on professional technicians for three-dimensional model generation, the lack of three-dimensional model details, insufficient ability to understand language instructions, and difficulty in dynamically adjusting the model, thereby lowering the threshold for three-dimensional model generation and use, and improving the efficiency, accuracy, flexibility, ease of use and user experience of three-dimensional model modeling.

[0323] In addition, by parsing the basic descriptive information in the user input text, it is possible to identify basic features and further analyze the underlying semantic meaning and contextual relationships. The method provided by the present invention can more accurately convert the user's intention into specific three-dimensional geometric features.

[0324] Users are allowed to provide real-time feedback during the generation process. Based on this feedback, the generation strategy is automatically adjusted to optimize the output. As usage increases, the system accumulates more data to improve its performance. This mechanism enhances the system's adaptability and personalized service capabilities.

[0325] By leveraging a wide range of data resources for training, the model can improve the quality and diversity of generated results while maintaining high efficiency. The training data covers a wide range of object descriptions, which helps the system consistently output high-quality 3D models even when dealing with complex or rare object descriptions.

[0326] Traditional 3D modeling typically requires multiple steps, including manual sketching and parameter setting. In contrast, the method and system provided by the present invention allow users to simply describe the desired object in natural language to quickly obtain the corresponding 3D model, simplifying the modeling process and improving efficiency.

[0327] Considering the diverse needs of different user groups, particularly non-professional users who may lack the necessary modeling knowledge and technical background, the method and system provided by this invention place particular emphasis on user experience. Users can easily create 3D models by simply expressing their ideas in natural language, without requiring any specialized knowledge, thus enhancing ease of use.

[0328] The method and system provided by this invention support a multi-round dialogue mode. After the initial generation, users can further refine their requirements based on actual conditions, such as adjusting the proportions of specific parts or changing material textures. This flexibility ensures that even application scenarios with strict requirements on details can be met, providing flexible and dynamic adjustment capabilities.

[0329] The method and system provided by this invention were designed with cross-domain compatibility and portability in mind. Whether in game development, film and television production, or education and research, any field involving 3D content creation can seamlessly integrate this technology. This provides a convenient 3D content generation solution for various industries, offering broad applicability and excellent scalability.

[0330] Each embodiment in this specification is described in a related manner. The same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

[0331] The above description is only a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention are included in the scope of protection of the present invention.

Claims

1. A method for intelligently generating a three-dimensional model based on natural language, characterized in that: The method comprises: S1, defines the 3D model data structure and the denoising symbol set, and defines and builds a stop word list and common sense knowledge base according to the scene type; S2, receiving a natural language description input by a user in real time, parsing the natural language description based on the 3D model data structure, the denoising symbol set, and the stop word list, extracting and generating key information data of the 3D model based on the parsing result, and mapping the key information data into a 3D model parameter set; S3: training and generating a 3D model generation network based on the scene type and the data structure of the 3D model network; calling the 3D model generation network corresponding to the scene type described in the natural language and the 3D model parameter set to generate an initial 3D model; and performing real-time rendering and visualization of the initial 3D model based on a 3D model rendering engine; S4, based on the initial 3D model, receives and analyzes the model optimization instructions input by the user in real time, and updates the geometric structure, detailed features and physical properties of the initial 3D model according to the analysis results, generates a dynamic interactive model, and generates and outputs a 3D model file of the dynamic interactive model according to the user's request; in, The denoising symbol set is used to store punctuation marks in natural language, the stop word list is used to store the segmented words that constitute natural language in the form of strings, and the segmented words include pronouns, conjunctions, prepositions, quantifiers, modal particles, high-frequency conjunctions and common words. The common sense knowledge base is used to store the default rules of the spatial position relationship between entities in various scenarios; The three-dimensional model generation network is a network structure of a three-dimensional model that is pre-defined and trained according to the scene type, as well as a data storage space and a data transmission channel for storing and transmitting key information data of the three-dimensional model; The data structure of the three-dimensional model network includes scene type, entity object ID, entity object name, network hierarchy, geometry data, texture data and material data, wherein the network hierarchy corresponds to the entity relationship of each entity object; The three-dimensional model data structure includes scene type, entity object, entity relationship, attribute parameters, detail features, and physical attributes; in, The entity relationship includes entity level and topology; The attribute parameters include spatial position, shape parameters, size parameters, material parameters and color; The detailed features include texture, material, smoothness and concave-convex features; The physical properties include light reflection properties, hardness properties and transparency; In step S3, the step of calling a 3D model generation network corresponding to the scene type described in the natural language and the 3D model parameter set to generate an initial 3D model further includes: S321, global structure generation: Inputting the 3D model parameter set into the 3D model generation network corresponding to the scene type described in the natural language, determining the spatial coordinates and geometric shapes of each entity object according to the entity levels of the entity objects corresponding to the natural language description and the entity relationships between the entity objects, in descending order of entity levels, entity object names, and attribute parameters, to generate an initial global geometric structure of the 3D model; S322, local detail generation: Based on the global structure of the initial three-dimensional model, the texture, material, smoothness, and concave-convex features of the physical objects corresponding to the natural language descriptions are added using detail features in the three-dimensional model parameter set to generate a local enhanced geometric structure of the initial three-dimensional model; S323, physical property enhancement: Based on the local detail enhancement structure of the initial three-dimensional model, the physical properties in the three-dimensional model parameter set are used to add the lighting reflection characteristics, hardness characteristics and transparency of each entity object corresponding to the natural language description to generate an initial three-dimensional model.

2. The method according to claim 1, wherein In step S2, the step of parsing the natural language description based on the three-dimensional model data structure, the denoising symbol set, and the stop word list, and extracting and generating key information data of the three-dimensional model according to the parsing result further includes: S21, obtaining text: parsing the user's natural language description to determine the type of the natural language description, and performing text conversion processing on the natural language description based on the parsing result and the type of the natural language description based on the three-dimensional model data structure to obtain the text of the natural language description; S22, text preprocessing: performing word segmentation and denoising processing on the text described in the natural language based on the denoising symbol set and the stop word list, removing punctuation marks and word segmentation in the text, and obtaining a preprocessed text; S23, syntax parsing: constructing a syntax tree according to the preprocessed text, and determining the entity objects, scene trunks, and components of the natural language description based on the constructed syntax tree and the preprocessed text, and marking the parts of speech constituting the natural language description; S24, semantic parsing: performing semantic extraction based on the entity objects, scene backbones, and components described in the natural language, as well as the syntax tree and the preprocessed text, identifying and determining the scene type described in the natural language, attribute parameters of the entity objects, and entity relationships between the entity objects, generating the key information data, and mapping it into a set of 3D model parameters; in, The type of the natural language description, including text and voice; The text includes words, punctuation marks, conjunctions and repeated words; The key information data includes: the scene type described in the natural language, the entity object, the attribute parameters of the entity object, and the entity relationship between the entity objects.

3. The method according to claim 2, wherein The step S21 also includes the steps of speech-to-text conversion and analysis, specifically: If the type of the natural language description is text, obtaining the text of the natural language description according to the parsing result of the natural language description; If the type of the natural language description is speech, the speech corresponding to the natural language description is converted according to the parsing result of the natural language description to obtain the text of the natural language description.

4. The method according to claim 2, wherein In step S22, the text preprocessing step further includes: S22-1, denoising: decomposing the text of the natural language description into words, phrases, and punctuation marks, and converting each of the words, phrases, and punctuation marks into character strings, traversing each of the converted character strings based on the denoising symbol set, and if any of the converted character strings contains a character string that matches a punctuation mark in the denoising symbol set, deleting the matching character string from the character string corresponding to the natural language description, thereby generating a denoised character string; S22-2, stop word filtering: Based on the stop word list, the denoised character string is traversed. If there is a character string in the denoised character string that matches the word in the stop word list, the character string that matches the word in the stop word list is deleted from the denoised character string to obtain the preprocessed text.

5. The method according to claim 2, wherein In step S23, the grammar parsing step further includes: S23-1, constructing a syntax tree: constructing a syntax tree based on the preprocessed text, and annotating the parts of speech of words and phrases in the preprocessed text based on the syntax tree, determining the scene trunk of the natural language description, and the components corresponding to each scene trunk, Wherein, the parts of speech include: core verbs and object nouns; S23-2, entity recognition: Identify the scene type corresponding to the natural language description based on the object noun in the preprocessed text, and determine the entity object based on the scene trunk of the natural language description and the components corresponding to each scene trunk, and mark the parts of speech that constitute the natural language description.

6. The method according to claim 2, wherein In step S24, the semantic parsing step further includes: S24-1, attribute extraction: extracting attribute data of each entity object corresponding to the natural language description based on the name of the entity object, the attribute data including attribute parameters, detailed features and physical properties; S24-2, Relationship Identification: Determine the entity relationships between the entity objects based on the attribute parameters of the entity objects corresponding to the natural language description. If any of the attribute parameters of the entity objects contain null values, call the common sense knowledge base corresponding to the natural language description based on the scene type corresponding to the natural language description and the scene type of the common sense knowledge base to determine the entity relationships between the entity objects corresponding to the natural language description. S24-3, parameter generation: generating the key information data based on the scene type, entity objects, attribute data of the entity objects, and entity relationships between the entity objects described in the natural language, and mapping the key information data into the three-dimensional model parameter set based on the scene type, name and attribute of the entity objects; in, The three-dimensional model parameter set automatically converts key information data described in natural language into an intermediate parameter set for generating a three-dimensional model. Based on the scene type, the name and attributes of the entity object, the entity relationship and attribute data between each entity object are hierarchically mapped in the order of spatial position, shape, material and color.

7. The method according to claim 1, wherein In step S3, the step of training and generating a 3D model generation network according to the scene type and the data structure of the 3D model network further includes: S311, data processing: obtaining a dataset of natural language descriptions corresponding to each scene type, and performing network standard conversion and feature extraction based on the obtained dataset according to each scene type to obtain 3D model generation network data and network feature data in a unified data format, as well as 3D model network features; S312, network model training: generating an initial generation network model based on the three-dimensional model generation network data and the three-dimensional model network features; S313, network model evaluation: optimizing the initial generation network model based on the loss function and the network feature data to generate the three-dimensional model generation network; The formula of the loss function is: Where L is the difference between the predicted result of the generated network model and the actual result, n is the number of network features of the 3D model, n is an integer greater than 1, and x i represents the network features of the i-th real 3D model, Represents the i-th 3D model network feature predicted by the generative network model, Represents the square of the L2 norm of the vector, which is used to calculate the square of the Euclidean distance between the true value and the predicted value of the generative network model. , is a hyperparameter that controls the strength of regularization, and R(θ) is the regularization term with respect to the model parameter θ.

8. A system for implementing the natural language-based intelligent generation method of three-dimensional models according to claim 1, characterized in that: The system includes an initialization module, a data processing module, an initial model building module and a dynamic model interaction module; in, The initialization module is used to define the 3D model data structure and the denoising symbol set, and to define and construct a stop word list and a common sense knowledge base according to the scene type; The data processing module is configured to receive a natural language description input by a user in real time, parse the natural language description based on the 3D model data structure, the denoising symbol set, and the stop word list, extract and generate key information data of the 3D model based on the parsing result, and map the key information data into a 3D model parameter set; The initial model construction module is configured to train and generate a 3D model generation network based on the scene type and the data structure of the 3D model network, call the 3D model generation network corresponding to the scene type described in the natural language and the 3D model parameter set to generate an initial 3D model based on the scene type described in the natural language and the 3D model parameter set, and perform real-time rendering and visualization of the initial 3D model based on the 3D model rendering engine; The dynamic model interaction module is used to receive and analyze the model optimization instructions input by the user in real time based on the initial three-dimensional model, and update the geometric structure, detailed features and physical properties of the initial three-dimensional model according to the analysis results to generate a dynamic interaction model, and generate and output a three-dimensional model file of the dynamic interaction model according to the user request; in, The denoising symbol set is used to store punctuation marks in natural language, the stop word list is used to store the segmented words that constitute natural language in the form of strings, and the segmented words include pronouns, conjunctions, prepositions, quantifiers, modal particles, high-frequency conjunctions and common words. The common sense knowledge base is used to store the default rules of the spatial position relationship between entities in various scenarios; The three-dimensional model generation network is a network structure of a three-dimensional model that is pre-defined and trained according to the scene type, as well as a data storage space and a data transmission channel for storing and transmitting key information data of the three-dimensional model; The data structure of the three-dimensional model network includes scene type, entity object ID, entity object name, network hierarchy, geometry data, texture data and material data, wherein the network hierarchy corresponds to the entity relationship of each entity object; The three-dimensional model data structure includes scene type, entity object, entity relationship, attribute parameters, detail features, and physical attributes; in, The entity relationship includes entity level and topology; The attribute parameters include spatial position, shape parameters, size parameters, material parameters and color; The detailed features include texture, material, smoothness and concave-convex features; The physical properties include light reflection properties, hardness properties and transparency.

Images