3D modeling method and device based on natural language processing, equipment and storage medium

Abstract

The invention discloses a 3D modeling method and device based on natural language processing, equipment and a storage medium, and relates to the technical field of computers. The method comprises the steps of obtaining text data used for describing a target scene, and performing semantic analysis on the text data to obtain a plurality of entities contained in the text data and an association relationship among the plurality of entities; a scene graph is generated according to the multiple entities and the incidence relation, nodes in the scene graph represent the entities, and edges in the scene graph represent the relation between the nodes; traversing each node in the scene graph, and screening out a model component corresponding to each node from a preset 3D model library; and based on the relationship between the nodes, performing spatial layout processing on each model component to obtain a 3D model corresponding to the target scene. According to the method, the corresponding 3D model can be automatically generated by describing the text requirement of the target scene through the natural language, professional modeling skills and software operation skills do not need to be mastered, the use threshold and learning cost of 3D modeling are reduced, and the modeling efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a 3D modeling method, apparatus, device and storage medium based on natural language processing. Background Technology

[0002] 3D modeling technology is widely used in fields such as virtual reality, game development, and industrial design. Its purpose is to transform conceptual scenes or objects into visual 3D digital models to convey spatial information and visual effects.

[0003] Existing 3D modeling methods typically involve professional modelers manually drawing according to design requirements using modeling software, or creating 3D models based on sketches or images. However, professional modeling software is complex to operate and requires modelers to have in-depth professional knowledge and rich practical experience. While 3D modeling methods based on sketches and images lower the barrier to entry, they still require a certain level of professional knowledge. Therefore, existing 3D modeling methods consume a lot of time and effort, have high learning costs, and result in low modeling efficiency, making it difficult to meet the needs of efficient modeling. Summary of the Invention

[0004] This application provides a 3D modeling method, apparatus, device, and storage medium based on natural language processing, which solves the problem of low modeling efficiency caused by professional modelers manually drawing according to design requirements or performing 3D modeling based on sketches and images in the prior art, thereby improving the efficiency of 3D modeling and reducing modeling costs.

[0005] Firstly, this application provides a 3D modeling method based on natural language processing, including: Obtain text data used to describe the target scene, perform semantic parsing on the text data, and obtain multiple entities contained in the text data and the relationships between the multiple entities; A scene graph is generated based on the multiple entities and the relationships, wherein nodes in the scene graph represent the entities and edges represent the relationships between the nodes; Traverse each node in the scene graph and select the model component corresponding to each node from the preset 3D model library; Based on the relationships between the nodes, spatial layout processing is performed on each of the model components to obtain the 3D model corresponding to the target scene.

[0006] Optionally, the spatial layout processing of each model component based on the relationship between the nodes includes: Determine the relationship type corresponding to the relationship between each of the nodes; Based on the relationship type, determine the spatial position of the model component in three-dimensional space corresponding to the node associated with each relationship; Based on the spatial location, spatial layout processing is performed on each of the model components.

[0007] Optionally, determining the spatial location of the model component in three-dimensional space corresponding to each node associated with the relationship based on the relationship type includes: Determine the corresponding spatial layout rules based on the relationship type; Determine the geometric features of the model components corresponding to the nodes associated with each of the relationships, and determine the spatial layout constraints based on the geometric features; Based on the spatial layout rules and spatial layout constraints, determine the spatial position of the model component in three-dimensional space corresponding to each node associated with the relationship.

[0008] Optionally, determining the corresponding spatial layout rules based on the relationship type includes: Obtain the preset mapping table between the relationship type and the spatial layout rules; Match the relationship type with the relationship type in the mapping table; Based on the matching results, the corresponding spatial layout rules are retrieved from the mapping table.

[0009] Optionally, determining the spatial layout constraints based on the geometric features includes: Based on the relationship type, determine the interaction conditions under which the model component implements the relationship; Based on the aforementioned geometric features and interaction conditions, spatial layout constraints are determined.

[0010] Optionally, determining the spatial position of the model component corresponding to each of the related nodes in three-dimensional space according to the spatial layout rules and spatial layout constraints includes: The initial spatial position of the model component in three-dimensional space corresponding to each node associated with the relationship is determined according to the spatial layout rules. Based on the spatial layout constraints, the initial spatial position is adjusted to obtain the spatial position of the model component corresponding to each of the related nodes in three-dimensional space.

[0011] Optionally, the method further includes: If no model component corresponding to the node is found in the preset 3D model library, obtain the preset three-dimensional voxel mesh space and the preset voxel mesh design parameters. Based on the entity and the voxel mesh design parameters, a voxel model representing the three-dimensional shape of the entity is generated in the voxel mesh space; Convert the voxel model into a polygonal mesh model; The polygonal mesh model is post-processed to obtain the model component corresponding to the entity.

[0012] Secondly, this application provides a 3D modeling apparatus based on natural language processing, comprising: The semantic parsing module is used to acquire text data describing the target scene, perform semantic parsing on the text data, and obtain multiple entities contained in the text data and the relationships between the multiple entities. A scene graph generation module is used to generate a scene graph based on the multiple entities and the relationships, wherein nodes in the scene graph represent the entities and edges represent the relationships between the nodes; The model component filtering module is used to traverse each node in the scene graph and filter out the model components corresponding to each node from the preset 3D model library. The spatial layout module is used to perform spatial layout processing on each of the model components based on the relationship between the nodes, so as to obtain the 3D model corresponding to the target scene.

[0013] Thirdly, this application provides a 3D modeling device based on natural language processing, comprising: One or more processors; A memory that stores one or more programs that, when executed by one or more processors, enable the one or more processors to implement the natural language processing-based 3D modeling method as described in the first aspect.

[0014] Fourthly, this application provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the 3D modeling method based on natural language processing as described in the first aspect.

[0015] In summary, compared with the prior art, the beneficial effects of the technical solution provided in this application include at least the following: In this application, text data describing a target scene is acquired, and semantic parsing is performed on the text data to obtain multiple entities contained in the text data and the relationships between these entities. A scene graph is generated based on the multiple entities and their relationships, where nodes in the scene graph represent entities, and edges represent relationships between nodes. Each node in the scene graph is traversed, and model components corresponding to each node are selected from a pre-set 3D model library. Based on the relationships between nodes, spatial layout processing is performed on each model component to obtain a 3D model corresponding to the target scene. In this technical approach, a corresponding 3D model can be automatically generated simply by describing the target scene's text requirements in natural language. No professional modeling or software operation skills are required, allowing ordinary users to efficiently complete modeling, thus lowering the barrier to entry and learning cost of 3D modeling and improving modeling efficiency. Attached Figure Description

[0016] Figure 1 This is a flowchart of a 3D modeling method based on natural language processing provided in an embodiment of this application; Figure 2 This is a flowchart of generating corresponding model components based on entities associated with nodes, as provided in the embodiments of this application; Figure 3 This is a flowchart of spatial layout processing for each model component provided in an embodiment of this application; Figure 4 This is a flowchart illustrating the spatial location of the model components corresponding to the nodes associated with each relationship in three-dimensional space, provided in an embodiment of this application. Figure 5 This is a schematic diagram of the structure of a 3D modeling device based on natural language processing provided in an embodiment of this application; Figure 6 This is a schematic diagram of the structure of a 3D modeling device based on natural language processing provided in an embodiment of this application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, specific embodiments of this application will be described in further detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely for explaining this application and not for limiting it. It should also be noted that, for ease of description, only the parts relevant to this application are shown in the drawings, not all of them. Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe operations (or steps) as sequential processes, many of these operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. A process can be terminated when its operation is completed, but it may also have additional steps not included in the drawings. A process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0018] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0019] Existing 3D modeling methods typically involve professional modelers manually drawing according to design requirements using modeling software, or creating 3D models based on sketches or images. However, professional modeling software is complex to operate and requires modelers to have in-depth professional knowledge and rich practical experience. While 3D modeling methods based on sketches and images lower the barrier to entry, they still require a certain level of professional knowledge. Therefore, existing 3D modeling methods consume a lot of time and effort, have high learning costs, and result in low modeling efficiency, making it difficult to meet the needs of efficient modeling.

[0020] To address the problem of low modeling efficiency in existing technologies where professional modelers manually draw according to design requirements using specialized modeling software, or perform 3D modeling based on sketches or images, this application provides a 3D modeling method based on natural language processing. By describing the textual requirements of the target scene in natural language, the corresponding 3D model can be automatically generated. No professional modeling or software operation skills are required, and ordinary users can efficiently complete the modeling process, reducing the barrier to entry and learning cost of 3D modeling and improving modeling efficiency.

[0021] The 3D modeling method based on natural language processing provided in this embodiment can be executed by a 3D modeling device based on natural language processing. The 3D modeling device based on natural language processing can be implemented by software and / or hardware. The 3D modeling device based on natural language processing can be composed of two or more physical entities, or it can be composed of a single physical entity.

[0022] The 3D modeling device based on natural language processing is equipped with at least one type of operating system, including but not limited to Android, Linux, and Windows. The 3D modeling device can install at least one application on the operating system; this application can be a built-in application of the operating system or an application downloaded from a third-party device or server. In this embodiment, the 3D modeling device based on natural language processing has at least one application capable of executing 3D modeling methods based on natural language processing.

[0023] The technical solutions provided by the embodiments of this application are described in detail below with reference to the accompanying drawings.

[0024] Figure 1 A flowchart of a 3D modeling method based on natural language processing provided in an embodiment of this application is given. Please refer to... Figure 1 The 3D modeling method based on natural language processing specifically includes: S110. Obtain text data used to describe the target scene, perform semantic parsing on the text data, and obtain multiple entities contained in the text data and the association relationships between the multiple entities.

[0025] The text data used to describe the target scene refers to text information that describes the types, attributes, spatial relationships, and logical relationships of objects in the three-dimensional scene using natural language. An entity refers to an object with three-dimensional representation capabilities identified from the text data. For example, this application embodiment uses natural language processing technology to perform semantic parsing on the text data used to describe the target scene to obtain multiple entities contained in the text data and the relationships between them. Further, firstly, the text data describing the target scene is cleaned (irrelevant characters and standardized punctuation marks are removed), and then the cleaned text data is segmented into word sequences, with each word tagged with a part-of-speech tag. Then, entity objects (such as "table," "vase," etc.) in the text data are identified. Simultaneously, through syntactic analysis and dependency parsing, the main-subordinate relationships in the sentences are analyzed to determine the relationships between entities.

[0026] For example, identifying entity objects in text includes: vectorizing the text sequence after word segmentation and part-of-speech tagging, that is, mapping each word to a dense vector in a high-dimensional space to capture the semantic information and contextual features of the words; then inputting the word vector sequence into a pre-trained deep learning-based sequence labeling model (such as BiLSTM-CRF or BERT) to predict the labels of the vectorized word vector sequence to output the entity label corresponding to each word.

[0027] For example, parsing the subject-subject relationship in a sentence and determining the relationship between entities through syntactic analysis and dependency parsing includes: performing syntactic analysis on the text to generate a syntactic tree to identify the sentence structure, including components such as subject, predicate, and object, which helps to initially locate entities and their positions in the sentence; then, performing dependency parsing to extract semantic dependencies between words based on the syntactic tree and identify the dominance relationship between entities, such as "on..." indicating a positional relationship and "inserted" indicating an inclusion relationship, etc.

[0028] S120. Generate a scene graph based on the plurality of entities and the association relationships, wherein the nodes in the scene graph represent the entities and the edges represent the relationships between the nodes.

[0029] Scene graphs are graphical data structures used to structurally describe entities and their relationships in a 3D scene. The core idea is to abstract objects (entities) in the scene as nodes in the graph, and to abstract the semantic relationships between objects (such as spatial location, logical interaction, etc.) as edges connecting nodes, thereby presenting the composition structure and semantic information of the scene in a graphical way.

[0030] For example, generating a scene graph based on multiple entities and the relationships between them includes the following steps: Create a node for each entity; determine the nodes corresponding to entities with relationships; establish connecting edges between the nodes corresponding to the entities with relationships to represent the corresponding relationships; and construct a structured scene graph based on the nodes and connecting edges.

[0031] S130. Traverse each node in the scene graph and select the model component corresponding to each node from the preset 3D model library.

[0032] In this embodiment of the application, if no model component corresponding to a node is found in the preset 3D model library, a corresponding model component will be generated based on the entity associated with the node. For example, Figure 2 This is a flowchart illustrating the generation of corresponding model components based on node-association entities provided in this application embodiment; please refer to... Figure 2The steps for generating corresponding model components based on the entities associated with nodes include S1301-S1304: S1301. Obtain the preset three-dimensional voxel mesh space and the preset voxel mesh design parameters.

[0033] In this context, the 3D voxel mesh space refers to the discretized 3D spatial framework used to carry out the process of generating the 3D shape of an entity. It can be understood as a three-dimensional framework composed of countless tiny cubic units (voxels) arranged neatly. Voxel mesh design parameters are used to define and adjust the characteristics of the 3D voxel mesh space and control the voxel model generation process. For example, these parameters include, but are not limited to, voxel size and resolution. Voxel size determines the actual size of each voxel unit in 3D space, affecting the model's accuracy and level of detail; resolution describes the number of voxels arranged along each coordinate axis within a given spatial range, and together with voxel size, determines the richness of detail the model can represent.

[0034] S1302. Based on the entity and the voxel mesh design parameters, generate a voxel model representing the three-dimensional shape of the entity in the voxel mesh space.

[0035] Among them, the voxel model is a model that uses voxels to represent three-dimensional objects; Specifically, in the voxel mesh space, different components of the entity are individually voxel-filled according to the entity's structure to generate a voxel model representing the entity's 3D shape. This includes structural analysis of the identified entity, determining its main components and the relative positions and connections between them. For example, for the entity "table," analysis reveals it typically consists of a tabletop and legs, with the legs located below the tabletop and evenly distributed at the four corners or specific locations. This structural analysis breaks the entity down into multiple processable components. For each component, its position in the 3D voxel mesh space is determined based on the relative positions of the components. Voxel filling is then performed based on each component's position in the 3D voxel mesh space to obtain a voxel model representing the entity's 3D shape.

[0036] S1303. Convert the voxel model into a polygon mesh model.

[0037] Specifically, a voxel model is a pixelated 3D structure composed of a large number of discrete 3D voxel units (such as cubes). Its surface is formed by splicing the boundaries of voxel units, exhibiting discrete characteristics and making it difficult to represent smooth curved surfaces or fine structures. On the other hand, a polygonal mesh model is a continuous surface representation formed by the topological connection of vertices, edges, and faces (polygons). Each face is a plane or curved surface segment. Therefore, it is necessary to convert the voxel model into a polygonal mesh model in order to more accurately reproduce the morphological characteristics of the entity. For example, this application uses an isosurface extraction algorithm to extract continuous surface contours from a voxel model composed of discrete voxels, and then forms a mesh model with a topological structure by splicing polygons. Specifically, it includes: traversing all voxel units in the voxel model; for each voxel unit, checking the numerical values ​​of its neighboring voxel units; determining whether there is an isosurface at the location of the voxel unit by comparing the values ​​of neighboring voxel units with the isosurface threshold; if so, calculating the intersection points of the isosurface with each edge of the voxel unit using linear interpolation and other methods based on the numerical distribution of neighboring voxel units; then, based on the intersection points, splicing adjacent isosurfaces according to certain topological rules to gradually construct a continuous surface contour; wherein, the surface contour is composed of multiple polygonal faces (such as triangular faces), ultimately forming a polygonal mesh model with a topological structure, thereby converting the discrete voxel model into a polygonal mesh model that can more accurately reproduce the morphological features of the entity.

[0038] S1304. Post-process the polygonal mesh model to obtain the model component corresponding to the entity.

[0039] In this embodiment of the application, the post-processing includes: performing mesh optimization processing on the converted polygon mesh model, the mesh optimization processing including at least one of mesh simplification and smooth shading; and generating a UV unfolded map for the optimized polygon mesh model, that is, converting the three-dimensional mesh surface into two dimensions, and assigning corresponding material properties to the polygon mesh model for which the UV unfolded map is generated based on entity properties or requirements, so as to obtain the post-processed model component.

[0040] S140. Based on the relationship between the nodes, perform spatial layout processing on each of the model components to obtain the 3D model corresponding to the target scene.

[0041] Spatial layout processing refers to the process of arranging and positioning the selected or generated model components corresponding to each node in three-dimensional space according to the entity relationships represented by the nodes in the scene graph, so as to construct a 3D model that conforms to the target scene described by the text data.

[0042] For example, Figure 3 This is a flowchart illustrating the spatial layout processing of various model components provided in this application embodiment; please refer to [link / reference]. Figure 3The steps for spatial layout processing of each model component include S1401-S1403: S1401. Determine the relationship type corresponding to the relationship between the nodes.

[0043] Among them, relationship type refers to the category identifier used to describe the different nature of the association between nodes (i.e., entities) in the scene graph; it is an abstraction and generalization of the semantic and spatial relationships between entities, which can clearly understand the key information such as the interaction and relative position between entities, thereby providing accurate guidance for the spatial layout of model components in 3D modeling. In the embodiments of this application, relationship type may include positional relationships (such as "above / below / left / right", "adjacent", "enclosed", etc.), directional relationships (such as "facing", "back to", "parallel to", "perpendicular to"), connection relationships (such as "fixed to", "suspended to", "supported"), hierarchical relationships (such as "contained in", "partially belonging to"), etc.

[0044] S1402. Based on the relationship type, determine the spatial position of the model component in three-dimensional space corresponding to the node associated with each relationship.

[0045] Spatial location refers to the position of each model component within the entire 3D scene in a 3D coordinate system. For example, Figure 4 This is a flowchart illustrating the determination of the spatial location of the model components corresponding to the nodes associated with each relationship in three-dimensional space, as provided in the embodiments of this application; please refer to... Figure 4 The step of determining the spatial location of the model component corresponding to each relation-related node in three-dimensional space includes S14021-S14023: S14021. Determine the corresponding spatial layout rules based on the relationship type.

[0046] Spatial layout rules refer to the methods used to guide the arrangement, combination, or mutual positioning of model components in three-dimensional space. For example, in this embodiment, the step of determining the corresponding spatial layout rule based on the relationship type includes: obtaining a preset mapping table between relationship types and spatial layout rules; matching the relationship type with the relationship type in the mapping table; and retrieving the corresponding spatial layout rule from the mapping table based on the matching result.

[0047] S14022. Determine the geometric features of the model components corresponding to the nodes associated with each relationship, and determine the spatial layout constraints based on the geometric features.

[0048] Spatial layout constraints refer to the restrictions that need to be met during the spatial layout process. These constraints are related to the geometric features of the model components, interaction conditions, and the limiting factors of the actual scene. Geometric features include the size (length, width, height) and shape (regular geometric shapes such as cubes and spheres, or irregular shapes) of the model components.

[0049] For example, the steps for determining spatial layout constraints include: Based on the relationship type, determine the interaction conditions for the model components to realize the relationship; for example, for the positional relationship of "place on top of", the interaction condition may be that the contact surfaces of the two model components must fit together; for the relationship of "inserted into", the interaction condition may be that a part of one model component can be completely embedded in the internal space of another model component.

[0050] Based on geometric features and interaction conditions, spatial layout constraints are determined. For example, suppose there are two model components: model component A (representing a table) and model component B (representing a vase). According to the text data description, the vase needs to be placed on the table, and its interaction condition is that the bottom surface of model B must fit with the top surface of model A. Then, based on the fact that the vase needs to be placed on the table and its bottom fits with the tabletop, as well as the geometric features of the table and the vase, the layout constraints are determined as follows: in terms of position, the Y-coordinate of the center of the vase's bottom surface is equal to the Y-coordinate of the tabletop; the vase is vertical, and the normal of its bottom surface is consistent with the negative direction of the Y-axis; at the same time, the projection of the vase's bottom must be within the projection range of the top of the tabletop.

[0051] S14023. Based on the spatial layout rules and spatial layout constraints, determine the spatial position of the model component corresponding to each of the nodes associated with the relationship in the three-dimensional space.

[0052] For example, the step of determining the spatial location of the model component corresponding to each relationally associated node in three-dimensional space includes: The initial spatial positions of the model components corresponding to each related node in 3D space are determined according to spatial layout rules. Based on spatial layout constraints, these initial spatial positions are adjusted to obtain the final spatial positions of the model components corresponding to each related node in 3D space. Adjusting the initial spatial positions based on spatial layout constraints takes into account that the geometric features and interaction conditions of the model components will affect their actual placement. By comprehensively considering these conditions, the initial spatial positions are precisely adjusted to obtain the final spatial positions of the model components corresponding to each related node in 3D space that meet the requirements of the actual scene, ensuring that the generated 3D model can realistically represent the target scene described by the text data.

[0053] S1403. Based on the spatial location, perform spatial layout processing on each of the model components.

[0054] In this embodiment of the application, after obtaining the 3D model corresponding to the text of the target scene, the method further includes: The generated 3D model is optimized, for example, by simplifying the model structure and smoothing the surface. Then, based on the preset rendering requirements, the rendering parameters are determined. Finally, the optimized 3D model is rendered according to the rendering parameters to obtain the final 3D model.

[0055] In summary, the 3D modeling method based on natural language processing provided in this application involves: acquiring text data describing a target scene; performing semantic parsing on the text data to obtain multiple entities and their relationships; generating a scene graph based on the entities and relationships, where nodes represent entities and edges represent relationships between nodes; traversing each node in the scene graph and selecting model components corresponding to each node from a pre-set 3D model library; and performing spatial layout processing on the model components based on the relationships between nodes to obtain the 3D model corresponding to the target scene. In this technical approach, a corresponding 3D model can be automatically generated simply by describing the target scene in natural language, without requiring professional modeling or software operation skills. Ordinary users can efficiently complete modeling, reducing the barrier to entry and learning cost of 3D modeling and improving modeling efficiency.

[0056] Figure 5 This is a schematic diagram of a 3D modeling device based on natural language processing, provided as an embodiment of this application. (Reference) Figure 5 The 3D modeling device based on natural language processing provided in this embodiment specifically includes: a semantic parsing module 21, a scene graph generation module 22, a model component selection module 23, and a spatial layout module 24.

[0057] The semantic parsing module 21 is used to acquire text data describing the target scene, perform semantic parsing on the text data, and obtain multiple entities contained in the text data and the association relationships between the multiple entities. Scene graph generation module 22 is used to generate a scene graph based on the plurality of entities and the association relationship, wherein the nodes in the scene graph represent the entities and the edges represent the relationship between the nodes; The model component filtering module 23 is used to traverse each node in the scene graph and filter out the model components corresponding to each node from the preset 3D model library. The spatial layout module 24 is used to perform spatial layout processing on each of the model components based on the relationship between the nodes, so as to obtain the 3D model corresponding to the target scene.

[0058] Based on the above embodiments, the spatial layout module 24 includes: A type determination unit is used to determine the relationship type corresponding to the relationship between the nodes. A spatial location determination unit is used to determine the spatial location of the model component corresponding to each node associated with the relationship in three-dimensional space according to the relationship type. A spatial layout unit is used to perform spatial layout processing on each of the model components based on the spatial location.

[0059] Based on the above embodiments, the spatial location determination unit includes: The layout rule determines the sub-unit, which is used to determine the corresponding spatial layout rule according to the relationship type; The constraint determination subunit is used to determine the geometric features of the model components corresponding to the nodes associated with each of the relationships, and to determine the spatial layout constraints based on the geometric features. The spatial location determination subunit is used to determine the spatial location of the model component corresponding to each of the related nodes in three-dimensional space according to the spatial layout rules and the spatial layout constraints.

[0060] Based on the above embodiments, the layout rules for determining sub-units include: Obtain the preset mapping table between the relationship type and the spatial layout rules; Match the relationship type with the relationship type in the mapping table; Based on the matching results, the corresponding spatial layout rules are retrieved from the mapping table.

[0061] Based on the above embodiments, the constraint condition determination subunit includes: Based on the relationship type, determine the interaction conditions under which the model component implements the relationship; Based on the aforementioned geometric features and interaction conditions, spatial layout constraints are determined.

[0062] Based on the above embodiments, the spatial location determination subunit includes: The initial spatial position of the model component in three-dimensional space corresponding to each node associated with the relationship is determined according to the spatial layout rules. Based on the spatial layout constraints, the initial spatial position is adjusted to obtain the spatial position of the model component corresponding to each of the related nodes in three-dimensional space.

[0063] Based on the above embodiments, it also includes: If no model component corresponding to the node is found in the preset 3D model library, obtain the preset three-dimensional voxel mesh space and the preset voxel mesh design parameters. Based on the entity and the voxel mesh design parameters, a voxel model representing the three-dimensional shape of the entity is generated in the voxel mesh space; Convert the voxel model into a polygonal mesh model; The polygonal mesh model is post-processed to obtain the model component corresponding to the entity.

[0064] In summary, the 3D modeling apparatus based on natural language processing provided in this application acquires text data describing a target scene, performs semantic parsing on the text data to obtain multiple entities contained in the text data and the relationships between these entities; generates a scene graph based on the multiple entities and their relationships, where nodes in the scene graph represent entities and edges represent relationships between nodes; traverses each node in the scene graph and selects model components corresponding to each node from a preset 3D model library; and performs spatial layout processing on each model component based on the relationships between nodes to obtain a 3D model corresponding to the target scene. In this technical approach, a corresponding 3D model can be automatically generated simply by describing the target scene's text requirements using natural language. No professional modeling or software operation skills are required, allowing ordinary users to efficiently complete modeling, thus lowering the barrier to entry and learning cost of 3D modeling and improving modeling efficiency.

[0065] The 3D modeling apparatus based on natural language processing provided in this application embodiment can be used to execute the 3D modeling method based on natural language processing provided in the above embodiment, and has corresponding functions and beneficial effects.

[0066] Figure 6 This is a schematic diagram of the structure of a 3D modeling device based on natural language processing provided in an embodiment of this application. (Refer to...) Figure 6 The natural language processing-based 3D modeling device includes a processor 31, a memory 32, a communication device 33, an input device 34, and an output device 35. The number of processors 31 and the number of memories 32 in the natural language processing-based 3D modeling device can be one or more. The processor 31, memory 32, communication device 33, input device 34, and output device 35 of the natural language processing-based 3D modeling device can be connected via a bus or other means.

[0067] The memory 32, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as program instructions / modules corresponding to the natural language processing-based 3D modeling method in any embodiment of this application (e.g., semantic parsing module 21, scene graph generation module 22, model component filtering module 23, and spatial layout module 24 in a natural language processing-based 3D modeling device). The memory 32 may primarily include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a function; the data storage area may store data created based on the use of the device, etc. Furthermore, the memory 32 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0068] The communication device 33 is used for data transmission.

[0069] The processor 31 executes various functional applications and data processing of the device by running software programs, instructions and modules stored in the memory 32, thereby realizing the above-mentioned 3D modeling method based on natural language processing.

[0070] Input device 34 can be used to receive input digital or character information, and to generate key signal inputs related to user settings and function control of the device. Output device 35 may include display devices such as a display screen.

[0071] The 3D modeling device based on natural language processing provided above can be used to execute the 3D modeling method based on natural language processing provided in the above embodiments, and has corresponding functions and beneficial effects.

[0072] This application embodiment also provides a storage medium containing computer-executable instructions. When executed by a computer processor, the computer-executable instructions are used to execute a 3D modeling method based on natural language processing. The 3D modeling method based on natural language processing includes: acquiring text data describing a target scene; performing semantic parsing on the text data to obtain multiple entities contained in the text data and the relationships between the multiple entities; generating a scene graph based on the multiple entities and the relationships, wherein nodes in the scene graph represent entities and edges represent relationships between nodes; traversing each node in the scene graph and selecting model components corresponding to each node from a preset 3D model library; and performing spatial layout processing on each model component based on the relationships between nodes to obtain a 3D model corresponding to the target scene.

[0073] Storage medium – any type of memory device or storage device. The term “storage medium” is intended to include: mounting media, such as CD-ROM, floppy disk, or magnetic tape devices; computer system memory or random access memory, such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory, such as flash memory, magnetic media (e.g., hard disk or optical storage); registers or other similar types of memory elements, etc. Storage medium may also include other types of memory or combinations thereof. Furthermore, storage medium may reside in a first computer system in which the program is executed, or it may reside in a different second computer system connected to the first computer system via a network (such as the Internet). The second computer system can provide program instructions to the first computer for execution. The term “storage medium” can include two or more storage media residing in different locations (e.g., in different computer systems connected via a network). Storage medium may store program instructions (e.g., specifically implemented as a computer program) executable by one or more processors.

[0074] Of course, the computer-executable instructions provided in the embodiments of this application are not limited to the 3D modeling method based on natural language processing as described above, but can also execute related operations in the 3D modeling method based on natural language processing provided in any embodiment of this application.

[0075] The 3D modeling apparatus, storage medium, and 3D modeling device based on natural language processing provided in the above embodiments can execute the 3D modeling method based on natural language processing provided in any embodiment of this application. For technical details not described in detail in the above embodiments, please refer to the 3D modeling method based on natural language processing provided in any embodiment of this application.

[0076] The above description is merely a preferred embodiment and the technical principles employed in this application. This application is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions that can be made by those skilled in the art will not depart from the scope of protection of this application. Therefore, although this application has been described in detail through the above embodiments, this application is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of this application. The scope of this application is determined by the scope of the claims.

Claims

1. A 3D modeling method based on natural language processing, characterized in that, include: Obtain text data used to describe the target scene, perform semantic parsing on the text data, and obtain multiple entities contained in the text data and the relationships between the multiple entities; A scene graph is generated based on the multiple entities and the relationships, wherein nodes in the scene graph represent the entities and edges represent the relationships between the nodes; Traverse each node in the scene graph and select the model component corresponding to each node from the preset 3D model library; Based on the relationships between the nodes, spatial layout processing is performed on each of the model components to obtain the 3D model corresponding to the target scene.

2. The 3D modeling method based on natural language processing according to claim 1, characterized in that, The spatial layout processing of each model component based on the relationship between the nodes includes: Determine the relationship type corresponding to the relationship between each of the nodes; Based on the relationship type, determine the spatial position of the model component in three-dimensional space corresponding to the node associated with each relationship; Based on the spatial location, spatial layout processing is performed on each of the model components.

3. The 3D modeling method based on natural language processing according to claim 2, characterized in that, The step of determining the spatial location in three-dimensional space of the model component corresponding to each node associated with the relationship based on the relationship type includes: Determine the corresponding spatial layout rules based on the relationship type; Determine the geometric features of the model components corresponding to the nodes associated with each of the relationships, and determine the spatial layout constraints based on the geometric features; Based on the spatial layout rules and spatial layout constraints, determine the spatial position of the model component in three-dimensional space corresponding to each node associated with the relationship.

4. The 3D modeling method based on natural language processing according to claim 3, characterized in that, The step of determining the corresponding spatial layout rules based on the relationship type includes: Obtain the preset mapping table between the relationship type and the spatial layout rules; Match the relationship type with the relationship type in the mapping table; Based on the matching results, the corresponding spatial layout rules are retrieved from the mapping table.

5. The 3D modeling method based on natural language processing according to claim 3, characterized in that, The determination of spatial layout constraints based on the geometric features includes: Based on the relationship type, determine the interaction conditions under which the model component implements the relationship; Based on the aforementioned geometric features and interaction conditions, spatial layout constraints are determined.

6. The 3D modeling method based on natural language processing according to claim 3, characterized in that, The step of determining the spatial position of the model component in three-dimensional space corresponding to each node associated with the relationship, based on the spatial layout rules and spatial layout constraints, includes: The initial spatial position of the model component in three-dimensional space corresponding to each node associated with the relationship is determined according to the spatial layout rules. Based on the spatial layout constraints, the initial spatial position is adjusted to obtain the spatial position of the model component corresponding to each of the related nodes in three-dimensional space.

7. The 3D modeling method based on natural language processing according to any one of claims 1-6, characterized in that, The method further includes: If no model component corresponding to the node is found in the preset 3D model library, obtain the preset three-dimensional voxel mesh space and the preset voxel mesh design parameters. Based on the entity and the voxel mesh design parameters, a voxel model representing the three-dimensional shape of the entity is generated in the voxel mesh space; Convert the voxel model into a polygonal mesh model; The polygonal mesh model is post-processed to obtain the model component corresponding to the entity.

8. A 3D modeling device based on natural language processing, characterized in that, include: The semantic parsing module is used to acquire text data describing the target scene, perform semantic parsing on the text data, and obtain multiple entities contained in the text data and the relationships between the multiple entities. A scene graph generation module is used to generate a scene graph based on the multiple entities and the relationships, wherein nodes in the scene graph represent the entities and edges represent the relationships between the nodes; The model component filtering module is used to traverse each node in the scene graph and filter out the model components corresponding to each node from the preset 3D model library. The spatial layout module is used to perform spatial layout processing on each of the model components based on the relationship between the nodes, so as to obtain the 3D model corresponding to the target scene.

9. A 3D modeling device based on natural language processing, characterized in that, include: One or more processors; A memory that stores one or more programs, which, when executed by one or more processors, enable the one or more processors to implement the 3D modeling method based on natural language processing as described in any one of claims 1-7.

10. A storage medium containing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are used to perform the 3D modeling method based on natural language processing as described in any one of claims 1-7.

Images