Text-to-three-dimensional generation architecture optimization method and system based on hierarchical feature fusion

Abstract

The present application relates to the technical field of model construction, and a text-to-three-dimensional generation architecture optimization method and system based on hierarchical feature fusion, comprising: obtaining text data, performing a preprocessing operation on the text data to obtain a standard vocabulary type set, performing feature extraction on the standard vocabulary type set to obtain a text feature set, classifying the text feature set to obtain a feature type set, obtaining a geometric feature set, performing hierarchical feature fusion on the feature type set and the geometric feature set to obtain a multi-modal feature set, inputting the multi-modal feature set into a three-dimensional generation model to obtain an initial three-dimensional model, performing parameter verification on the initial three-dimensional model to obtain a target three-dimensional model, and completing text-to-three-dimensional generation architecture optimization. The present application can solve the problem of low precision in the construction process of a three-dimensional model.

Description

Technical Field

[0001] This invention relates to the field of model building technology, and in particular to a text-to-3D generation architecture optimization method and system based on hierarchical feature fusion. Background Technology

[0002] With the rapid development of artificial intelligence and computer graphics, 3D models are increasingly widely used in fields such as virtual reality and intelligent manufacturing. Generating 3D models based on text descriptions is an important method for this purpose.

[0003] Traditional 3D model generation methods primarily rely on images for modeling. While these methods can generate 3D models from image information, they lack accuracy when handling complex 3D structures and details, resulting in models with insufficient precision to meet practical needs. Therefore, current methods for constructing 3D models suffer from low accuracy. Summary of the Invention

[0004] This invention provides a text-to-3D generation architecture optimization method and system based on hierarchical feature fusion, the main purpose of which is to solve the problem of low accuracy in the construction of 3D models.

[0005] To achieve the above objectives, this invention provides a text-to-3D generation architecture optimization method based on hierarchical feature fusion, comprising:

[0006] A 3D generation environment is defined, wherein the 3D generation environment includes: a text processing unit, a feature processing unit, and a feature fusion unit;

[0007] The text data is acquired, and the text processing unit is used to preprocess the text data to obtain a standard vocabulary type set.

[0008] Based on the feature processing unit, feature extraction is performed on the standard vocabulary type set to obtain a text feature set;

[0009] The text feature set is classified to obtain a feature type set;

[0010] Obtain a geometric feature set, and use the feature fusion unit to perform hierarchical feature fusion with the feature type set and the geometric feature set to obtain a multimodal feature set;

[0011] The multimodal feature set is input into a preset 3D generation model to obtain an initial 3D model;

[0012] The parameters of the initial 3D model are validated to obtain the target 3D model, thus completing the optimization of the text-to-3D generation architecture.

[0013] Optionally, the step of using the text processing unit to preprocess the text data to obtain a standard vocabulary type set includes:

[0014] The text processing unit is used to clean the text data to obtain standard text data.

[0015] The standard text data is segmented to obtain a text vocabulary set;

[0016] For each text word in the text vocabulary set, the following operation is performed:

[0017] The text words are marked with stop words according to a preset stop word library to obtain the marking results;

[0018] If the marking result is a preset stop mark, then the text words are marked as stop words;

[0019] The stop words are summarized to obtain a stop word set;

[0020] The stop word set is removed from the text vocabulary set to obtain the effective text vocabulary set;

[0021] Part-of-speech tagging is performed on the effective text vocabulary set to obtain a vocabulary type set;

[0022] The vocabulary type set is standardized according to a preset standardization method to obtain a standard vocabulary type set.

[0023] Optionally, the step of performing part-of-speech tagging on the effective text vocabulary set to obtain a vocabulary type set includes:

[0024] For each valid text word in the set of valid text words, the following operation is performed:

[0025] Determine the semantic features of effective text vocabulary;

[0026] Based on the semantic features and the preset part-of-speech tagging method, the effective text words are matched by attribute categories to obtain the word matching results;

[0027] The valid text words are labeled according to the word matching results to obtain the word type;

[0028] By summarizing the aforementioned vocabulary types, a vocabulary type set is obtained.

[0029] Optionally, the step of extracting features from the standard vocabulary type set based on the feature processing unit to obtain a text feature set includes:

[0030] Using a pre-built word embedding model, vector mapping is performed on the standard vocabulary type set to obtain a word vector set, wherein the word vector set contains multiple word vectors, and the word vectors correspond one-to-one with the standard vocabulary types in the standard vocabulary type set;

[0031] Obtain the original text order, and sort the word vector set according to the original text order to obtain the word vector sequence;

[0032] Obtain a context feature extraction model, and based on the context feature extraction model, perform semantic encoding on the word vector sequence to obtain a context feature vector set;

[0033] The context feature vector set is pooled to obtain a comprehensive text feature vector;

[0034] For each context feature vector in the set of context feature vectors, the following operation is performed:

[0035] Using the feature processing unit, the comprehensive text feature vector and the context feature vector are fused to obtain a fused feature vector;

[0036] The fused feature vectors are then combined to obtain the text feature set.

[0037] Optionally, the acquisition of the context feature extraction model includes:

[0038] Obtain multiple word vector sequence samples;

[0039] The multiple word vector sequence samples are divided according to a preset division ratio to obtain a training sample set and a validation sample set, wherein the training sample set contains multiple training samples and the validation sample set contains multiple validation samples.

[0040] A pre-built machine learning model is trained using multiple training samples from the training sample set to obtain a candidate feature extraction model.

[0041] Multiple verification feature vectors are obtained based on multiple verification samples in the verification sample set, wherein each verification feature vector corresponds one-to-one with a verification sample.

[0042] The candidate feature extraction model is validated using multiple validation samples, resulting in multiple predicted feature vectors, each corresponding one-to-one with a validation sample.

[0043] The average similarity is calculated based on the multiple predicted feature vectors and multiple verification feature vectors.

[0044] If the average similarity is less than a preset similarity threshold, the hyperparameters of the candidate feature extraction model are obtained, the hyperparameters are adjusted using a preset adjustment method to obtain an updated feature extraction model, the updated feature extraction model is used as a candidate feature extraction model, and the steps of training the pre-built machine learning model using multiple training samples in the training sample set are returned until the average similarity is greater than or equal to the similarity threshold.

[0045] If the average similarity is greater than or equal to the similarity threshold, then the candidate feature extraction model is determined as the context feature extraction model.

[0046] Optionally, classifying the text feature set to obtain a feature type set includes:

[0047] Identify model application scenarios;

[0048] A set of 3D generation requirements is determined based on the application scenario of the model, wherein the set of 3D generation requirements includes multiple 3D generation requirements;

[0049] For each 3D generation requirement in the set of 3D generation requirements, the following operation is performed:

[0050] The 3D generation requirements are analyzed to obtain initial feature labels;

[0051] The initial feature labels are summarized to obtain the initial feature label set;

[0052] The initial feature label set is subjected to clustering and normalization to obtain the feature label set;

[0053] A semantic feature identifier set is determined from the text feature set, wherein the semantic feature identifier set contains multiple semantic feature identifiers, and the semantic feature identifiers correspond one-to-one with the fused feature vectors in the text feature set;

[0054] Based on the feature label set, feature type matching is performed on the semantic feature identifier set to obtain the feature type set.

[0055] Optionally, obtaining the geometric feature set includes:

[0056] Obtain a model feature type set, wherein the model feature type set includes: basic structural features, size parameter features, and surface morphology features;

[0057] Based on the aforementioned basic structural features, a set of model topologies is determined, wherein the set of model topologies contains one or more model topologies;

[0058] A model size parameter set is obtained based on the size parameter features, wherein the model size parameter set includes one or more model size parameters;

[0059] The surface morphology features are transformed to obtain a surface morphology parameter set, wherein the surface morphology parameter set contains one or more surface morphology parameters.

[0060] A geometric feature set is constructed based on the model topology set, model size parameter set, and surface morphology parameter set.

[0061] Optionally, the step of using the feature fusion unit to perform hierarchical feature fusion of the feature type set and the geometric feature set to obtain a multimodal feature set includes:

[0062] The set of basic structure types, the set of size parameter types, and the set of surface morphology types are identified from the set of feature types. The set of basic structure types contains one or more basic structure types, the set of size parameter types contains one or more size parameter types, and the set of surface morphology types contains one or more surface morphology types.

[0063] Based on the set of basic structure types and the set of model topology structures in the set of geometric features, a set of structural similarities is calculated, wherein the set of structural similarities contains one or more structural similarities, and each structural similarity corresponds to a basic structure type and a model topology structure.

[0064] Based on the set of size parameter types and the set of model size parameters, a set of parameter matching degrees is calculated, wherein the set of parameter matching degrees contains one or more parameter matching degrees, and each parameter matching degree corresponds to a size parameter type and a model size parameter;

[0065] Based on the set of surface morphology types and the set of surface morphology parameters, a morphology similarity set is calculated, wherein the morphology similarity set contains one or more morphology similarities, and each morphology similarity corresponds to a surface morphology type and a surface morphology parameter.

[0066] Based on the structural similarity set, the feature fusion unit is used to fuse the basic structure type set and the model topology set to obtain the basic structure fusion feature vector set.

[0067] Based on the parameter matching degree set, parameter feature fusion is performed on the size parameter type set and the model size parameter set to obtain a size fusion feature vector set;

[0068] Based on the morphological similarity set, morphological feature fusion is performed on the surface morphological type set and the surface morphological parameter set to obtain a morphological fusion feature vector set.

[0069] The basic structure fusion feature vector set, size fusion feature vector set and morphology fusion feature vector set are aggregated to obtain a multimodal feature set.

[0070] Optionally, the step of performing parameter verification on the initial 3D model to obtain the target 3D model includes:

[0071] Based on the initial 3D model, obtain the initial model parameter set;

[0072] The target model parameter set is obtained based on the multimodal feature set;

[0073] Calculate the parameter deviation values ​​based on the initial model parameter set and the target model parameter set;

[0074] If the parameter deviation value is greater than or equal to the preset deviation threshold, the 3D generation model is adjusted according to the preset parameter adjustment method to obtain the adjusted 3D generation model. The adjusted 3D generation model is used as the 3D generation model, and the process of inputting the multimodal feature set into the preset 3D generation model to obtain the initial 3D model is returned until the parameter deviation value is less than the deviation threshold.

[0075] If the parameter deviation value is less than the deviation threshold, the initial 3D model is confirmed as the target 3D model.

[0076] To achieve the above objectives, the present invention also provides a text-to-3D generation architecture optimization system based on hierarchical feature fusion, comprising:

[0077] An environment configuration module is used to determine the 3D generation environment, wherein the 3D generation environment includes: a text processing unit, a feature processing unit, and a feature fusion unit;

[0078] The text processing module is used to acquire text data, perform preprocessing operations on the text data using the text processing unit to obtain a standard vocabulary type set, extract features from the standard vocabulary type set based on the feature processing unit to obtain a text feature set, and classify the text feature set to obtain a feature type set.

[0079] The feature fusion module is used to acquire a geometric feature set and, using the feature fusion unit, to perform hierarchical feature fusion of the feature type set and the geometric feature set to obtain a multimodal feature set.

[0080] The model generation module is used to input the multimodal feature set into a preset 3D generation model to obtain an initial 3D model, perform parameter verification on the initial 3D model to obtain a target 3D model, and complete the optimization of the text-to-3D generation architecture.

[0081] To address the above problems, the present invention also provides an electronic device, the electronic device comprising:

[0082] A memory that stores at least one instruction; and a processor that executes the instructions stored in the memory to implement the above-described text-to-3D generation architecture optimization method based on hierarchical feature fusion.

[0083] To address the aforementioned issues, the present invention also provides a computer-readable storage medium storing at least one instruction, which is executed by a processor in an electronic device to implement the above-described text-to-3D generation architecture optimization method based on hierarchical feature fusion.

[0084] To address the problems described in the background art, this invention first establishes a 3D generation environment, comprising a text processing unit, a feature processing unit, and a feature fusion unit. This environment acquires text data, gathering user-inputted text information to provide a data foundation for subsequent steps. The text processing unit preprocesses the text data to obtain a standard vocabulary type set. This preprocessing removes irrelevant data interference and standardizes the text data format, facilitating subsequent feature extraction. Based on the feature processing unit, features are extracted from the standard vocabulary type set to obtain a text feature set. This text feature set is then classified to obtain a feature type set and a geometric feature set. This invention maps standard vocabulary to low-dimensional word vectors using a word embedding model, constructs a word vector sequence based on the original text order, and semantically encodes the word vector sequence using a context feature extraction model to obtain the text feature set. This ensures that the obtained text feature set fully reflects the 3D requirements of the text description. By utilizing the feature fusion unit, the feature type set and the geometric feature set are fused hierarchically to obtain a multimodal feature set. This multimodal feature set is then input into a preset 3D generation model to obtain an initial 3D model. Parameter validation is performed on the initial 3D model to obtain the target 3D model, thus completing the optimization of the text-to-3D generation architecture. Therefore, this invention can solve the problem of low accuracy in the 3D model construction process. Attached Figure Description

[0085] Figure 1 This is a flowchart illustrating an embodiment of the text-to-3D generation architecture optimization method based on hierarchical feature fusion provided by the present invention.

[0086] Figure 2 A functional block diagram of a text-to-3D generation architecture optimization system based on hierarchical feature fusion provided in an embodiment of the present invention;

[0087] Figure 3 This is a schematic diagram of the structure of an electronic device that implements the text-to-3D generation architecture optimization method based on hierarchical feature fusion, according to an embodiment of the present invention.

[0088] Explanation of reference numerals in the attached figures:

[0089] 10. Electronic device; 11. Processor; 12. Memory; 13. Bus.

[0090] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0091] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0092] This application provides a text-to-3D generation architecture optimization method based on hierarchical feature fusion. The execution entity of this method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application: a server, a terminal, etc. In other words, the text-to-3D generation architecture optimization method based on hierarchical feature fusion can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster.

[0093] Reference Figure 1 The diagram shown is a flowchart illustrating a text-to-3D generation architecture optimization method based on hierarchical feature fusion according to an embodiment of the present invention. In this embodiment, the text-to-3D generation architecture optimization method based on hierarchical feature fusion includes:

[0094] S1. Determine the three-dimensional generation environment, wherein the three-dimensional generation environment includes: a text processing unit, a feature processing unit, and a feature fusion unit.

[0095] It should be noted that the text processing unit refers to the functional module used to perform preprocessing operations on the input text data, specifically text cleaning, word segmentation, and part-of-speech tagging. Its core function is to transform unstructured text data into a structured set of standard vocabulary types. The feature processing unit refers to the functional module used to extract semantic and contextual features from the set of standard vocabulary types. Its main functions include word embedding and context encoding, which can convert semantic information in text data into numerical vector form. The feature fusion unit refers to the functional module used to achieve cross-modal fusion of text features and geometric features. It supports hierarchical fusion logic based on basic structure, size parameters, and surface morphology. By calculating quantitative indicators such as feature similarity and parameter matching degree, it fuses the feature type set and the geometric feature set into a multimodal feature set, providing data support for the subsequent generation of the initial 3D model.

[0096] S2. Obtain text data, and use the text processing unit to preprocess the text data to obtain a standard vocabulary type set.

[0097] It should be explained that the text data refers to natural language statements or paragraphs used to drive the generation of 3D models, such as a red dining chair with four curved metal legs.

[0098] In detail, the preprocessing operation of the text data using the text processing unit to obtain a standard vocabulary type set includes:

[0099] The text processing unit is used to clean the text data to obtain standard text data.

[0100] The standard text data is segmented to obtain a text vocabulary set;

[0101] For each text word in the text vocabulary set, the following operation is performed:

[0102] The text words are marked with stop words according to a preset stop word library to obtain the marking results;

[0103] If the marking result is a preset stop mark, then the text words are marked as stop words;

[0104] The stop words are summarized to obtain a stop word set;

[0105] The stop word set is removed from the text vocabulary set to obtain the effective text vocabulary set;

[0106] Part-of-speech tagging is performed on the effective text vocabulary set to obtain a vocabulary type set;

[0107] The vocabulary type set is standardized according to a preset standardization method to obtain a standard vocabulary type set.

[0108] It is understandable that the data cleaning refers to the standardized processing of text data, specifically including: removing noise, removing duplicate characters, unifying formats, etc. The technologies required for the standardized processing are all existing technologies and will not be elaborated here. The standard text data refers to the text data after data processing. The word segmentation processing refers to the operation of splitting the continuous standard text data into multiple independent words according to semantic logic by using a word segmentation algorithm. Optionally, the maximum matching algorithm is used as the word segmentation algorithm. The text vocabulary set refers to the set composed of multiple words obtained after word segmentation processing. The stop word library refers to a database artificially set in advance and containing words without practical effects, including: auxiliary words (such as "de", "di", etc.), conjunctions (such as "he", "yu", etc.) and other words that have no substantial impact on the generation of three-dimensional models. The stop word marking refers to the process of comparing text vocabulary with the stop word library to obtain a marking result. The marking result refers to the result obtained after comparing text vocabulary with the stop word library, which is divided into stop marking and non-stop marking. The stop marking refers to a specific identifier indicating that the word is a stop word in the marking result, and the non-stop marking refers to a specific identifier indicating that the word is not a stop word in the marking result. The stop word refers to the text vocabulary that is the same as a certain word in the stop word library, indicating that the text vocabulary has no substantial impact on the generation of three-dimensional models and can be removed. The stop word set refers to the set composed of all stop words. The effective text vocabulary set refers to the set of all remaining words containing words that have a practical impact on the generation of three-dimensional models after removing the stop word set from the text vocabulary set. The standardizing the vocabulary type set according to a preset standardizing method to obtain a standard vocabulary type set means: using a standardizing method to unify the vocabulary types in different formats in the vocabulary type set, and the vocabulary type set after being uniformly formatted is the standard vocabulary type set. The standardizing method refers to the rule for unifying the format expression of the vocabulary type set. Optionally, the text normalization method is used as the standardizing method.

[0109] Specifically, the performing a词性标注 on the effective text vocabulary set to obtain a vocabulary type set includes:

[0110] Performing the following operations on each effective text vocabulary in the effective text vocabulary set:

[0111] Determining the semantic features of the effective text vocabulary;

[0112] Performing an attribute category matching on the effective text vocabulary according to the semantic features and a preset词性标注 method to obtain a vocabulary matching result;

[0113] Marking the effective text vocabulary according to the vocabulary matching result to obtain a vocabulary type;

[0114] Summarizing the vocabulary types to obtain a vocabulary type set. It should be noted that the term "词性标注" in the original text seems to be a specific Chinese term for a certain operation in text processing, and it may need to be accurately translated according to the specific context and professional knowledge. Here, a more general term "词性标注" is used for translation. If there is a more accurate English term for this operation, it should be replaced accordingly.

[0115] It should be understood that the semantic features refer to the information possessed by effective text words that reflects their semantic category and properties. This information can be obtained through semantic analysis of effective text words. For example, if an effective text word is "cylindrical shape," its semantic feature after semantic analysis is "cylindrical." The semantic analysis of effective text words is existing technology and will not be elaborated upon here. Optionally, semantic features can be obtained using a word vector cosine similarity calculation method. The specific process involves converting effective text words into vectors using a language model (such as Word2Vec), and then performing cosine similarity calculations between these vectors and the vectors of semantic features in a pre-defined semantic feature library to match the corresponding semantic features. This semantic feature library can be obtained from existing literature. The part-of-speech tagging method refers to the method for determining the word category corresponding to the semantic features of effective text words. Optionally, a sequence labeling algorithm can be used as the part-of-speech tagging algorithm. The word category refers to the functional classification of words in a grammatical system, including nouns, verbs, adjectives, etc. The attribute category matching refers to using a sequence labeling algorithm to determine the word category corresponding to the semantic features of effective text words. The vocabulary matching result refers to the correspondence between valid text words and vocabulary categories obtained after attribute category matching. Specifically, it is a key-value pair mapping of "valid text words - vocabulary categories," where the key is the valid text word and the value is the specific vocabulary category matched (e.g., "red" → "adjective," "cylinder" → "noun," etc.). Labeling the valid text words according to the vocabulary matching result to obtain vocabulary types means assigning corresponding vocabulary categories to valid text words based on the vocabulary matching result; the resulting combination of {valid text words, vocabulary categories} is the vocabulary type. The vocabulary type set refers to the collection composed of all vocabulary types.

[0116] S3. Based on the feature processing unit, feature extraction is performed on the standard vocabulary type set to obtain a text feature set.

[0117] Furthermore, based on the feature processing unit, feature extraction is performed on the standard vocabulary type set to obtain a text feature set, including:

[0118] Using a pre-built word embedding model, vector mapping is performed on the standard vocabulary type set to obtain a word vector set, wherein the word vector set contains multiple word vectors, and the word vectors correspond one-to-one with the standard vocabulary types in the standard vocabulary type set;

[0119] Obtain the original text order, and sort the word vector set according to the original text order to obtain the word vector sequence;

[0120] Obtain a context feature extraction model, and based on the context feature extraction model, perform semantic encoding on the word vector sequence to obtain a context feature vector set;

[0121] The context feature vector set is pooled to obtain a comprehensive text feature vector;

[0122] For each context feature vector in the set of context feature vectors, the following operation is performed:

[0123] Using the feature processing unit, the comprehensive text feature vector and the context feature vector are fused to obtain a fused feature vector;

[0124] The fused feature vectors are then combined to obtain the text feature set.

[0125] It should be noted that the word embedding model refers to a neural network model that can map input standard vocabulary types into low-dimensional vectors. Optionally, the Word2Vec model can be selected as the word embedding model. The vector mapping refers to the process of using the word embedding model to transform the standard vocabulary type set into low-dimensional vectors. The word vector set refers to the collection composed of all word vectors. The word vector refers to the low-dimensional numerical vector (such as a 100-dimensional vector, a 300-dimensional vector, etc.) generated after vector mapping for a certain vocabulary type. The original text order refers to the arrangement order of vocabulary types in the standard vocabulary type set, which is consistent with the word order in the user-input text data. The step of sorting the word vector set according to the original text order to obtain a word vector sequence means: rearranging the word vectors in the word vector set according to the original text order to generate an ordered sequence, which is the word vector sequence.

[0126] It should be explained that the context feature extraction model refers to a language model that captures semantic dependencies (such as modification, subordination, association, etc.) between words by analyzing the contextual relationships of word vectors in a word vector sequence, and transforms these dependencies into association information of word vectors. Optionally, the BERT model can be used as the context feature extraction model. Semantic encoding refers to the process of using the context feature extraction model to process the input word vector sequence and fuse each word vector with its contextual relationship. The context feature vector set refers to the set of word vectors after semantic encoding. The pooling operation refers to the process of aggregating information from all context feature vectors in the context feature vector set, extracting semantic information from multiple context feature vectors through specific strategies (such as mean pooling, max pooling, etc.) to generate a vector that can represent the semantics of the entire context feature vector set. The comprehensive text feature vector refers to the vector obtained after pooling the context feature vector set. Feature fusion refers to the process of weighted summation of the comprehensive text feature vector and the context feature vector. The weighted summation of the comprehensive text feature vector and the context feature vector is an existing technology and will not be elaborated here. The fused feature vector refers to an enhanced vector generated by fusing text feature vectors and individual context feature vectors, containing both the global semantics of the context feature vector set and the semantic information of the individual context feature vectors. For example, given a word vector sequence: "a red wooden chair," the fused feature vector for the word "red" includes both the global concept of the chair as a whole and emphasizes the local detail of red as a surface attribute of the object. The text feature set refers to the collection composed of all fused feature vectors.

[0127] In detail, the context feature extraction model includes:

[0128] Obtain multiple word vector sequence samples;

[0129] The multiple word vector sequence samples are divided according to a preset division ratio to obtain a training sample set and a validation sample set, wherein the training sample set contains multiple training samples and the validation sample set contains multiple validation samples.

[0130] A pre-built machine learning model is trained using multiple training samples from the training sample set to obtain a candidate feature extraction model.

[0131] Multiple verification feature vectors are obtained based on multiple verification samples in the verification sample set, wherein each verification feature vector corresponds one-to-one with a verification sample.

[0132] The candidate feature extraction model is validated using multiple validation samples, resulting in multiple predicted feature vectors, each corresponding one-to-one with a validation sample.

[0133] The average similarity is calculated based on the multiple predicted feature vectors and multiple verification feature vectors.

[0134] If the average similarity is less than a preset similarity threshold, the hyperparameters of the candidate feature extraction model are obtained, the hyperparameters are adjusted using a preset adjustment method to obtain an updated feature extraction model, the updated feature extraction model is used as a candidate feature extraction model, and the steps of training the pre-built machine learning model using multiple training samples in the training sample set are returned until the average similarity is greater than or equal to the similarity threshold.

[0135] If the average similarity is greater than or equal to the similarity threshold, then the candidate feature extraction model is determined as the context feature extraction model.

[0136] It is understood that the word vector sequence samples refer to the word vector sequences used for training and validating the machine learning model. The partition ratio refers to the proportion used to divide multiple word vector sequence samples, for example, 65% for the training sample set and 35% for the validation sample set. The training samples refer to the word vector sequence samples used for training the machine learning model parameters. The validation samples refer to the word vector sequence samples used to evaluate the performance of the candidate feature extraction model. The training sample set refers to the set composed of training samples. The validation sample set refers to the set composed of validation samples. The machine learning model refers to the model architecture used to extract contextual semantic features from the word vector sequence; optionally, the BERT model is used as the machine learning model. The candidate feature extraction model refers to the machine learning model after training. Training refers to iteratively optimizing the parameters of the machine learning model using the training sample set, enabling the machine learning model to extract contextual features of word vectors from the word vector sequence. The validation feature vector refers to the contextual feature vector corresponding to the validation sample. Validation refers to the process of inputting the validation sample into the candidate feature extraction model and evaluating the performance of the candidate feature extraction model (i.e., the accuracy of extracting contextual features of word vectors) by comparing the predicted feature vector output by the model with the validation feature vector. The predicted feature vector refers to the vector output by the candidate feature extraction model after the validation sample is input into the candidate feature extraction model. The average similarity refers to the arithmetic mean of the similarities between all predicted feature vectors and their corresponding validation feature vectors. The similarity is a quantitative indicator used to measure the semantic similarity between the predicted and validation feature vectors; optionally, cosine similarity is used, and its calculation formula is as follows: ,in, Represents the predicted feature vector. This represents the verification feature vector. Represents cosine similarity. This represents the L2 norm of the predicted feature vector. This represents the L2 norm of the verification feature vector.

[0137] It should be understood that the similarity threshold refers to a pre-defined critical value for similarity. When the average similarity is greater than or equal to the similarity threshold, it indicates that the performance of the candidate feature extraction model has reached expectations, and the candidate feature extraction model can be determined as the context feature extraction model. The hyperparameters refer to the parameters of the machine learning model that are pre-defined before training begins (the hyperparameters are not automatically adjusted during training). The adjustment method refers to the method of obtaining optimized hyperparameters from the pre-defined hyperparameter combinations. Optionally, grid search can be used as the adjustment method. The hyperparameter combinations can be obtained from existing literature. Updating the prediction model refers to the candidate feature extraction model adjusted by the adjustment method.

[0138] S4. Classify the text feature set to obtain a feature type set.

[0139] Specifically, classifying the text feature set to obtain a feature type set includes:

[0140] Identify model application scenarios;

[0141] A set of 3D generation requirements is determined based on the application scenario of the model, wherein the set of 3D generation requirements includes multiple 3D generation requirements;

[0142] For each 3D generation requirement in the set of 3D generation requirements, the following operation is performed:

[0143] The 3D generation requirements are analyzed to obtain initial feature labels;

[0144] The initial feature labels are summarized to obtain the initial feature label set;

[0145] The initial feature label set is subjected to clustering and normalization to obtain the feature label set;

[0146] A semantic feature identifier set is determined from the text feature set, wherein the semantic feature identifier set contains multiple semantic feature identifiers, and the semantic feature identifiers correspond one-to-one with the fused feature vectors in the text feature set;

[0147] Based on the feature label set, feature type matching is performed on the semantic feature identifier set to obtain the feature type set.

[0148] It should be explained that the "model application scenario" refers to the application field or scenario of the generated 3D model. The "3D generation requirement" refers to the functions that the user expects the 3D model to fulfill. The "3D generation requirement set" refers to the set composed of all 3D generation requirements. The "feature analysis" refers to the process of parsing the 3D generation requirements and extracting the key characteristics or attributes that can characterize those requirements. Parsing the 3D generation requirements is a prior art technique; optionally, a sequence labeling algorithm can be used as the method for parsing the 3D generation requirements. The "initial feature label" refers to the key characteristics or attributes of the 3D generation requirements obtained after feature extraction, such as: structure type, surface morphology, size parameters, etc. The "initial feature label set" refers to the set composed of all initial feature labels. The "clustering and normalization processing" refers to the process of using a clustering algorithm (such as the K-means algorithm) to merge semantically similar or identical initial feature labels in the initial feature label set and unify the label representation format. Optionally, a lexical reconstruction method can be used as the method for unifying the label representation format. The "feature label set" refers to the initial feature label set after clustering and normalization processing. The semantic feature identifier refers to the identifier information extracted from the fused feature vector in the text feature set. This identifier information represents the overall semantics of the fused feature vector. Extracting identifier information from the fused feature vector in the text feature set is an existing technique. Optionally, principal component analysis can be used as the method for extracting identifier information from the fused feature vector in the text feature set. The semantic feature identifier set refers to the collection of all semantic feature identifiers. The feature type matching refers to the process of matching each semantic feature identifier in the semantic feature identifier set with a feature label set using a pre-defined matching method (i.e., determining a corresponding feature label for each semantic feature identifier). Optionally, cosine similarity calculation can be used as the matching method. The specific matching process is consistent with the specific process of obtaining semantic features using the word vector cosine similarity calculation method described above, and will not be repeated here. The feature type set refers to the collection of multiple feature types, each feature type corresponding to a semantic feature identifier and its matched feature label.

[0149] S5. Obtain the geometric feature set, and use the feature fusion unit to perform hierarchical feature fusion of the feature type set and the geometric feature set to obtain a multimodal feature set.

[0150] Furthermore, the acquisition of the geometric feature set includes:

[0151] Obtain a model feature type set, wherein the model feature type set includes: basic structural features, size parameter features, and surface morphology features;

[0152] Based on the aforementioned basic structural features, a set of model topologies is determined, wherein the set of model topologies contains one or more model topologies;

[0153] A model size parameter set is obtained based on the size parameter features, wherein the model size parameter set includes one or more model size parameters;

[0154] The surface morphology features are transformed to obtain a surface morphology parameter set, wherein the surface morphology parameter set contains one or more surface morphology parameters.

[0155] A geometric feature set is constructed based on the model topology set, model size parameter set, and surface morphology parameter set.

[0156] It should be understood that the model feature type set refers to a pre-defined set of feature types used in the 3D model generation process, specifically including: basic structural features, dimensional parameter features, and surface morphology features. The basic structural features refer to feature categories used to define the geometric shape of the 3D model, such as solids of revolution (e.g., cylinders, cones), polyhedral structures (e.g., cubes, triangular pyramids), etc. The dimensional parameter features refer to feature categories used to quantify the spatial size of the 3D model, such as length, diameter, and height. The surface morphology features refer to feature categories used to describe the surface shape, texture, and other appearance geometric attributes of the 3D model. The model topology refers to graph structure data representing the connections and compositional relationships between 3D model components, generated based on the basic structural features through 3D shape grammar or graph neural networks. The model topology set refers to the set composed of all model topologies. The model parameter dimensions refer to the specific numerical information extracted from the dimensional parameter features used to describe the spatial structural size of the 3D model, for example, a cylinder with a diameter of 5cm. The model parameter dimension set refers to the set composed of all model parameter dimensions. The parameter transformation refers to the process of converting surface morphological features into quantifiable parameters recognizable by a 3D modeling system. The method for converting surface morphological features into quantifiable parameters recognizable by a 3D modeling system is existing technology. Optionally, mesh parameterization can be used as the method for converting surface morphological features into quantifiable parameters recognizable by a 3D modeling system. The surface morphological parameters refer to the specific parameters obtained after parameter transformation, used to quantify and describe the appearance attributes of the model surface, such as a surface roughness of Ra 1.6 μm and a texture resolution of 2048×2048. The surface morphological parameter set refers to the set composed of all surface morphological parameters. Constructing a geometric feature set based on the model topology set, model size parameter set, and surface morphological parameter set means: summarizing and integrating the model topology set, model size parameter set, and surface morphological parameter set into a single set, which is the geometric feature set.

[0157] In detail, the feature fusion unit is used to perform hierarchical feature fusion of the feature type set and the geometric feature set to obtain a multimodal feature set, including:

[0158] The set of basic structure types, the set of size parameter types, and the set of surface morphology types are identified from the set of feature types. The set of basic structure types contains one or more basic structure types, the set of size parameter types contains one or more size parameter types, and the set of surface morphology types contains one or more surface morphology types.

[0159] Based on the set of basic structure types and the set of model topology structures in the set of geometric features, a set of structural similarities is calculated, wherein the set of structural similarities contains one or more structural similarities, and each structural similarity corresponds to a basic structure type and a model topology structure.

[0160] Based on the set of size parameter types and the set of model size parameters, a set of parameter matching degrees is calculated, wherein the set of parameter matching degrees contains one or more parameter matching degrees, and each parameter matching degree corresponds to a size parameter type and a model size parameter;

[0161] Based on the set of surface morphology types and the set of surface morphology parameters, a morphology similarity set is calculated, wherein the morphology similarity set contains one or more morphology similarities, and each morphology similarity corresponds to a surface morphology type and a surface morphology parameter.

[0162] Based on the structural similarity set, the feature fusion unit is used to fuse the basic structure type set and the model topology set to obtain the basic structure fusion feature vector set.

[0163] Based on the parameter matching degree set, parameter feature fusion is performed on the size parameter type set and the model size parameter set to obtain a size fusion feature vector set;

[0164] Based on the morphological similarity set, morphological feature fusion is performed on the surface morphological type set and the surface morphological parameter set to obtain a morphological fusion feature vector set.

[0165] The basic structure fusion feature vector set, size fusion feature vector set and morphology fusion feature vector set are aggregated to obtain a multimodal feature set.

[0166] It is understood that the "basic structure type" refers to the feature type extracted from the feature type set that is related to basic structures (such as cylinders, cuboids, etc.). The "basic structure type set" refers to the set consisting of all basic structure types. The "size parameter type" refers to the feature type extracted from the feature type set that is related to size parameters (such as length, height, etc.). The "size parameter type set" refers to the set consisting of all size parameters. The "surface morphology type" refers to the feature type extracted from the feature type set that is related to surface morphology (such as roughness, texture resolution, etc.). The "surface morphology type set" refers to the set consisting of all surface morphology types.

[0167] It should be understood that the structural similarity refers to the degree of semantic and structural matching between the basic structure type and the model topology; a higher value indicates a better match between the basic structure type and the model topology. The calculation method for structural similarity is the same as that for overall similarity, and will not be repeated here. The structural similarity set refers to the collection of all structural similarities. The parameter matching degree refers to the degree of matching between the size parameter type and the model size parameters; a higher value indicates a better match between the size parameter type and the model size parameters. The parameter matching degree is calculated as follows: The parameter matching degree set refers to the set composed of all parameter matching degrees. The morphological similarity refers to the degree of similarity between surface morphological types and surface morphological parameters; a higher value indicates greater similarity. The calculation method for morphological similarity is the same as that for structural similarity, and will not be elaborated further here. The morphological similarity set refers to the set composed of all morphological similarities. The structural feature fusion refers to the process of weighted integration of basic structural types and model topology (with structural similarity as the weight). The method for weighted integration of basic structural types and model topology is existing technology. Optionally, a cross-modal feature fusion method can be used as the method for weighted integration of basic structural types and model topology. The basic structural fusion feature vector refers to the set of comprehensive vectors obtained after structural feature fusion that can comprehensively represent the relationship between basic structural types and corresponding model topology. The parameter feature fusion refers to the process of integrating size parameter types and model size parameters. The method for integrating size parameter types and model size parameters is the same as the method for weighted integration of basic structural types and model topology, and will not be elaborated further here. The size fusion feature vector set refers to the combination of comprehensive vectors obtained after parameter feature fusion, which can comprehensively represent the relationship between size parameter types and corresponding model size parameters. The morphological feature fusion refers to the process of integrating surface morphological types with surface morphological parameters. The method for integrating surface morphological types with surface morphological parameter sets is consistent with the method for weighted integration of basic structure types and model topology, and will not be elaborated further here. The morphological fusion feature vector set refers to the set of comprehensive vectors obtained after morphological feature fusion, which can comprehensively represent the relationship between surface morphological types and corresponding surface morphological parameters. The feature aggregation refers to the process of integrating the basic structure fusion feature vector, size fusion feature vector, and morphological fusion feature vector into a unified vector through vector concatenation, for example, concatenating three 128-dimensional vectors into a 384-dimensional vector. The multimodal feature set refers to the set of multimodal features obtained after feature aggregation of the basic structure fusion feature vector set, size fusion feature vector set, and morphological fusion feature vector set, where each multimodal feature is the unified vector.

[0168] S6. Input the multimodal feature set into a preset three-dimensional generation model to obtain an initial three-dimensional model.

[0169] It should be noted that the 3D generative model refers to a generative architecture based on deep learning, capable of mapping multimodal features to a 3D geometric representation. Optionally, a diffusion model can be used as the 3D generative model. The initial 3D model refers to the 3D model obtained after inputting the multimodal feature set into the 3D generative model.

[0170] S7. Perform parameter verification on the initial 3D model to obtain the target 3D model and complete the optimization of the text-to-3D generation architecture.

[0171] It should be understood that the parameter verification of the initial 3D model to obtain the target 3D model includes:

[0172] Based on the initial 3D model, obtain the initial model parameter set;

[0173] The target model parameter set is obtained based on the multimodal feature set;

[0174] Calculate the parameter deviation values ​​based on the initial model parameter set and the target model parameter set;

[0175] If the parameter deviation value is greater than or equal to the preset deviation threshold, the 3D generation model is adjusted according to the preset parameter adjustment method to obtain the adjusted 3D generation model. The adjusted 3D generation model is used as the 3D generation model, and the process of inputting the multimodal feature set into the preset 3D generation model to obtain the initial 3D model is returned until the parameter deviation value is less than the deviation threshold.

[0176] If the parameter deviation value is less than the deviation threshold, the initial 3D model is confirmed as the target 3D model.

[0177] It should be explained that the initial model parameter set refers to the set of parameters extracted from the initial 3D model to quantify its geometric properties, including: structural parameters (such as topology), dimensional parameters (such as length and diameter), and surface morphology parameters (such as roughness and texture resolution). The target model parameter set refers to the set of ideal parameters that the target 3D model should possess, parsed from the multimodal feature set, and the parameters in this set correspond to those in the initial model parameter set (such as structural parameters, dimensional parameters, and surface morphology parameters). The parameter deviation value is a quantitative indicator of the degree of difference between the initial model parameter set and the target model parameter set; a larger parameter deviation value indicates a greater deviation between the two sets. The parameter deviation value is calculated as follows: ,in, This represents the total number of parameters in the target model parameter set and the initial model parameter set. This represents the j-th parameter in the initial model parameter set. This represents the j-th parameter in the target model parameter set. Represents the absolute value symbol.

[0178] In detail, the deviation threshold refers to a pre-set critical value for the deviation. When the parameter deviation value is less than the deviation threshold, it means that the accuracy of the initial 3D model meets the user's expectations. The parameter adjustment method refers to a technical method for optimizing the 3D generated model when the parameter deviation value is greater than or equal to the deviation threshold. Optionally, the method for adjusting the hyperparameters in the 3D generated model described above can be used as the parameter adjustment method. The method for adjusting the hyperparameters in the 3D generated model is consistent with the adjustment method and will not be repeated here. The adjusted 3D generated model refers to the 3D generated model after being adjusted by the parameter adjustment method. The target 3D model refers to the initial 3D model whose parameter deviation value is less than the deviation threshold. When the parameter deviation value is less than the deviation threshold, it means that the accuracy of the initial 3D model meets the user's requirements, and the initial 3D model can be determined as the target 3D model.

[0179] To address the problems described in the background art, this invention first establishes a 3D generation environment, comprising a text processing unit, a feature processing unit, and a feature fusion unit. This environment acquires text data, gathering user-inputted text information to provide a data foundation for subsequent steps. The text processing unit preprocesses the text data to obtain a standard vocabulary type set. This preprocessing removes irrelevant data interference and standardizes the text data format, facilitating subsequent feature extraction. Based on the feature processing unit, features are extracted from the standard vocabulary type set to obtain a text feature set. This text feature set is then classified to obtain a feature type set and a geometric feature set. This invention maps standard vocabulary to low-dimensional word vectors using a word embedding model, constructs a word vector sequence based on the original text order, and semantically encodes the word vector sequence using a context feature extraction model to obtain the text feature set. This ensures that the obtained text feature set fully reflects the 3D requirements of the text description. By utilizing the feature fusion unit, the feature type set and the geometric feature set are fused hierarchically to obtain a multimodal feature set. This multimodal feature set is then input into a preset 3D generation model to obtain an initial 3D model. Parameter validation is performed on the initial 3D model to obtain the target 3D model, thus completing the optimization of the text-to-3D generation architecture. Therefore, this invention can solve the problem of low accuracy in the 3D model construction process.

[0180] like Figure 2 The diagram shown is a functional block diagram of a text-to-3D generation architecture optimization system based on hierarchical feature fusion provided in an embodiment of the present invention.

[0181] The text-to-3D generation architecture optimization system 100 based on hierarchical feature fusion described in this invention can be installed in an electronic device. Depending on the functions implemented, the text-to-3D generation architecture optimization system 100 may include an environment configuration module 101, a text processing module 102, a feature fusion module 103, and a model generation module 104. The module described in this invention can also be referred to as a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and which are stored in the memory of the electronic device.

[0182] The environment configuration module 101 is used to determine the three-dimensional generation environment, wherein the three-dimensional generation environment includes: a text processing unit, a feature processing unit, and a feature fusion unit;

[0183] The text processing module 102 is used to acquire text data, perform preprocessing operations on the text data using the text processing unit to obtain a standard vocabulary type set, extract features from the standard vocabulary type set based on the feature processing unit to obtain a text feature set, and classify the text feature set to obtain a feature type set.

[0184] The feature fusion module 103 is used to acquire a geometric feature set and use the feature fusion unit to perform hierarchical feature fusion of the feature type set and the geometric feature set to obtain a multimodal feature set;

[0185] The model generation module 104 is used to input the multimodal feature set into a preset three-dimensional generation model to obtain an initial three-dimensional model, perform parameter verification on the initial three-dimensional model to obtain a target three-dimensional model, and complete the optimization of the text-to-three-dimensional generation architecture.

[0186] In detail, the modules in the text-to-3D generation architecture optimization system 100 based on hierarchical feature fusion described in this embodiment of the invention employ the same methods as described above. Figure 1 The method uses the same techniques as the text-to-3D generation architecture optimization method based on hierarchical feature fusion described in the article, and can produce the same technical effects, so it will not be elaborated here.

[0187] like Figure 3 The diagram shown is a structural schematic of an electronic device that implements a text-to-3D generation architecture optimization method based on hierarchical feature fusion, according to an embodiment of the present invention.

[0188] The electronic device 1 may include a processor 10, a memory 11 and a bus 12, and may also include a computer program stored in the memory 11 and capable of running on the processor 10, such as a text-to-3D generation architecture optimization method program based on hierarchical feature fusion.

[0189] The memory 11 includes at least one type of readable storage medium, such as flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of the electronic device 1, such as a portable hard drive. In other embodiments, the memory 11 can be an external storage device of the electronic device 1, such as a plug-in portable hard drive, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the electronic device 1. Furthermore, the memory 11 includes both internal storage units and external storage devices of the electronic device 1. The memory 11 can be used not only to store application software and various types of data installed on the electronic device 1, such as the code of a text-to-3D generation architecture optimization method program based on hierarchical feature fusion, but also to temporarily store data that has been output or will be output.

[0190] In some embodiments, the processor 10 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 10 is the control unit of the electronic device, connecting various components of the entire electronic device through various interfaces and lines. It executes programs or modules stored in the memory 11 (e.g., a text-to-3D generation architecture optimization method program based on hierarchical feature fusion) and calls data stored in the memory 11 to perform various functions of the electronic device 1 and process data.

[0191] The bus 12 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus 12 can be divided into an address bus, a data bus, a control bus, etc. The bus 12 is configured to realize the connection and communication between the memory 11 and at least one processor 10, etc.

[0192] Figure 3 Only electronic devices with components are shown; it will be understood by those skilled in the art that... Figure 3The structure shown does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0193] For example, although not shown, the electronic device 1 may also include a power supply (such as a battery) to power the various components. Preferably, the power supply can be logically connected to the at least one processor 10 through a power management device, thereby enabling functions such as charging management, discharging management, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device 1 may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0194] Furthermore, the electronic device 1 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, a Bluetooth interface, etc.), which is typically used to establish communication connections between the electronic device 1 and other electronic devices.

[0195] Optionally, the electronic device 1 may further include a user interface, which may be a display, an input unit (such as a keyboard), and optionally, a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the electronic device 1 and to display a visual user interface.

[0196] The text-to-3D generation architecture optimization method program based on hierarchical feature fusion stored in the memory 11 of the electronic device 1 is a combination of multiple instructions. When run in the processor 10, it can achieve the following:

[0197] A 3D generation environment is defined, wherein the 3D generation environment includes: a text processing unit, a feature processing unit, and a feature fusion unit;

[0198] The text data is acquired, and the text processing unit is used to preprocess the text data to obtain a standard vocabulary type set.

[0199] Based on the feature processing unit, feature extraction is performed on the standard vocabulary type set to obtain a text feature set;

[0200] The text feature set is classified to obtain a feature type set;

[0201] Obtain a geometric feature set, and use the feature fusion unit to perform hierarchical feature fusion with the feature type set and the geometric feature set to obtain a multimodal feature set;

[0202] The multimodal feature set is input into a preset 3D generation model to obtain an initial 3D model;

[0203] The parameters of the initial 3D model are validated to obtain the target 3D model, thus completing the optimization of the text-to-3D generation architecture.

[0204] Specifically, the processor 10's implementation method for the above instructions can be found in [reference needed]. Figures 1 to 3 The descriptions of the relevant steps in the corresponding embodiments are not repeated here.

[0205] Furthermore, if the modules / units integrated in the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0206] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor of an electronic device, can perform the following:

[0207] A 3D generation environment is defined, wherein the 3D generation environment includes: a text processing unit, a feature processing unit, and a feature fusion unit;

[0208] The text data is acquired, and the text processing unit is used to preprocess the text data to obtain a standard vocabulary type set.

[0209] Based on the feature processing unit, feature extraction is performed on the standard vocabulary type set to obtain a text feature set;

[0210] The text feature set is classified to obtain a feature type set;

[0211] Obtain a geometric feature set, and use the feature fusion unit to perform hierarchical feature fusion with the feature type set and the geometric feature set to obtain a multimodal feature set;

[0212] The multimodal feature set is input into a preset 3D generation model to obtain an initial 3D model;

[0213] The parameters of the initial 3D model are validated to obtain the target 3D model, thus completing the optimization of the text-to-3D generation architecture.

[0214] In the embodiments provided by this invention, it should be understood that the disclosed devices, systems, and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative, and actual implementations may have other classification methods.

[0215] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0216] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0217] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0218] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A text-to-three-dimensional generation architecture optimization method based on hierarchical feature fusion, characterized in that, The method comprises: determining a three-dimensional generation environment, wherein the three-dimensional generation environment comprises a text processing unit, a feature processing unit and a feature fusion unit; obtaining text data, and performing a preprocessing operation on the text data by using the text processing unit to obtain a standard vocabulary type set; based on the feature processing unit, performing feature extraction on the standard vocabulary type set to obtain a text feature set; classifying the text feature set to obtain a feature type set; obtaining a geometric feature set, and performing hierarchical feature fusion on the feature type set and the geometric feature set by using the feature fusion unit to obtain a multi-modal feature set, comprising: confirming a basic structure type set, a size parameter type set and a surface morphology type set from the feature type set, wherein the basic structure type set contains one or more basic structure types, the size parameter type set contains one or more size parameter types, and the surface morphology type set contains one or more surface morphology types; based on the basic structure type set and a model topology structure set in the geometric feature set, calculating a structure similarity set, wherein the structure similarity set contains one or more structure similarities, and each structure similarity corresponds to a basic structure type and a model topology structure; based on the size parameter type set and a model size parameter set, calculating a parameter matching degree set, wherein the parameter matching degree set contains one or more parameter matching degrees, and each parameter matching degree corresponds to a size parameter type and a model size parameter; based on the surface morphology type set and a surface morphology parameter set, calculating a morphology similarity set, wherein the morphology similarity set contains one or more morphology similarities, and each morphology similarity corresponds to a surface morphology type and a surface morphology parameter; performing structure feature fusion on the basic structure type set and the model topology structure set by using the feature fusion unit according to the structure similarity set, to obtain a basic structure fusion feature vector set; performing parameter feature fusion on the size parameter type set and the model size parameter set according to the parameter matching degree set, to obtain a size fusion feature vector set; performing morphology feature fusion on the surface morphology type set and the surface morphology parameter set according to the morphology similarity set, to obtain a morphology fusion feature vector set; performing feature aggregation on the basic structure fusion feature vector set, the size fusion feature vector set and the morphology fusion feature vector set, to obtain a multi-modal feature set; inputting the multi-modal feature set into a preset three-dimensional generation model to obtain an initial three-dimensional model; performing parameter verification on the initial three-dimensional model to obtain a target three-dimensional model, and completing text-to-three-dimensional generation architecture optimization.

2. The hierarchical feature fusion based text-to-3D generation architecture optimization method of claim 1, wherein, The preprocessing operation on the text data by using the text processing unit to obtain a standard vocabulary type set comprises: performing data cleaning on the text data by using the text processing unit to obtain standard text data; performing word segmentation processing on the standard text data to obtain a text vocabulary set; performing the following operations on each text vocabulary in the text vocabulary set: performing stop word marking on the text vocabulary according to a preset stop word library to obtain a marking result; If the mark result is a preset stop mark, mark the text vocabulary as a stop word; aggregate the stop words to obtain a stop word set; remove the stop word set from the text vocabulary set to obtain an effective text vocabulary set; perform part-of-speech tagging on the effective text vocabulary set to obtain a vocabulary type set; perform standardization processing on the vocabulary type set according to a preset standardization method to obtain a standard vocabulary type set.

3. The text-to-3D generation architecture optimization method based on hierarchical feature fusion of claim 2, wherein, The part-of-speech tagging on the effective text vocabulary set to obtain a vocabulary type set comprises: perform the following operations on each effective text vocabulary in the effective text vocabulary set: determine the semantic features of the effective text vocabulary; perform attribute category matching on the effective text vocabulary according to the semantic features and a preset part-of-speech tagging method to obtain a vocabulary matching result; mark the effective text vocabulary according to the vocabulary matching result to obtain a vocabulary type; aggregate the vocabulary types to obtain a vocabulary type set.

4. The hierarchical feature fusion based text-to-3D generation architecture optimization method of claim 3, wherein, The feature extraction on the standard vocabulary type set based on the feature processing unit to obtain a text feature set comprises: perform vector mapping on the standard vocabulary type set by using a pre-constructed word embedding model to obtain a word vector set, wherein the word vector set comprises a plurality of word vectors, and each word vector corresponds to a standard vocabulary type in the standard vocabulary type set; obtain an original text order, sort the word vector set according to the original text order to obtain a word vector sequence; obtain a context feature extraction model, and perform semantic coding on the word vector sequence based on the context feature extraction model to obtain a context feature vector set; perform a pooling operation on the context feature vector set to obtain a comprehensive text feature vector; perform the following operations on each context feature vector in the context feature vector set: perform feature fusion on the comprehensive text feature vector and the context feature vector by using the feature processing unit to obtain a fused feature vector; aggregate the fused feature vectors to obtain a text feature set.

5. The hierarchical feature fusion based text-to-3D generation architecture optimization method of claim 4, wherein, The obtaining of the context feature extraction model comprises: obtain a plurality of word vector sequence samples; divide the plurality of word vector sequence samples according to a preset division ratio to obtain a training sample set and a verification sample set, wherein the training sample set comprises a plurality of training samples, and the verification sample set comprises a plurality of verification samples; train a pre-constructed machine learning model by using the plurality of training samples in the training sample set to obtain a candidate feature extraction model; obtain a plurality of verification feature vectors based on the plurality of verification samples in the verification sample set, wherein each verification feature vector corresponds to a verification sample; verify the candidate feature extraction model by using the plurality of verification samples to obtain a plurality of predicted feature vectors, wherein each predicted feature vector corresponds to a verification sample; calculate an average similarity according to the plurality of predicted feature vectors and the plurality of verification feature vectors; If the average similarity is less than a preset similarity threshold, a hyperparameter of the candidate feature extraction model is obtained, the hyperparameter is adjusted using a preset adjustment method to obtain an updated feature extraction model, the updated feature extraction model is taken as the candidate feature extraction model, and the step of training the pre-constructed machine learning model using the plurality of training samples in the training sample set is returned until the average similarity is greater than or equal to the similarity threshold; If the average similarity is greater than or equal to the similarity threshold, the candidate feature extraction model is determined as the context feature extraction model.

6. The hierarchical feature fusion based text-to-3D generation architecture optimization method of claim 5, wherein, The classification of the text feature set to obtain a feature type set comprises: obtaining a model application scenario; determining a three-dimensional generation requirement set according to the model application scenario, wherein the three-dimensional generation requirement set comprises a plurality of three-dimensional generation requirements; performing the following operations on each three-dimensional generation requirement in the three-dimensional generation requirement set: performing feature analysis on the three-dimensional generation requirement to obtain an initial feature label; summarizing the initial feature labels to obtain an initial feature label set; performing clustering normalization processing on the initial feature label set to obtain a feature label set; determining a semantic feature identifier set from the text feature set, wherein the semantic feature identifier set comprises a plurality of semantic feature identifiers, and each semantic feature identifier corresponds to a fusion feature vector in the text feature set; performing feature type matching on the semantic feature identifier set based on the feature label set to obtain a feature type set.

7. The hierarchical feature fusion based text-to-3D generation architecture optimization method of claim 6, wherein, The obtaining of the geometric feature set comprises: obtaining a model feature type set, wherein the model feature type set comprises a basic structure feature, a size parameter feature, and a surface morphology feature; determining a model topology structure set according to the basic structure feature, wherein the model topology structure set comprises one or more model topology structures; obtaining a model size parameter set according to the size parameter feature, wherein the model size parameter set comprises one or more model size parameters; performing parameter conversion on the surface morphology feature to obtain a surface morphology parameter set, wherein the surface morphology parameter set comprises one or more surface morphology parameters; constructing a geometric feature set according to the model topology structure set, the model size parameter set, and the surface morphology parameter set.

8. The hierarchical feature fusion based text-to-3D generation architecture optimization method of claim 7, wherein, The parameter verification of the initial three-dimensional model to obtain a target three-dimensional model comprises: obtaining an initial model parameter set based on the initial three-dimensional model; obtaining a target model parameter set according to the multi-modal feature set; calculating a parameter deviation value according to the initial model parameter set and the target model parameter set; if the parameter deviation value is greater than or equal to a preset deviation threshold, adjusting the three-dimensional generation model according to a preset parameter adjustment method to obtain an adjusted three-dimensional generation model, taking the adjusted three-dimensional generation model as the three-dimensional generation model, and returning to the step of inputting the multi-modal feature set into the preset three-dimensional generation model to obtain the initial three-dimensional model until the parameter deviation value is less than the deviation threshold; if the parameter deviation value is less than the deviation threshold, confirming the initial three-dimensional model as the target three-dimensional model.

9. A hierarchical feature fusion based text-to-3D generation architecture optimization system for the method of any one of claims 1 to 8, characterized in that, The system comprises: An environment configuration module is configured to determine a three-dimensional generation environment, wherein the three-dimensional generation environment comprises a text processing unit, a feature processing unit, and a feature fusion unit; A text processing module is configured to obtain text data, perform a preprocessing operation on the text data by using the text processing unit to obtain a standard vocabulary type set, perform feature extraction on the standard vocabulary type set based on the feature processing unit to obtain a text feature set, classify the text feature set to obtain a feature type set; A feature fusion module is configured to obtain a geometric feature set, and perform hierarchical feature fusion on the feature type set and the geometric feature set by using the feature fusion unit to obtain a multi-modal feature set; A model generation module is configured to input the multi-modal feature set into a preset three-dimensional generation model to obtain an initial three-dimensional model, perform parameter verification on the initial three-dimensional model to obtain a target three-dimensional model, and complete architecture optimization from text to three-dimensional generation.

Images