Text-to-three-dimensional generation architecture optimization method and system based on hierarchical feature fusion

Abstract

The invention relates to the technical field of model construction, in particular to a method and system for optimizing a text to a three-dimensional generation architecture based on hierarchical feature fusion, and the method comprises the steps: obtaining text data, carrying out the preprocessing operation on the text data, obtaining a standard vocabulary type set, carrying out the feature extraction on the standard vocabulary type set, obtaining a text feature set, and classifying the text feature set, the method comprises the steps of obtaining a feature type set, obtaining a geometric feature set, carrying out hierarchical feature fusion on the feature type set and the geometric feature set to obtain a multi-modal feature set, inputting the multi-modal feature set into a three-dimensional generation model to obtain an initial three-dimensional model, carrying out parameter verification on the initial three-dimensional model to obtain a target three-dimensional model, and completing the optimization of the text-to-three-dimensional generation architecture. According to the invention, the problem of low precision in the construction process of the three-dimensional model can be solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of model construction, and in particular to a text-to-three-dimensional generation architecture optimization method and system based on hierarchical feature fusion. BACKGROUND

[0002] With the rapid development of artificial intelligence and computer graphics, three-dimensional models are increasingly widely used in virtual reality, intelligent manufacturing and other fields. Generating three-dimensional models based on text descriptions is an important method for generating three-dimensional models.

[0003] Traditional three-dimensional model generation methods mainly use images for modeling. Although image information can be used to generate three-dimensional models, it is not accurate enough to handle complex three-dimensional structures and details, resulting in insufficient model accuracy to meet actual needs. Therefore, there is a problem of low accuracy in the construction process of three-dimensional models. SUMMARY

[0004] The present application provides a text-to-three-dimensional generation architecture optimization method and system based on hierarchical feature fusion, which mainly aims to solve the problem of low accuracy in the construction process of three-dimensional models.

[0005] To achieve the above-mentioned purpose, the present application provides a text-to-three-dimensional generation architecture optimization method based on hierarchical feature fusion, which comprises:

[0006] determining a three-dimensional generation environment, wherein the three-dimensional generation environment comprises a text processing unit, a feature processing unit and a feature fusion unit;

[0007] obtaining text data, and using the text processing unit to perform a preprocessing operation on the text data to obtain a standard vocabulary type set;

[0008] based on the feature processing unit, performing feature extraction on the standard vocabulary type set to obtain a text feature set;

[0009] classifying the text feature set to obtain a feature type set;

[0010] obtaining a geometric feature set, and using the feature fusion unit to perform hierarchical feature fusion on the feature type set and the geometric feature set to obtain a multi-modal feature set;

[0011] inputting the multi-modal feature set into a predetermined three-dimensional generation model to obtain an initial three-dimensional model;

[0012] performing parameter verification on the initial three-dimensional model to obtain a target three-dimensional model, and completing the text-to-three-dimensional generation architecture optimization.

[0013] Optionally, the preprocessing operation on the text data by the text processing unit comprises:

[0014] The text data is cleaned by the text processing unit to obtain standard text data.

[0015] The standard text data is subjected to word segmentation processing to obtain a text vocabulary set.

[0016] Each text vocabulary in the text vocabulary set is subjected to the following operations:

[0017] The text vocabulary is subjected to stop word marking according to a preset stop word library to obtain a marking result.

[0018] If the marking result is a preset stop word marking, the text vocabulary is marked as a stop word.

[0019] The stop words are summarized to obtain a stop word set.

[0020] The stop word set is removed from the text vocabulary set to obtain an effective text vocabulary set.

[0021] The effective text vocabulary set is subjected to part-of-speech tagging to obtain a vocabulary type set.

[0022] The vocabulary type set is subjected to standardization processing according to a preset standardization method to obtain a standard vocabulary type set.

[0023] Optionally, the part-of-speech tagging on the effective text vocabulary set to obtain a vocabulary type set comprises:

[0024] Each effective text vocabulary in the effective text vocabulary set is subjected to the following operations:

[0025] The semantic features of the effective text vocabulary are determined.

[0026] The effective text vocabulary is subjected to attribute category matching according to the semantic features and a preset part-of-speech tagging method to obtain a vocabulary matching result.

[0027] The effective text vocabulary is marked according to the vocabulary matching result to obtain a vocabulary type.

[0028] The vocabulary types are summarized to obtain a vocabulary type set.

[0029] Optionally, the feature extraction on the standard vocabulary type set by the feature processing unit to obtain a text feature set comprises:

[0030] mapping the set of standard vocabulary types into a set of word vectors using a pre-constructed word embedding model, wherein the set of word vectors comprises a plurality of word vectors, and each word vector corresponds to a standard vocabulary type in the set of standard vocabulary types;

[0031] obtaining an original text order, and sorting the set of word vectors according to the original text order to obtain a sequence of word vectors;

[0032] obtaining a context feature extraction model, and performing semantic coding on the sequence of word vectors based on the context feature extraction model to obtain a set of context feature vectors;

[0033] performing a pooling operation on the set of context feature vectors to obtain a comprehensive text feature vector;

[0034] performing the following operations on each context feature vector in the set of context feature vectors:

[0035] performing feature fusion on the comprehensive text feature vector and the context feature vector using a feature processing unit to obtain a fused feature vector;

[0036] summarizing the fused feature vector to obtain a set of text features.

[0037] Optionally, the obtaining the context feature extraction model comprises:

[0038] obtaining a plurality of word vector sequence samples;

[0039] dividing the plurality of word vector sequence samples according to a preset division ratio to obtain a training sample set and a validation sample set, wherein the training sample set comprises a plurality of training samples, and the validation sample set comprises a plurality of validation samples;

[0040] training a pre-constructed machine learning model using the plurality of training samples in the training sample set to obtain a candidate feature extraction model;

[0041] obtaining a plurality of validation feature vectors based on the plurality of validation samples in the validation sample set, wherein each validation feature vector corresponds to a validation sample;

[0042] verifying the candidate feature extraction model using the plurality of validation samples to obtain a plurality of predicted feature vectors, wherein each predicted feature vector corresponds to a validation sample;

[0043] calculating an average similarity degree according to the plurality of predicted feature vectors and the plurality of validation feature vectors;

[0044] if the average similarity is less than a preset similarity threshold, obtaining hyperparameters of the candidate feature extraction model, adjusting the hyperparameters using a preset adjustment method to obtain an updated feature extraction model, taking the updated feature extraction model as the candidate feature extraction model, and returning to the step of training the pre-constructed machine learning model using the plurality of training samples in the training sample set until the average similarity is greater than or equal to the similarity threshold;

[0045] if the average similarity is greater than or equal to the similarity threshold, determining the candidate feature extraction model as the context feature extraction model.

[0046] Optionally, the classification of the text feature set to obtain a feature type set comprises:

[0047] obtaining a model application scenario;

[0048] determining a three-dimensional generation requirement set according to the model application scenario, wherein the three-dimensional generation requirement set comprises a plurality of three-dimensional generation requirements;

[0049] performing the following operations on each three-dimensional generation requirement in the three-dimensional generation requirement set:

[0050] performing feature analysis on the three-dimensional generation requirement to obtain an initial feature label;

[0051] summarizing the initial feature labels to obtain an initial feature label set;

[0052] performing clustering and normalization processing on the initial feature label set to obtain a feature label set;

[0053] determining a semantic feature identifier set from the text feature set, wherein the semantic feature identifier set comprises a plurality of semantic feature identifiers, and each semantic feature identifier corresponds to a fusion feature vector in the text feature set;

[0054] performing feature type matching on the semantic feature identifier set based on the feature label set to obtain a feature type set.

[0055] Optionally, the obtaining of the geometric feature set comprises:

[0056] obtaining a model feature type set, wherein the model feature type set comprises a basic structure feature, a size parameter feature, and a surface morphology feature;

[0057] determining a model topology structure set according to the basic structure feature, wherein the model topology structure set comprises one or more model topology structures;

[0058] obtaining a model size parameter set according to the size parameter feature, wherein the model size parameter set comprises one or more model size parameters;

[0059] performing parameter conversion on the surface morphology features to obtain a surface morphology parameter set, wherein the surface morphology parameter set comprises one or more surface morphology parameters;

[0060] constructing a geometric feature set according to the model topology set, the model size parameter set and the surface morphology parameter set.

[0061] Optionally, the feature type set and the geometric feature set are subjected to hierarchical feature fusion by the feature fusion unit to obtain a multi-modal feature set, comprising:

[0062] confirming a basic structure type set, a size parameter type set and a surface morphology type set from the feature type set, wherein the basic structure type set comprises one or more basic structure types, the size parameter type set comprises one or more size parameter types, and the surface morphology type set comprises one or more surface morphology types;

[0063] calculating a structure similarity set based on the basic structure type set and the model topology set in the geometric feature set, wherein the structure similarity set comprises one or more structure similarities, each structure similarity corresponding to a basic structure type and a model topology;

[0064] calculating a parameter matching degree set based on the size parameter type set and the model size parameter set, wherein the parameter matching degree set comprises one or more parameter matching degrees, each parameter matching degree corresponding to a size parameter type and a model size parameter;

[0065] calculating a morphology similarity set based on the surface morphology type set and the surface morphology parameter set, wherein the morphology similarity set comprises one or more morphology similarities, each morphology similarity corresponding to a surface morphology type and a surface morphology parameter;

[0066] performing structure feature fusion on the basic structure type set and the model topology set by the feature fusion unit according to the structure similarity set to obtain a basic structure fusion feature vector set;

[0067] performing parameter feature fusion on the size parameter type set and the model size parameter set according to the parameter matching degree set to obtain a size fusion feature vector set;

[0068] performing morphology feature fusion on the surface morphology type set and the surface morphology parameter set according to the morphology similarity set to obtain a morphology fusion feature vector set;

[0069] performing feature aggregation on the basic structure fusion feature vector set, the size fusion feature vector set and the morphology fusion feature vector set to obtain a multi-modal feature set.

[0070] Optionally, the parameter checking on the initial three-dimensional model is performed to obtain a target three-dimensional model, including:

[0071] Based on the initial three-dimensional model, an initial model parameter set is obtained;

[0072] According to the multi-modal feature set, a target model parameter set is obtained;

[0073] According to the initial model parameter set and the target model parameter set, a parameter deviation value is calculated;

[0074] If the parameter deviation value is greater than or equal to a preset deviation threshold value, a three-dimensional generation model is adjusted according to a preset parameter adjustment method to obtain an adjusted three-dimensional generation model, the adjusted three-dimensional generation model is taken as the three-dimensional generation model, and the step of inputting the multi-modal feature set into a preset three-dimensional generation model to obtain the initial three-dimensional model is returned until the parameter deviation value is less than the deviation threshold value;

[0075] If the parameter deviation value is less than the deviation threshold value, the initial three-dimensional model is confirmed as the target three-dimensional model.

[0076] To achieve the above object, the application further provides a text-to-three-dimensional generation architecture optimization system based on hierarchical feature fusion, comprising:

[0077] An environment configuration module is configured to determine a three-dimensional generation environment, wherein the three-dimensional generation environment comprises a text processing unit, a feature processing unit and a feature fusion unit;

[0078] A text processing module is configured to obtain text data, perform a preprocessing operation on the text data by using the text processing unit to obtain a standard vocabulary type set, perform feature extraction on the standard vocabulary type set based on the feature processing unit to obtain a text feature set, and classify the text feature set to obtain a feature type set;

[0079] A feature fusion module is configured to obtain a geometric feature set, perform hierarchical feature fusion on the feature type set and the geometric feature set by using the feature fusion unit to obtain a multi-modal feature set;

[0080] A model generation module is configured to input the multi-modal feature set into a preset three-dimensional generation model to obtain an initial three-dimensional model, perform parameter checking on the initial three-dimensional model to obtain a target three-dimensional model, and complete text-to-three-dimensional generation architecture optimization.

[0081] To solve the above problems, the application further provides an electronic device, comprising:

[0082] The memory stores at least one instruction; and the processor executes the instruction stored in the memory to implement the above-mentioned text-to-three-dimensional generation architecture optimization method based on hierarchical feature fusion.

[0083] To solve the above problems, the application further provides a computer readable storage medium, which stores at least one instruction, and the at least one instruction is executed by a processor in an electronic device to implement the above-mentioned text-to-three-dimensional generation architecture optimization method based on hierarchical feature fusion.

[0084] To solve the problems in the background art, the application first determines a three-dimensional generation environment, wherein the three-dimensional generation environment includes a text processing unit, a feature processing unit and a feature fusion unit, and text data is obtained. By obtaining the text data, the application collects the text information input by the user, thereby providing a data basis for the subsequent steps. The text processing unit is used to pre-process the text data to obtain a standard vocabulary type set. Through the pre-processing operation, irrelevant data interference in the text data is removed, and the format of the text data is unified, thereby facilitating subsequent feature extraction. Based on the feature processing unit, feature extraction is performed on the standard vocabulary type set to obtain a text feature set. The text feature set is classified to obtain a feature type set, and a geometric feature set is obtained. The application maps the standard vocabulary to a low-dimensional word vector through a word embedding model, constructs a word vector sequence in combination with the original text order, performs semantic coding on the word vector sequence by using a context feature extraction model, and then obtains the text feature set, so that the obtained text feature set can completely reflect the three-dimensional requirements of the text description. The feature fusion unit is used to perform hierarchical feature fusion on the feature type set and the geometric feature set to obtain a multi-modal feature set. The multi-modal feature set is input into a preset three-dimensional generation model to obtain an initial three-dimensional model. The initial three-dimensional model is subjected to parameter verification to obtain a target three-dimensional model, and the text-to-three-dimensional generation architecture optimization is completed. Therefore, the application can solve the problem of low precision in the construction process of the three-dimensional model. BRIEF DESCRIPTION OF DRAWINGS

[0085] Figure 1 A flowchart of the text-to-three-dimensional generation architecture optimization method based on hierarchical feature fusion provided by an embodiment of the application is shown in the figure.

[0086] Figure 2 A functional module diagram of the text-to-three-dimensional generation architecture optimization system based on hierarchical feature fusion provided by an embodiment of the application is shown in the figure.

[0087] Figure 3 A structural diagram of an electronic device for implementing the text-to-three-dimensional generation architecture optimization method based on hierarchical feature fusion provided by an embodiment of the application is shown in the figure.

[0088] Reference signs:

[0089] 1. Electronic device; 10. Processor; 11. Memory; 12. Bus.

[0090] The implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0091] It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

[0092] An embodiment of the present application provides a text-to-three-dimensional generation architecture optimization method based on hierarchical feature fusion. The execution subject of the text-to-three-dimensional generation architecture optimization method based on hierarchical feature fusion includes but is not limited to at least one of electronic devices such as a server and a terminal, which can be configured to execute the method provided by the embodiment of the present application. In other words, the text-to-three-dimensional generation architecture optimization method based on hierarchical feature fusion can be executed by software or hardware installed in a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to a single server, a server cluster, a cloud server, or a cloud server cluster.

[0093] Referring to Figure 1 Fig. 1 is a flowchart of a text-to-three-dimensional generation architecture optimization method based on hierarchical feature fusion provided by an embodiment of the present application. In the embodiment, the text-to-three-dimensional generation architecture optimization method based on hierarchical feature fusion includes:

[0094] S1, determining a three-dimensional generation environment, wherein the three-dimensional generation environment includes a text processing unit, a feature processing unit, and a feature fusion unit.

[0095] It should be noted that the text processing unit refers to a functional module for performing a preprocessing operation on input text data, and can specifically implement text cleaning, word segmentation, and part-of-speech tagging operations. The core role is to convert unstructured text data into a structured standard vocabulary type set. The feature processing unit refers to a functional module for extracting semantic features and context features from the standard vocabulary type set. Its main functions include word embedding and context encoding, which can convert semantic information in text data into numerical vector form. The feature fusion unit refers to a functional module for realizing cross-modal fusion of text features and geometric features, supporting hierarchical fusion logic according to basic structures, size parameters, and surface morphology, and fusing the feature type set and the geometric feature set into a multi-modal feature set by calculating feature similarity and parameter matching degree. Quantitative indicators provide data support for subsequent generation of an initial three-dimensional model.

[0096] S2, obtaining text data, and performing a preprocessing operation on the text data by using the text processing unit to obtain a standard vocabulary type set.

[0097] It should be explained that the text data refers to a natural language sentence or paragraph used to drive the generation of a three-dimensional model, for example, a red dining chair with four curved metal legs.

[0098] In detail, the preprocessing operation on the text data by using the text processing unit to obtain a standard vocabulary type set includes:

[0099] performing data cleaning on the text data by using the text processing unit to obtain standard text data;

[0100] performing word segmentation processing on the standard text data to obtain a text vocabulary set;

[0101] performing the following operation on each text vocabulary in the text vocabulary set:

[0102] performing stopword marking on the text vocabulary according to a preset stopword library to obtain a marking result;

[0103] if the marking result is a preset stopword marking, marking the text vocabulary as a stopword;

[0104] summarizing the stopwords to obtain a stopword set;

[0105] excluding the stopword set from the text vocabulary set to obtain an effective text vocabulary set;

[0106] performing part-of-speech tagging on the effective text vocabulary set to obtain a vocabulary type set;

[0107] performing standardization processing on the vocabulary type set according to a preset standardization method to obtain a standard vocabulary type set.

[0108] It can be understood that the data cleaning refers to the standard processing of text data, specifically including: removing noise, removing repeated characters, unifying formats, etc. The technologies required for the standard processing are all prior art, and will not be described here. The standard text data refers to the text data after data processing. The word segmentation processing refers to the operation of using a word segmentation algorithm to split continuous standard text data into multiple independent words according to semantic logic. Optionally, the maximum matching algorithm is used as the word segmentation algorithm. The text vocabulary set refers to a set of multiple words obtained after word segmentation processing. The stop word library refers to a database containing words with no actual effect, which is set artificially in advance, and includes: auxiliary words (such as de, etc.), conjunctions (such as and, etc.) and other words that have no substantial impact on three-dimensional model generation. The stop word marking refers to the process of comparing the text vocabulary with the stop word library to obtain a marking result. The marking result refers to the result obtained after comparing the text vocabulary with the stop word library, which is divided into stop marking and non-stop marking. The stop marking refers to a specific identifier in the marking result indicating that the word is a stop word. The non-stop marking refers to a specific identifier in the marking result indicating that the word is not a stop word. The stop word refers to a text word consistent with a word in the stop word library, indicating that the text word has no substantial impact on three-dimensional model generation and can be removed. The stop word set refers to a set composed of all stop words. The effective text vocabulary set refers to a set of all remaining words containing words that have a substantial impact on three-dimensional model generation after removing the stop word set from the text vocabulary set. The standardization processing of the vocabulary type set according to the preset standardization method to obtain the standard vocabulary type set refers to: using a standardization method to unify the formats of different formats of vocabulary types in the vocabulary type set. The vocabulary type set after unifying the format is the standard vocabulary type set. The standardization method refers to a rule for unifying the format of the vocabulary type set. Optionally, a text normalization method is used as the standardization method.

[0109] Specifically, the part-of-speech tagging of the effective text vocabulary set to obtain the vocabulary type set includes:

[0110] The following operations are performed on each effective text vocabulary in the effective text vocabulary set:

[0111] Determine the semantic features of the effective text vocabulary;

[0112] According to the semantic features and the preset part-of-speech tagging method, the attribute category matching of the effective text vocabulary is performed to obtain a vocabulary matching result;

[0113] According to the vocabulary matching result, the effective text vocabulary is marked to obtain a vocabulary type;

[0114] The vocabulary types are summarized to obtain a vocabulary type set.

[0115] It should be understood that the semantic feature refers to the information possessed by the effective text vocabulary, which can reflect its semantic category and nature, which can be obtained by semantic analysis of the effective text vocabulary, for example, the effective text vocabulary is: cylindrical shape, and the semantic feature is cylinder after semantic analysis. The semantic analysis of the effective text vocabulary is prior art, which will not be repeated here. Optionally, the semantic feature is obtained by using a word vector cosine similarity calculation method, and the specific process is as follows: through a language model (such as Word2Vec), the effective text vocabulary is converted into a vector, and the vector is calculated with the vector of the semantic feature in the pre-set semantic feature library, and then matched to the corresponding semantic feature. The semantic feature library can be obtained from existing literature. The part-of-speech tagging method refers to a method for determining the semantic feature of the effective text vocabulary. Optionally, a sequence labeling algorithm is used as the part-of-speech tagging method. The part-of-speech refers to the functional classification of the vocabulary in the grammar system, including: noun, verb, adjective, etc. The attribute category matching refers to using a sequence labeling algorithm to determine the semantic feature of the effective text vocabulary. The vocabulary matching result refers to the corresponding relationship between the effective text vocabulary and the part-of-speech after attribute category matching, which is specifically: the "effective text vocabulary-part-of-speech" mapping in the form of key-value pair, wherein the key is the effective text vocabulary, and the value is the specific part-of-speech matched (such as "red"→ "adjective", "cylinder"→ "noun", etc.). The marking of the effective text vocabulary according to the vocabulary matching result to obtain the part-of-speech refers to: based on the vocabulary matching result, the effective text vocabulary is assigned to the corresponding part-of-speech, and the {effective text vocabulary, part-of-speech} combination formed is the part-of-speech. The part-of-speech set refers to a set composed of all part-of-speech.

[0116] S3, based on the feature processing unit, extracting features from the standard vocabulary type set to obtain a text feature set.

[0117] Further, the feature processing unit based on the feature processing unit, the feature extraction of the standard vocabulary type set to obtain a text feature set, comprising:

[0118] Using a pre-constructed word embedding model, the standard vocabulary type set is mapped to a vector to obtain a word vector set, wherein the word vector set contains a plurality of word vectors, and the word vector corresponds to a standard vocabulary type in the standard vocabulary type set one by one;

[0119] Obtain the original text order, and sort the word vector set according to the original text order to obtain a word vector sequence;

[0120] Obtain the context feature extraction model, and based on the context feature extraction model, the word vector sequence is semantically encoded to obtain a context feature vector set;

[0121] performing a pooling operation on the context feature vector set to obtain a comprehensive text feature vector;

[0122] performing the following operation on each context feature vector in the context feature vector set:

[0123] performing feature fusion on the comprehensive text feature vector and the context feature vector by using a feature processing unit to obtain a fusion feature vector;

[0124] obtaining a text feature set by aggregating the fusion feature vector.

[0125] It should be noted that the word embedding model refers to a neural network model that can map an input standard vocabulary type to a low-dimensional vector. Optionally, a Word2Vec model is selected as the word embedding model. The vector mapping refers to a process of using a word embedding model to convert a standard vocabulary type set into a low-dimensional vector. The word vector set refers to a set composed of all word vectors. The word vector refers to a low-dimensional numerical vector (such as a 100-dimensional vector, a 300-dimensional vector, etc.) generated after a certain word type is mapped by a vector. The original text order refers to the arrangement order of the word types in the standard vocabulary type set, which is consistent with the word order in the text data input by the user. The word vector set is sorted according to the original text order to obtain a word vector sequence, which means that the word vectors in the word vector set are rearranged according to the original text order to generate an ordered sequence, and the generated ordered sequence is the word vector sequence.

[0126] It needs to be explained that the context feature extraction model refers to a language model, which can capture the semantic dependency (such as modification, subordination, association and other logic) between words by analyzing the context relationship of word vectors in the word vector sequence, and convert the dependency relationship into the association information of the word vector. Optionally, the BERT model is used as the context feature extraction model. The semantic encoding refers to the process of fusing each word vector with its context relationship by processing the input word vector sequence using the context feature extraction model. The context feature vector set refers to the set of word vectors after semantic encoding. The pooling operation refers to the process of aggregating information of all context feature vectors in the context feature vector set, extracting semantic information from multiple context feature vectors through a specific strategy (such as mean pooling, maximum pooling, etc.), and generating a vector that can represent the semantics of the entire context feature vector set. The integrated text feature vector refers to the vector obtained after the pooling operation on the context feature vector set. The feature fusion refers to the process of weighted summation of the integrated text feature vector and the context feature vector. The weighted summation of the integrated text feature vector and the context feature vector is a prior art, which will not be described here. The fusion feature vector refers to an enhanced vector that contains both the global semantics of the context feature vector set and the semantic information of the single context feature vector after the integrated text feature vector and the single context feature vector are fused. For example, a word vector sequence is: a red wooden chair. The fusion feature vector for the word "red" contains both the global concept of the chair as a whole object and the local detail of red as the surface property of the object. The text feature set refers to a set composed of all fusion feature vectors.

[0127] In detail, the context feature extraction model is obtained, including:

[0128] A plurality of word vector sequence samples are obtained.

[0129] The plurality of word vector sequence samples are divided according to a preset division ratio to obtain a training sample set and a validation sample set, wherein the training sample set contains a plurality of training samples, and the validation sample set contains a plurality of validation samples.

[0130] The plurality of training samples in the training sample set are used to train the pre-constructed machine learning model to obtain a candidate feature extraction model.

[0131] A plurality of validation feature vectors are obtained based on the plurality of validation samples in the validation sample set, wherein the validation feature vector corresponds to the validation sample one by one.

[0132] The plurality of validation samples are used to verify the candidate feature extraction model to obtain a plurality of predicted feature vectors, wherein the predicted feature vector corresponds to the validation sample one by one.

[0133] calculate an average similarity according to the plurality of predicted feature vectors and a plurality of validation feature vectors;

[0134] If the average similarity is less than a preset similarity threshold, obtain hyperparameters of a candidate feature extraction model, adjust the hyperparameters using a preset adjustment method to obtain an updated feature extraction model, take the updated feature extraction model as the candidate feature extraction model, and return to the step of training the pre-constructed machine learning model using the plurality of training samples in the training sample set until the average similarity is greater than or equal to the similarity threshold;

[0135] If the average similarity is greater than or equal to the similarity threshold, determine the candidate feature extraction model as the context feature extraction model.

[0136] It can be understood that the word vector sequence sample refers to a word vector sequence used for training and validating a machine model. The division ratio refers to a ratio for dividing a plurality of word vector sequence samples, for example, the training sample set accounts for 65%, and the validation sample set accounts for 35%. The training sample refers to a word vector sequence sample used for training parameters of a machine learning model. The validation sample refers to a word vector sequence sample used for evaluating performance of a candidate feature extraction model. The training sample set refers to a set composed of training samples. The validation sample set refers to a set composed of validation samples. The machine learning model refers to a model architecture used for extracting context semantic features from a word vector sequence. Optionally, the BERT model is taken as the machine learning model. The candidate feature extraction model refers to a trained machine learning model. The training refers to iteratively optimizing parameters of the machine learning model using the training sample set, so that the machine learning model can extract context features of a word vector from a word vector sequence. The validation feature vector refers to a context feature vector corresponding to a validation sample. The validation refers to a process of inputting the validation sample into the candidate feature extraction model, and evaluating performance (i.e., accuracy of extracting context features of a word vector) of the candidate feature extraction model by comparing a predicted feature vector output by the model with the validation feature vector. The predicted feature vector refers to a vector output by the candidate feature extraction model after inputting the validation sample into the candidate feature extraction model. The average similarity refers to an arithmetic mean of similarities of all predicted feature vectors and corresponding validation feature vectors. The similarity refers to a quantitative index used for measuring semantic closeness between a predicted feature vector and a validation feature vector. Optionally, the cosine similarity is taken as the similarity, and a calculation formula thereof is as follows: wherein, represents the predicted feature vector, represents the validation feature vector, represents the cosine similarity, represents an L2 norm of the predicted feature vector, represents an L2 norm of the validation feature vector.

[0137] It should be understood that the similarity threshold refers to a critical value of the pre-manually set similarity, and when the average similarity is greater than or equal to the similarity threshold, it indicates that the performance of the candidate feature extraction model reaches the expectation, and the candidate feature extraction model can be determined as the context feature extraction model. The hyperparameter refers to a parameter of the machine learning model that is pre-manually set before the machine learning model training starts (the hyperparameter will not be automatically adjusted in the training process). The adjustment method refers to a method of obtaining an optimized hyperparameter from a pre-manually set hyperparameter combination. Optionally, the grid search is used as the adjustment method. The hyperparameter combination can be obtained through existing literature. The updated prediction model refers to the candidate feature extraction model adjusted by the adjustment method.

[0138] S4, classifying the text feature set to obtain a feature type set.

[0139] Specifically, the classification of the text feature set to obtain the feature type set comprises:

[0140] obtaining a model application scenario;

[0141] determining a three-dimensional generation requirement set according to the model application scenario, wherein the three-dimensional generation requirement set comprises a plurality of three-dimensional generation requirements;

[0142] For each three-dimensional generation requirement in the three-dimensional generation requirement set, the following operations are performed:

[0143] performing feature analysis on the three-dimensional generation requirement to obtain an initial feature label;

[0144] summarizing the initial feature labels to obtain an initial feature label set;

[0145] performing clustering normalization processing on the initial feature label set to obtain a feature label set;

[0146] determining a semantic feature identifier set from the text feature set, wherein the semantic feature identifier set comprises a plurality of semantic feature identifiers, and each semantic feature identifier corresponds to a fusion feature vector in the text feature set;

[0147] performing feature type matching on the semantic feature identifier set based on the feature label set to obtain a feature type set.

[0148] It needs to be explained that the model application scenario refers to the application field or application scenario of the generated three-dimensional model. The three-dimensional generation requirement refers to the function that the user expects the three-dimensional model to meet. The three-dimensional generation requirement set refers to a set composed of all three-dimensional generation requirements. The feature analysis refers to a process of analyzing the three-dimensional generation requirement to extract the key characteristics or attributes that can represent the three-dimensional generation requirement, wherein the three-dimensional generation requirement is analyzed as prior art, and optionally, a sequence labeling algorithm is used as the method for analyzing the three-dimensional generation requirement. The initial feature label refers to the key characteristics or attributes of the three-dimensional generation requirement obtained after feature extraction, such as structure type, surface morphology, size parameter, etc. The initial feature label set refers to a set composed of all initial feature labels. The clustering normalization processing refers to a process of using a clustering algorithm (such as the K-means algorithm) to merge initial feature labels with similar or identical semantics in the initial feature label set, and unify the label expression format. Optionally, a morphological reduction method is used as the method for unifying the label expression format. The feature label set refers to the initial feature label set after clustering normalization processing. The semantic feature identifier refers to the identification information extracted from the fusion feature vector in the text feature set, which can represent the overall semantics of the fusion feature vector, wherein extracting identification information from the fusion feature vector in the text feature set is prior art, and optionally, principal component analysis is used as the method for extracting identification information from the fusion feature vector in the text feature set. The semantic feature identifier set refers to a set composed of all semantic feature identifiers. The feature type matching refers to a process of using a pre-set matching method to match each semantic feature identifier in the semantic feature identifier set with the feature label set (i.e., determining the corresponding feature label for each semantic feature identifier), and optionally, cosine similarity calculation is used as the matching method, and the specific matching process is consistent with the specific process of obtaining semantic features using the cosine similarity calculation method of word vectors, which will not be described here. The feature type set refers to a set composed of multiple feature types, each feature type corresponding to a semantic feature identifier and its matched feature label.

[0149] S5, obtaining a geometric feature set, using the feature fusion unit to perform hierarchical feature fusion between the feature type set and the geometric feature set to obtain a multi-modal feature set.

[0150] Further, the obtaining of the geometric feature set comprises:

[0151] Obtaining a model feature type set, wherein the model feature type set includes: basic structure features, size parameter features, and surface morphology features;

[0152] According to the basic structure features, determining a model topology structure set, wherein the model topology structure set includes one or more model topology structures;

[0153] obtaining a model size parameter set according to the size parameter feature, wherein the model size parameter set comprises one or more model size parameters;

[0154] performing parameter conversion on the surface morphology feature to obtain a surface morphology parameter set, wherein the surface morphology parameter set comprises one or more surface morphology parameters;

[0155] constructing a geometric feature set according to the model topology structure set, the model size parameter set and the surface morphology parameter set.

[0156] It should be understood that the model feature type set refers to a set of feature types in a three-dimensional model generation process, which is artificially set in advance, and specifically includes: a basic structure feature, a size parameter feature and a surface morphology feature. The basic structure feature refers to a feature category used to define the geometric morphology of a three-dimensional model, for example, a rotational body structure (such as a cylinder, a cone), a polyhedral structure (such as a cube, a triangular pyramid), etc. The size parameter feature refers to a feature category used to quantify the spatial size of a three-dimensional model, for example, length, diameter, height, etc. The surface morphology feature refers to a feature category used to describe the appearance geometric attributes of a three-dimensional model, such as surface shape, texture, etc. The model topology structure refers to a graph structure data representing the connection and composition relationship between components of a three-dimensional model, which is generated based on the basic structure feature through a three-dimensional shape grammar or a graph neural network. The model topology structure set refers to a set composed of all model topology structures. The model parameter size refers to specific numerical information used to describe the spatial structure size of a three-dimensional model, which is extracted from the size parameter feature, for example, a cylinder diameter of 5 cm. The model parameter size set refers to a set composed of all model parameter sizes. The parameter conversion refers to a process of converting the surface morphology feature into quantifiable parameters recognizable by a three-dimensional modeling system, wherein the method of converting the surface morphology feature into quantifiable parameters recognizable by a three-dimensional modeling system is a prior art. Optionally, mesh parameterization is adopted as the method of converting the surface morphology feature into quantifiable parameters recognizable by a three-dimensional modeling system. The surface morphology parameter refers to a specific parameter used to quantitatively describe the appearance attributes of a model surface after parameter conversion, for example, a surface roughness of Ra1.6 μm and a texture resolution of 2048×2048. The surface morphology parameter set refers to a set composed of all surface morphology parameters. The geometric feature set is constructed according to the model topology structure set, the model size parameter set and the surface morphology parameter set, which means that the model topology structure set, the model size parameter set and the surface morphology parameter set are summarized and integrated into a set, which is the geometric feature set.

[0157] In detail, the feature fusion unit is used to perform hierarchical feature fusion on the feature type set and the geometric feature set to obtain a multi-modal feature set, which includes:

[0158] confirming a basic structure type set, a size parameter type set and a surface morphology type set from the feature type set, wherein the basic structure type set contains one or more basic structure types, the size parameter type set contains one or more size parameter types, and the surface morphology type set contains one or more surface morphology types;

[0159] calculating a structure similarity set based on the basic structure type set and a model topology set in the geometric feature set, wherein the structure similarity set contains one or more structure similarities, and each structure similarity corresponds to a basic structure type and a model topology structure;

[0160] calculating a parameter matching degree set based on the size parameter type set and a model size parameter set, wherein the parameter matching degree set contains one or more parameter matching degrees, and each parameter matching degree corresponds to a size parameter type and a model size parameter;

[0161] calculating a morphology similarity set based on the surface morphology type set and a surface morphology parameter set, wherein the morphology similarity set contains one or more morphology similarities, and each morphology similarity corresponds to a surface morphology type and a surface morphology parameter;

[0162] performing structure feature fusion on the basic structure type set and the model topology set by a feature fusion unit according to the structure similarity set, to obtain a basic structure fusion feature vector set;

[0163] performing parameter feature fusion on the size parameter type set and the model size parameter set according to the parameter matching degree set, to obtain a size fusion feature vector set;

[0164] performing morphology feature fusion on the surface morphology type set and the surface morphology parameter set according to the morphology similarity set, to obtain a morphology fusion feature vector set;

[0165] performing feature aggregation on the basic structure fusion feature vector set, the size fusion feature vector set and the morphology fusion feature vector set, to obtain a multi-modal feature set.

[0166] It can be understood that the basic structure type refers to a feature type related to a basic structure (such as a cylinder, a cuboid, etc.) extracted from the feature type set. The basic structure type set refers to a set composed of all basic structure types. The size parameter type refers to a feature type related to a size parameter (such as length, height, etc.) extracted from the feature type set. The size parameter type set refers to a set composed of all size parameters. The surface morphology type refers to a feature type related to a surface morphology (such as roughness, texture resolution, etc.) extracted from the feature type set. The surface morphology type set refers to a set composed of all surface morphology types.

[0167] It should be understood that the structure similarity refers to the matching degree between the basic structure type and the model topology structure in semantics and structure, and the higher the value is, the more matched the basic structure type and the model topology structure are. The calculation method of the structure similarity is consistent with the calculation method of the similarity, and will not be repeated here. The structure similarity set refers to a set composed of all structure similarities. The parameter matching degree refers to the matching degree between the size parameter type and the model size parameter, and the higher the value is, the more matched the size parameter type and the model size parameter are. The calculation method of the parameter matching degree is: . The parameter matching degree set refers to a set composed of all parameter matching degrees. The morphology similarity refers to the similarity between the surface morphology type and the surface morphology parameter, and the higher the value is, the more similar the surface morphology type and the surface morphology parameter are. The calculation method of the morphology similarity is consistent with the calculation method of the structure similarity, and will not be repeated here. The morphology similarity set refers to a set composed of all morphology similarities. The structure feature fusion refers to a process of integrating the basic structure type and the model topology structure by weighting (using the structure similarity as the weight). The method of integrating the basic structure type and the model topology structure by weighting is a prior art. Optionally, a cross-modal feature fusion method is used as the method of integrating the basic structure type and the model topology structure by weighting. The basic structure fusion feature vector refers to a set of comprehensive vectors obtained after structure feature fusion, which can comprehensively represent the relationship between the basic structure type and the corresponding model topology structure. The parameter feature fusion refers to a process of integrating the size parameter type and the model size parameter. The method of integrating the size parameter type and the model size parameter is consistent with the method of integrating the basic structure type and the model topology structure by weighting, and will not be repeated here. The size fusion feature vector set refers to a combination of comprehensive vectors obtained after parameter feature fusion, which can comprehensively represent the relationship between the size parameter type and the corresponding model size parameter. The morphology feature fusion refers to a process of integrating the surface morphology type and the surface morphology parameter. The method of integrating the surface morphology type and the surface morphology parameter set is consistent with the method of integrating the basic structure type and the model topology structure by weighting, and will not be repeated here. The morphology fusion feature vector set refers to a set of comprehensive vectors obtained after morphology feature fusion, which can comprehensively represent the relationship between the surface morphology type and the corresponding surface morphology parameter. The feature aggregation refers to a process of integrating the basic structure fusion feature vector, the size fusion feature vector and the morphology fusion feature vector into a unified vector by vector splicing, for example, splicing three 128-dimensional vectors into a 384-dimensional vector. The multi-modal feature set refers to a set of multi-modal features obtained after feature aggregation of the basic structure fusion feature vector set, the size fusion feature vector set and the morphology fusion feature vector set. Each multi-modal feature is the unified vector.

[0168] S6, inputting the multi-modal feature set into a preset three-dimensional generation model to obtain an initial three-dimensional model.

[0169] It should be noted that the three-dimensional generation model refers to a generative architecture based on deep learning, which can map multi-modal features to three-dimensional geometric representation. Optionally, a diffusion model is used as the three-dimensional generation model. The initial three-dimensional model refers to the three-dimensional model obtained after inputting the multi-modal feature set into the three-dimensional generation model.

[0170] S7, performing parameter verification on the initial three-dimensional model to obtain a target three-dimensional model, and completing the text-to-three-dimensional generation architecture optimization.

[0171] It should be understood that the parameter verification on the initial three-dimensional model to obtain a target three-dimensional model includes:

[0172] obtaining an initial model parameter set based on the initial three-dimensional model;

[0173] obtaining a target model parameter set according to the multi-modal feature set;

[0174] calculating a parameter deviation value according to the initial model parameter set and the target model parameter set;

[0175] If the parameter deviation value is greater than or equal to a preset deviation threshold, adjusting the three-dimensional generation model according to a preset parameter adjustment method to obtain an adjusted three-dimensional generation model, taking the adjusted three-dimensional generation model as the three-dimensional generation model, and returning to the step of inputting the multi-modal feature set into the preset three-dimensional generation model to obtain the initial three-dimensional model until the parameter deviation value is less than the deviation threshold;

[0176] If the parameter deviation value is less than the deviation threshold, the initial three-dimensional model is confirmed as the target three-dimensional model.

[0177] It should be explained that the initial model parameter set refers to a set of parameters extracted from the initial three-dimensional model for quantitatively describing the geometric properties thereof, including: structured parameters (such as topological structure), size parameters (such as length, diameter), surface morphology parameters (such as roughness, texture resolution), etc. The target model parameter set refers to a set of ideal parameters expected to be possessed by the target three-dimensional model, which corresponds to the initial model parameter set in the set (such as structured parameters, size parameters, and surface morphology parameters). The parameter deviation value refers to a quantitative index of the difference between the initial model parameter set and the target model parameter set. The larger the parameter deviation value, the greater the deviation between the initial model parameter set and the target model parameter set. The calculation method of the parameter deviation value is: wherein, N represents the total number of parameters in the target model parameter set and the initial model parameter set. This represents the j-th parameter in the initial model parameter set. This represents the j-th parameter in the target model parameter set. Represents the absolute value symbol.

[0178] In detail, the deviation threshold refers to a pre-set critical value for the deviation. When the parameter deviation value is less than the deviation threshold, it means that the accuracy of the initial 3D model meets the user's expectations. The parameter adjustment method refers to a technical method for optimizing the 3D generated model when the parameter deviation value is greater than or equal to the deviation threshold. Optionally, the method for adjusting the hyperparameters in the 3D generated model described above can be used as the parameter adjustment method. The method for adjusting the hyperparameters in the 3D generated model is consistent with the adjustment method and will not be repeated here. The adjusted 3D generated model refers to the 3D generated model after being adjusted by the parameter adjustment method. The target 3D model refers to the initial 3D model whose parameter deviation value is less than the deviation threshold. When the parameter deviation value is less than the deviation threshold, it means that the accuracy of the initial 3D model meets the user's requirements, and the initial 3D model can be determined as the target 3D model.

[0179] To address the problems described in the background art, this invention first establishes a 3D generation environment, comprising a text processing unit, a feature processing unit, and a feature fusion unit. This environment acquires text data, gathering user-inputted text information to provide a data foundation for subsequent steps. The text processing unit preprocesses the text data to obtain a standard vocabulary type set. This preprocessing removes irrelevant data interference and standardizes the text data format, facilitating subsequent feature extraction. Based on the feature processing unit, features are extracted from the standard vocabulary type set to obtain a text feature set. This text feature set is then classified to obtain a feature type set and a geometric feature set. This invention maps standard vocabulary to low-dimensional word vectors using a word embedding model, constructs a word vector sequence based on the original text order, and semantically encodes the word vector sequence using a context feature extraction model to obtain the text feature set. This ensures that the obtained text feature set fully reflects the 3D requirements of the text description. By utilizing the feature fusion unit, the feature type set and the geometric feature set are fused hierarchically to obtain a multimodal feature set. This multimodal feature set is then input into a preset 3D generation model to obtain an initial 3D model. Parameter validation is performed on the initial 3D model to obtain the target 3D model, thus completing the optimization of the text-to-3D generation architecture. Therefore, this invention can solve the problem of low accuracy in the 3D model construction process.

[0180] like Figure 2 The diagram shown is a functional block diagram of a text-to-3D generation architecture optimization system based on hierarchical feature fusion provided in an embodiment of the present invention.

[0181] The text-to-three-dimensional generation architecture optimization system 100 based on hierarchical feature fusion of the present application can be installed in an electronic device. According to the implemented functions, the text-to-three-dimensional generation architecture optimization system 100 based on hierarchical feature fusion can include an environment configuration module 101, a text processing module 102, a feature fusion module 103, and a model generation module 104. The modules of the present application can also be referred to as units, which refer to a series of computer program segments that can be executed by an electronic device processor and can complete a fixed function, which are stored in the memory of the electronic device.

[0182] The environment configuration module 101 is configured to determine a three-dimensional generation environment, wherein the three-dimensional generation environment includes a text processing unit, a feature processing unit, and a feature fusion unit.

[0183] The text processing module 102 is configured to obtain text data, perform preprocessing operations on the text data using the text processing unit to obtain a standard vocabulary type set, perform feature extraction on the standard vocabulary type set based on the feature processing unit to obtain a text feature set, and classify the text feature set to obtain a feature type set.

[0184] The feature fusion module 103 is configured to obtain a geometric feature set, and perform hierarchical feature fusion on the feature type set and the geometric feature set using the feature fusion unit to obtain a multi-modal feature set.

[0185] The model generation module 104 is configured to input the multi-modal feature set into a predetermined three-dimensional generation model to obtain an initial three-dimensional model, perform parameter verification on the initial three-dimensional model to obtain a target three-dimensional model, and complete text-to-three-dimensional generation architecture optimization.

[0186] In detail, the modules in the text-to-three-dimensional generation architecture optimization system 100 based on hierarchical feature fusion in the embodiments of the present application use the same technical means as the text-to-three-dimensional generation architecture optimization method based on hierarchical feature fusion in the above Figure 1 , and can produce the same technical effects, which will not be described here.

[0187] As shown in Figure 3 , it is a structural schematic diagram of an electronic device for implementing a text-to-three-dimensional generation architecture optimization method based on hierarchical feature fusion according to an embodiment of the present application.

[0188] The electronic device 1 can include a processor 10, a memory 11, and a bus 12, and can also include a computer program stored in the memory 11 and executable on the processor 10, such as a text-to-three-dimensional generation architecture optimization method based on hierarchical feature fusion program.

[0189] The memory 11 includes at least one type of readable storage medium, such as flash memory, mobile hard disk, multimedia card, card-type memory (e.g., SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of the electronic device 1, such as a mobile hard disk of the electronic device 1. In other embodiments, the memory 11 can also be an external storage device of the electronic device 1, such as a plug-in mobile hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 1. Further, the memory 11 includes both an internal storage unit and an external storage device of the electronic device 1. The memory 11 can be used not only to store application software and various data installed on the electronic device 1, such as the code of the text-to-three-dimensional generation architecture optimization method based on hierarchical feature fusion, but also to temporarily store data that has been output or will be output.

[0190] The processor 10 can be composed of an integrated circuit in some embodiments, such as a single packaged integrated circuit or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPU), microprocessors, digital processing chips, graphics processors, and combinations of various control chips, etc. The processor 10 is the control core of the electronic device, which connects various components of the entire electronic device through various interfaces and lines, executes programs or modules stored in the memory 11 (such as the text-to-three-dimensional generation architecture optimization method based on hierarchical feature fusion), and calls data stored in the memory 11 to perform various functions and process data of the electronic device 1.

[0191] The bus 12 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus 12 can be divided into an address bus, a data bus, a control bus, etc. The bus 12 is configured to realize the connection and communication between the memory 11 and at least one processor 10, etc.

[0192] Figure 3 Only an electronic device with components is shown, and those skilled in the art can understand that, Figure 3The illustrated structure does not constitute a limitation on the electronic device 1, and can include fewer or more components than illustrated, or combine certain components, or different component arrangements.

[0193] For example, although not shown, the electronic device 1 can also include a power source (such as a battery) to power the various components, and preferably the power source can be logically connected to the at least one processor 10 through a power management device, so that the power management device can implement functions such as charge management, discharge management, and power consumption management. The power source can also include one or more DC or AC power sources, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, and any other components. The electronic device 1 can also include various sensors, Bluetooth modules, Wi-Fi modules, and the like, which are not described here.

[0194] Further, the electronic device 1 can also include a network interface, which can optionally include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), and is typically used to establish a communication connection between the electronic device 1 and other electronic devices.

[0195] Optionally, the electronic device 1 can also include a user interface, which can be a display (Display), an input unit (such as a keyboard (Keyboard)), and optionally a standard wired interface, a wireless interface. Optionally, in some embodiments, the display can be an LED display, a liquid crystal display, a touch liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch, etc. The display can also be appropriately referred to as a display screen or a display unit, and is used to display information processed in the electronic device 1 and to display a visualized user interface.

[0196] The text-to-three-dimensional generation architecture optimization method program based on hierarchical feature fusion stored in the memory 11 in the electronic device 1 is a combination of multiple instructions, which, when executed in the processor 10, can implement:

[0197] determining a three-dimensional generation environment, wherein the three-dimensional generation environment includes a text processing unit, a feature processing unit, and a feature fusion unit;

[0198] obtaining text data, and pre-processing the text data using the text processing unit to obtain a standard vocabulary type set;

[0199] based on the feature processing unit, performing feature extraction on the standard vocabulary type set to obtain a text feature set;

[0200] classifying the text feature set to obtain a feature type set;

[0201] obtaining a geometric feature set, and performing hierarchical feature fusion on the feature type set and the geometric feature set by using the feature fusion unit to obtain a multi-modal feature set;

[0202] inputting the multi-modal feature set into a preset three-dimensional generation model to obtain an initial three-dimensional model;

[0203] performing parameter verification on the initial three-dimensional model to obtain a target three-dimensional model, and completing optimization of the text-to-three-dimensional generation architecture.

[0204] Specifically, the specific implementation method of the processor 10 on the above instructions can refer to Figures 1 to 3 The description of related steps in the corresponding embodiments will not be repeated here.

[0205] Further, the modules / units integrated in the electronic device 1, if realized in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. The computer readable storage medium can be volatile or non-volatile. For example, the computer readable medium can include any entity or device capable of carrying the computer program code, a recording medium, a U disk, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM, Read-Only Memory).

[0206] The application also provides a computer readable storage medium, which stores a computer program. When the computer program is executed by a processor of an electronic device, the following can be achieved:

[0207] determining a three-dimensional generation environment, wherein the three-dimensional generation environment includes a text processing unit, a feature processing unit, and a feature fusion unit;

[0208] obtaining text data, and performing a preprocessing operation on the text data by using the text processing unit to obtain a standard vocabulary type set;

[0209] performing feature extraction on the standard vocabulary type set based on the feature processing unit to obtain a text feature set;

[0210] classifying the text feature set to obtain a feature type set;

[0211] obtaining a geometric feature set, and performing hierarchical feature fusion on the feature type set and the geometric feature set by using the feature fusion unit to obtain a multi-modal feature set;

[0212] inputting the multi-modal feature set into a preset three-dimensional generation model to obtain an initial three-dimensional model;

[0213] The initial three-dimensional model is subjected to parameter checking to obtain a target three-dimensional model, and the text-to-three-dimensional generation architecture is optimized.

[0214] In several embodiments provided by the present application, it should be understood that the disclosed devices, systems and methods can be implemented in other ways. For example, the above-described system embodiments are only illustrative, and actual implementation can have other division ways.

[0215] The modules described as separate components can or can not be physically separated, and the components shown as modules can or can not be physical units, i.e., can be located in one place or distributed to multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment.

[0216] In addition, the functional modules in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of hardware plus software functional modules.

[0217] It is obvious for those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and the present application can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application.

[0218] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. A text-to-3D generation architecture optimization method based on hierarchical feature fusion, characterized in that, The method includes: A 3D generation environment is defined, wherein the 3D generation environment includes: a text processing unit, a feature processing unit, and a feature fusion unit; The text data is acquired, and the text processing unit is used to preprocess the text data to obtain a standard vocabulary type set. Based on the feature processing unit, feature extraction is performed on the standard vocabulary type set to obtain a text feature set; The text feature set is classified to obtain a feature type set; Obtain a geometric feature set, and use the feature fusion unit to perform hierarchical feature fusion with the feature type set to obtain a multimodal feature set, including: The basic structure type set, the size parameter type set, and the surface morphology type set are identified from the feature type set. The basic structure type set contains one or more basic structure types, the size parameter type set contains one or more size parameter types, and the surface morphology type set contains one or more surface morphology types. Based on the set of basic structure types and the set of model topology structures in the set of geometric features, a set of structural similarities is calculated, wherein the set of structural similarities contains one or more structural similarities, and each structural similarity corresponds to a basic structure type and a model topology structure. Based on the set of size parameter types and the set of model size parameters, a set of parameter matching degrees is calculated, wherein the set of parameter matching degrees contains one or more parameter matching degrees, and each parameter matching degree corresponds to a size parameter type and a model size parameter; Based on the set of surface morphology types and the set of surface morphology parameters, a morphology similarity set is calculated, wherein the morphology similarity set contains one or more morphology similarities, and each morphology similarity corresponds to a surface morphology type and a surface morphology parameter. Based on the structural similarity set, the feature fusion unit is used to fuse the basic structure type set and the model topology set to obtain the basic structure fusion feature vector set. Based on the parameter matching degree set, parameter feature fusion is performed on the size parameter type set and the model size parameter set to obtain a size fusion feature vector set; Based on the morphological similarity set, morphological feature fusion is performed on the surface morphological type set and the surface morphological parameter set to obtain a morphological fusion feature vector set. The basic structure fusion feature vector set, size fusion feature vector set, and morphology fusion feature vector set are aggregated to obtain a multimodal feature set; The multimodal feature set is input into a preset 3D generation model to obtain an initial 3D model; The parameters of the initial 3D model are validated to obtain the target 3D model, thus completing the optimization of the text-to-3D generation architecture.

2. The text-to-3D generation architecture optimization method based on hierarchical feature fusion as described in claim 1, characterized in that, The step of using the text processing unit to preprocess the text data to obtain a standard vocabulary type set includes: The text processing unit is used to clean the text data to obtain standard text data. The standard text data is segmented to obtain a text vocabulary set; For each text word in the text vocabulary set, the following operation is performed: The text words are marked with stop words according to a preset stop word library to obtain the marking results; If the marking result is a preset stop mark, then the text words are marked as stop words; The stop words are summarized to obtain a stop word set; The stop word set is removed from the text vocabulary set to obtain the effective text vocabulary set; Part-of-speech tagging is performed on the effective text vocabulary set to obtain a vocabulary type set; The vocabulary type set is standardized according to a preset standardization method to obtain a standard vocabulary type set.

3. The text-to-3D generation architecture optimization method based on hierarchical feature fusion as described in claim 2, characterized in that, The part-of-speech tagging of the effective text vocabulary set yields a vocabulary type set, including: For each valid text word in the set of valid text words, the following operation is performed: Determine the semantic features of effective text vocabulary; Based on the semantic features and the preset part-of-speech tagging method, the effective text words are matched by attribute categories to obtain the word matching results; The valid text words are labeled according to the word matching results to obtain the word type; By summarizing the aforementioned vocabulary types, a vocabulary type set is obtained.

4. The text-to-3D generation architecture optimization method based on hierarchical feature fusion as described in claim 3, characterized in that, The feature processing unit performs feature extraction on the standard vocabulary type set to obtain a text feature set, including: Using a pre-built word embedding model, vector mapping is performed on the standard vocabulary type set to obtain a word vector set, wherein the word vector set contains multiple word vectors, and the word vectors correspond one-to-one with the standard vocabulary types in the standard vocabulary type set; Obtain the original text order, and sort the word vector set according to the original text order to obtain the word vector sequence; Obtain a context feature extraction model, and based on the context feature extraction model, perform semantic encoding on the word vector sequence to obtain a context feature vector set; The context feature vector set is pooled to obtain a comprehensive text feature vector; For each context feature vector in the set of context feature vectors, the following operation is performed: Using the feature processing unit, the comprehensive text feature vector and the context feature vector are fused to obtain a fused feature vector; The fused feature vectors are then combined to obtain the text feature set.

5. The text-to-3D generation architecture optimization method based on hierarchical feature fusion as described in claim 4, characterized in that, The context feature extraction model includes: Obtain multiple word vector sequence samples; The multiple word vector sequence samples are divided according to a preset division ratio to obtain a training sample set and a validation sample set, wherein the training sample set contains multiple training samples and the validation sample set contains multiple validation samples. A pre-built machine learning model is trained using multiple training samples from the training sample set to obtain a candidate feature extraction model. Multiple verification feature vectors are obtained based on multiple verification samples in the verification sample set, wherein each verification feature vector corresponds one-to-one with a verification sample. The candidate feature extraction model is validated using multiple validation samples, resulting in multiple predicted feature vectors, each corresponding one-to-one with a validation sample. The average similarity is calculated based on the multiple predicted feature vectors and multiple verification feature vectors. If the average similarity is less than a preset similarity threshold, the hyperparameters of the candidate feature extraction model are obtained, the hyperparameters are adjusted using a preset adjustment method to obtain an updated feature extraction model, the updated feature extraction model is used as a candidate feature extraction model, and the steps of training the pre-built machine learning model using multiple training samples in the training sample set are returned until the average similarity is greater than or equal to the similarity threshold. If the average similarity is greater than or equal to the similarity threshold, then the candidate feature extraction model is determined as the context feature extraction model.

6. The text-to-3D generation architecture optimization method based on hierarchical feature fusion as described in claim 5, characterized in that, The classification of the text feature set to obtain a feature type set includes: Identify model application scenarios; A set of 3D generation requirements is determined based on the application scenario of the model, wherein the set of 3D generation requirements includes multiple 3D generation requirements; For each 3D generation requirement in the set of 3D generation requirements, the following operation is performed: The 3D generation requirements are analyzed to obtain initial feature labels; The initial feature labels are summarized to obtain the initial feature label set; The initial feature label set is subjected to clustering and normalization to obtain the feature label set; A semantic feature identifier set is determined from the text feature set, wherein the semantic feature identifier set contains multiple semantic feature identifiers, and the semantic feature identifiers correspond one-to-one with the fused feature vectors in the text feature set; Based on the feature label set, feature type matching is performed on the semantic feature identifier set to obtain the feature type set.

7. The text-to-3D generation architecture optimization method based on hierarchical feature fusion as described in claim 6, characterized in that, The acquisition of the geometric feature set includes: Obtain a model feature type set, wherein the model feature type set includes: basic structural features, size parameter features, and surface morphology features; Based on the aforementioned basic structural features, a set of model topologies is determined, wherein the set of model topologies contains one or more model topologies; A model size parameter set is obtained based on the size parameter features, wherein the model size parameter set includes one or more model size parameters; The surface morphology features are transformed to obtain a surface morphology parameter set, wherein the surface morphology parameter set contains one or more surface morphology parameters. A geometric feature set is constructed based on the model topology set, model size parameter set, and surface morphology parameter set.

8. The text-to-3D generation architecture optimization method based on hierarchical feature fusion as described in claim 7, characterized in that, The step of performing parameter verification on the initial 3D model to obtain the target 3D model includes: Based on the initial 3D model, obtain the initial model parameter set; The target model parameter set is obtained based on the multimodal feature set; Calculate the parameter deviation values ​​based on the initial model parameter set and the target model parameter set; If the parameter deviation value is greater than or equal to the preset deviation threshold, the 3D generation model is adjusted according to the preset parameter adjustment method to obtain the adjusted 3D generation model. The adjusted 3D generation model is used as the 3D generation model, and the process of inputting the multimodal feature set into the preset 3D generation model to obtain the initial 3D model is returned until the parameter deviation value is less than the deviation threshold. If the parameter deviation value is less than the deviation threshold, the initial 3D model is confirmed as the target 3D model.

9. A text-to-3D generation architecture optimization system based on hierarchical feature fusion, characterized in that, The system includes: An environment configuration module is used to determine the 3D generation environment, wherein the 3D generation environment includes: a text processing unit, a feature processing unit, and a feature fusion unit; The text processing module is used to acquire text data, perform preprocessing operations on the text data using the text processing unit to obtain a standard vocabulary type set, extract features from the standard vocabulary type set based on the feature processing unit to obtain a text feature set, and classify the text feature set to obtain a feature type set. The feature fusion module is used to acquire a geometric feature set and, using the feature fusion unit, to perform hierarchical feature fusion of the feature type set and the geometric feature set to obtain a multimodal feature set. The model generation module is used to input the multimodal feature set into a preset 3D generation model to obtain an initial 3D model, perform parameter verification on the initial 3D model to obtain a target 3D model, and complete the optimization of the text-to-3D generation architecture.

Images