3D art model intelligent generation system and method based on multi-source data fusion
The intelligent generation system, which integrates multi-source data, solves the problem of deviation in multi-dimensional design intent during 3D model generation, achieving efficient and accurate 3D model generation, reducing manual corrections, and enhancing the interactivity and robustness of the system.
Patent Information
- Application Number
- CN202511116787.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-11
- Publication Date
- 2026-02-17
- Estimated Expiration
- 2045-08-11
AI Technical Summary
Existing 3D model generation technologies are mostly based on a single data source, making it difficult to fully integrate multi-dimensional design intentions. The generated model deviates from the design intention, requiring a lot of manual correction, and interactive optimization support is limited.
The intelligent generation system employs multi-source data fusion, including a multi-source data input module, a feature extraction module, a condition generation module, a closed-loop optimization module, and an interactive optimization module. Through dynamic weight allocation and closed-loop optimization mechanisms, it collaboratively processes multi-source data to generate a 3D model that is highly consistent with the design intent.
It improves the accuracy and personalization of 3D model generation, reduces the amount of manual correction, enhances the robustness of the system, and supports real-time user feedback and conflict resolution.
Smart Images

Figure CN120997449B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer-aided design and graphics generation technology, specifically to a 3D art model intelligent generation system and method based on multi-source data fusion. Background Technology
[0002] In the field of digital content creation, the generation of 3D art models is a crucial step in game development, film and television production, and virtual scene construction. Traditional 3D modeling relies on manual operation, requiring multiple stages such as concept design, mesh construction, and texture mapping, which is not only time-consuming and labor-intensive but also demands extremely high professional skills from creators. With the development of artificial intelligence technology, data-driven 3D model generation technology has gradually emerged. Existing solutions are mostly based on a single data source (such as text descriptions or 2D images) for model generation. While this improves modeling efficiency to some extent, it is difficult to fully integrate multi-dimensional design intentions, limiting the diversity and accuracy of the generated models.
[0003] However, in actual creation, designers often convey design requirements through multiple sources of data, such as concept design drawings, text descriptions, and reference models. Existing technologies, when processing multi-source data, suffer from problems such as a lack of dynamic adjustment mechanisms for fusion strategies, imperfect arbitration logic when feature conflicts occur, and insufficient optimization capabilities for geometric and stylistic consistency. This leads to discrepancies between the generated model and the design intent, often requiring extensive manual post-production corrections. Furthermore, existing systems offer limited support for user-interactive optimization, making it difficult to efficiently integrate real-time feedback during the creative process. To address these issues, we propose an intelligent 3D art model generation system and method based on multi-source data fusion. Summary of the Invention
[0004] To address the aforementioned technical issues, this paper provides an intelligent 3D art model generation system and method based on multi-source data fusion. This technical solution solves the problems mentioned above, such as the lack of dynamic adjustment of fusion strategies when generating 3D models based on a single data source, imperfect feature conflict arbitration, insufficient geometric and style optimization, large deviation between the generated model and the design intent, the need for a large amount of manual correction, and limited support for interactive optimization.
[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:
[0006] A 3D art model intelligent generation system based on multi-source data fusion includes:
[0007] A multi-source data input module is used to receive conceptual design drawings, text descriptions, and reference 3D models;
[0008] The feature extraction module includes: an image feature extraction unit, used to extract style feature vectors and structural contour vectors from the concept design drawings; a text semantic parsing unit, used to output key attribute label vectors for text descriptions; and a model structure analysis unit, used to deconstruct the mesh topology matrix and material parameter set of the reference 3D model.
[0009] The conditional generation module, whose encoder normalizes and concatenates multi-source feature vectors, inputs them into the generative adversarial network. The generator outputs the vertex coordinates and UV texture coordinates of the initial 3D model, and the discriminator verifies the logical consistency between the generated model and the input features.
[0010] The closed-loop optimization module includes: a geometric error analysis unit, which calculates the structural contour deviation between the generated model mesh and the input conceptual design drawing; and a style consistency unit, which quantifies the color similarity between the material texture and the conceptual design drawing.
[0011] The interactive optimization module receives user correction instructions and converts them into parameter adjustment signals, which are then fed back to the condition generator.
[0012] A method for intelligent generation of 3D art models based on multi-source data fusion includes the following steps:
[0013] It can receive multi-source input data, including conceptual design drawings, text descriptions, and reference 3D models;
[0014] Perform feature extraction operations: extract the visual style feature vector and structural contour feature vector of the concept design drawing through the image parsing step; extract the semantic attribute feature vector of the text description through the text parsing step; and extract the topological structure feature matrix and material parameter feature set of the reference 3D model through the model parsing step.
[0015] Perform feature fusion operation, input the visual style feature vector, structural contour feature vector, semantic attribute feature vector, topological structure feature matrix and material parameter feature set into the condition to generate a model. The model fuses features according to a dynamic weight allocation strategy and generates initial 3D model mesh data and texture map.
[0016] Perform closed-loop optimization operations, calculate the structural contour deviation between the generated model mesh and the input conceptual design drawing through geometric error analysis, and compare the color distribution similarity between the generated model texture map and the conceptual design drawing through style consistency analysis.
[0017] When the deviation value or similarity does not reach the threshold, the optimization step is triggered to adjust the feature fusion weights and regenerate the model;
[0018] If a user modification instruction is received, semantic features are extracted and the generation parameters are updated through the instruction parsing step; the generation-evaluation-optimization operation is executed iteratively until the final 3D model is output.
[0019] Preferably, the dynamic weight allocation strategy specifically includes:
[0020] Confidence scores were calculated for each of the five input feature classes: the confidence score for the visual style feature vector was based on the image sharpness value of the concept design drawing and the proportion of the style saliency region to the total image area; the confidence score for the structural contour feature vector was based on the percentage of continuous edge pixels to the total edge pixels in the edge detection results; the confidence score for the semantic attribute feature vector was calculated by combining the frequency of entity keywords and the density of modifiers in the statistical text description; the confidence score for the topological structure feature matrix was determined by the mesh closure index of the reference 3D model and the inverse relationship between the number of non-manifold edges; and the confidence score for the material parameter feature set was correlated with the material map resolution value and the physical parameter completeness index.
[0021] The five confidence scores are converted into weight coefficients using a normalized exponential function, so that the values of each weight coefficient are in the range of zero to one and the sum is always one.
[0022] Finally, the features are fused by weighted summation, that is, each feature vector is multiplied by its corresponding weight coefficient and then summed to generate a fused feature vector.
[0023] Preferably, a dimension alignment operation is performed before feature fusion:
[0024] Visual style feature vectors and structural contour feature vectors are concatenated to form a joint image feature vector; semantic attribute feature vectors are mapped to the target dimension space through a fully connected neural network layer; the topological structure feature matrix is compressed by a graph convolutional network to generate a fixed-dimensional topological feature vector; and the material parameter feature set is encoded into a material feature vector through a multilayer perceptron.
[0025] A three-level feature pyramid fusion structure is adopted: the first-level fusion layer inputs the joint image feature vector and topological feature vector into the cross-attention mechanism module and outputs the geometric enhancement feature vector; the second-level fusion layer performs element-wise multiplication of the geometric enhancement feature vector and the material feature vector to generate the material binding feature vector; the third-level fusion layer fuses the material binding feature vector and the mapped semantic attribute feature vector through a gating mechanism and outputs the unified dimension conditional tensor required by the conditional generation model.
[0026] Preferably, the process of calculating the structural profile deviation value includes:
[0027] The initial 3D model mesh data is projected onto the view plane corresponding to the conceptual design drawing to generate a binary contour mask;
[0028] Perform edge detection on the conceptual design drawings to generate a reference contour mask;
[0029] The deviation value is calculated using the standardized Hausdorff distance algorithm: the average Euclidean distance from each pixel in the generated contour mask to the nearest reference contour mask pixel is calculated, then the average Euclidean distance from each pixel in the reference contour mask to the nearest generated contour mask pixel is calculated, and finally the arithmetic sum of the two averages is taken as the deviation value.
[0030] Preferably, the color distribution similarity is calculated as follows:
[0031] Extract 256-level histogram data from the concept design drawing and the generated texture map in the L channel, a channel, and b channel respectively;
[0032] For each color channel, the minimum sum of the two histograms at each gray level is calculated independently, and the arithmetic mean of the sums of the three channels is taken as the final similarity.
[0033] Simultaneously calculate style feature similarity: input both into the pre-trained VGG-19 network to extract the feature maps of the specified layers, calculate their Gram matrices respectively, and then calculate the sum of the squares of the differences between all elements of the two matrices, and then divide by the product of the square of the number of feature maps and the square of the feature map size.
[0034] Preferably, thresholds are preset for geometric error and style consistency respectively. When either geometric error or style consistency does not reach the threshold, the optimization step performs the following operations:
[0035] The fusion weight coefficient of the structural contour feature vector is increased by a preset ratio based on the structural contour deviation value; the fusion weight coefficient of the visual style feature vector is increased by a preset ratio based on the style feature similarity; at the same time, the weight coefficients of the other three types of features of non-visual style and structural contour are compressed by the original ratio, and the compression factor is one minus the sum of the aforementioned two weight increases divided by the sum of the weight coefficients of the other three types of features.
[0036] The adjusted weight coefficients are re-entered into the conditions to generate the model and the model is executed.
[0037] Preferably, the instruction parsing step employs a multi-level processing flow:
[0038] Semantic embedding vectors of user modification instructions are extracted using a pre-trained language model;
[0039] The vector is input into an intent classifier to identify the operation type, which includes three categories: geometric adjustment, material change, and style transfer.
[0040] The geometry adjustment branch outputs vertex displacement vector field parameters, the material change branch outputs material property modification amounts containing 12-dimensional parameters, and the style transfer branch outputs 512-dimensional style feature correction items.
[0041] The final generated parameter set includes the aforementioned displacement vector field parameters, material property modifications, and style feature correction terms, which are used to control model regeneration.
[0042] Preferably, the conditional generation model includes parallel mesh generation branches and texture generation branches:
[0043] The mesh generation branch adopts a graph attention mechanism, which processes and fuses features through an eight-layer residual graph convolutional network to output vertex coordinate matrix and face index matrix.
[0044] The texture generation branch adopts the U-Net structure, which generates RGBA texture maps of a specified resolution through an encoder-decoder architecture;
[0045] The two branches share a conditional encoder and exchange information in the intermediate feature layer through a cross-modal attention mechanism.
[0046] Preferably, a conflict resolution process is initiated when features from different input sources conflict:
[0047] Calculate the Euclidean distance between the structural outline feature vector of the conceptual design drawing and the topological feature vector of the reference model, and then divide by the larger norm value of the two to obtain the structural difference degree.
[0048] If the structural difference exceeds the warning value, the conflict arbitration mechanism will be activated, and the features of the conceptual design diagram will be used first, while the weight coefficient of the reference model will be reduced to one-fifth of the original value.
[0049] For conflicts between textual and visual features, calculate the cosine similarity between the semantic feature vector and the visual feature vector. If the similarity is less than 0.6, then correct the textual semantic feature weight coefficient to (0.5 + 0.5 × original value).
[0050] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0051] The intelligent 3D art model generation system and method proposed in this invention comprehensively integrates design intent through multi-source data input, including conceptual design drawings, text descriptions, and reference 3D models. This overcomes the limitations of generating models from a single data source. Its dynamic weight allocation strategy and closed-loop optimization mechanism ensure that the generated model is highly consistent with the design intent in terms of geometric shape and style characteristics, reducing the workload of manual post-processing corrections. At the same time, the interactive optimization module supports real-time user feedback, further improving the accuracy and personalization of the model. In addition, the conflict resolution process effectively handles the contradictions between features from different input sources, enhancing the robustness of the system and providing strong support for the rapid and accurate generation of 3D art models. Attached Figure Description
[0052] Figure 1 This is a system module framework diagram of the present invention;
[0053] Figure 2 This is a flowchart of the method of the present invention. Detailed Implementation
[0054] The following description is intended to disclose the invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious variations will occur to those skilled in the art.
[0055] Reference Figure 1 As shown, the intelligent generation system for 3D art models based on multi-source data fusion constructs a highly integrated creative framework. Its core lies in efficiently and collaboratively processing multiple heterogeneous data sources and accurately converting them into 3D digital assets. The system begins with a multi-source data input module, which not only receives concept design drawings, text descriptions, and reference 3D models, but also includes a preprocessing mechanism to ensure data quality. Concept design drawings undergo standardized size adjustments and RGB-to-Lab color space conversion, while also performing noise suppression. Text descriptions are processed through a natural language processing pipeline for part-of-speech tagging and dependency analysis, removing irrelevant function words and marking core entities such as materials and shapes. The reference 3D model undergoes triangular facet retopology optimization and global coordinate system unification, establishing a benchmark for subsequent feature extraction.
[0056] The feature extraction module employs a multimodal deep parsing architecture. The image feature extraction unit deploys an improved VGG-19 network, applying global average pooling to extract 128-dimensional style feature vectors on specific convolutional layer activation maps. Simultaneously, it combines phase-consistent edge detection and Hough line transform to generate 512-dimensional contour vectors describing the main structure. The text semantic parsing unit utilizes a pre-trained BERT model for bidirectional encoding, focusing on key modifiers through an attention mechanism, and outputting 256-dimensional label vectors containing semantic attributes such as material type and size ratio. The model structure analysis unit performs multi-scale analysis on the reference model, constructing a grid adjacency matrix using a topological traversal algorithm based on a half-edge data structure, and separating material parameter subsets such as diffuse reflection, specular highlights, and normals using a spectral clustering algorithm.
[0057] The conditional generation module, as the core engine of the system, first performs feature standardization and dimensional alignment in its encoder. Feature data of different dimensions are normalized by Z-score, and visual style and structural contour vectors are concatenated to form a 640-dimensional joint image feature. Semantic attribute vectors are mapped to a 512-dimensional target space through a two-layer fully connected network. The topological structure matrix is compressed into a 512-dimensional feature vector through a three-layer graph convolutional network. The material parameter set is encoded into a 256-dimensional vector through a multilayer perceptron with residual connections. A three-level pyramid fusion strategy is adopted: the first level uses a cross-attention mechanism to interact with the joint image features and topological features, generating geometrically enhanced features; the second level multiplies these features element-wise with the material vector to bind physical attributes; the third level fuses semantic attributes through a gated recurrent unit, outputting a unified conditional tensor. The generator employs a dual-branch architecture: the mesh generation branch contains an eight-layer residual graph convolutional network, with each layer integrating a graph attention mechanism to process vertex neighborhood relationships, ultimately outputting vertex coordinate matrices and triangular facet indices; the parallel texture generation branch uses a U-Net with skip connections, where the encoder performs five convolutional downsampling steps with a stride of 2, and after fusing the conditional tensor at the bottleneck layer, the decoder progressively reconstructs a 2048×2048 resolution RGBA texture map through transposed convolutions. The two branches exchange information through cross-modal attention in the fourth layer feature map to ensure spatial correspondence between geometry and texture. The discriminator constructs a three-scale evaluation network to verify local geometric continuity, global morphological proportions, and texture style consistency at resolutions of 64×64, 128×128, and 256×256, respectively.
[0058] The closed-loop optimization module establishes a quantitative feedback mechanism to continuously optimize the generated quality. The geometric error analysis unit performs precise spatial comparison: first, the generated model is projected onto a two-dimensional plane according to the viewpoint parameters of the conceptual design drawing, and a binary contour mask is generated through the improved Sobel-Feldman operator; simultaneously, Canny edge detection with adaptive thresholds is performed on the original design drawing to obtain the reference contour. The deviation calculation adopts the standardized bidirectional Hausdorff distance algorithm, calculating the mean Euclidean distance D_gen-ref from each pixel of the generated contour to the nearest reference contour pixel, and the mean D_ref-gen from each pixel of the reference contour to the nearest generated contour pixel, respectively. The final geometric deviation value is the arithmetic sum of the two. The style consistency unit implements a two-dimensional evaluation: In the color dimension, the Lab space histograms of the design drawing and the generated texture are extracted respectively, and the mean of the sum of the minimum values of the 256 grayscale histograms of each channel (L, a, b) is calculated as the color similarity; In the art style dimension, the ReLU4_1 layer feature maps of both inputs into the VGG-19 network are extracted, the Frobenius norm squared difference of their Gram matrix is calculated, and then the difference is standardized by dividing by the square of the number of feature map channels and the square of the spatial size.
[0059] When geometric deviation or style similarity does not reach the preset threshold, the system activates dynamic weight redistribution. First, the confidence scores of the five feature classes are recalculated: visual style confidence score is determined by the product of image resolution (0-1 normalized) and the proportion of style-salient regions (calculated through saliency detection); structural contour confidence score is based on edge continuity indicators (proportion of continuous edge pixels); semantic attribute confidence score is the harmonic mean of entity keyword frequency (number of occurrences per 100 words) and modifier density (proportion of adjectives and adverbs); topological structure confidence score is determined by the geometric mean of mesh closure (proportion of closed faces) and the number of non-manifold edges (normalized after taking the reciprocal); material confidence score is associated with texture resolution (0-1 normalized) and physical parameter completeness (coverage ratio of the preset parameter set). The five confidence scores are converted into initial weight coefficients using the Softmax function. Optimization phase: If geometric error exceeds the limit, the structural contour feature weight is increased proportionally to the error; if style consistency is insufficient, the visual style weight is increased simultaneously, while the weights of the other three feature classes are compressed proportionally, with a compression factor of 1 minus the sum of the first two increments divided by the sum of the remaining weights. The updated weights are then re-input into the condition generator for iterative optimization.
[0060] The interactive optimization module provides a fine-grained control channel for manual intervention. User correction commands are first transformed into 768-dimensional semantic embedding vectors using a BERT model, and then input into a three-layer fully connected classification network to identify the operation type (geometric adjustment / material change / style transfer). Geometric adjustment commands are processed by a graph neural network to output vertex displacement vector field parameters (including displacement direction and magnitude); material change commands are parsed into modifications to 12-dimensional parameters including base color, metallicity, and roughness; style transfer commands generate 512-dimensional style feature correction terms (obtained through style matrix eigenvalue decomposition). These parameters directly affect the corresponding control nodes of the generator, enabling controllable regeneration of the model.
[0061] The conflict resolution mechanism employs a hierarchical arbitration strategy. When the concept map and the reference model structure conflict, the Euclidean distance and norm ratio of their feature vectors are calculated. If the structural difference (distance / maximum norm) exceeds a threshold of 0.7, the reference model weights are forcibly reduced to 20% of their original value. For conflicts between textual and visual features, the cosine similarity between the semantic and visual feature vectors is calculated. If the similarity is below 0.6, a compromise strategy is adopted, adjusting the textual semantic weights to 0.5 plus 0.5 times the original value. Through multiple generation-evaluation-optimization loops, the system outputs a 3D art model that conforms to artistic and technical standards while ensuring the coordination of multi-source features.
[0062] refer to Figure 2As shown, the usage process of this invention is as follows: receiving multi-source data, including conceptual design drawings, text descriptions, and reference 3D models; extracting features, including visual and structural features of the design drawings, semantic features of the text, and topological and material features of the model; performing feature fusion, inputting various features into the conditions to generate a model, and generating an initial 3D model mesh and texture according to a dynamic weight allocation strategy; performing closed-loop optimization, calculating the structural deviation between the model mesh and the design drawings, and the similarity of texture colors; adjusting weights and regenerating if the threshold is not reached; receiving user modification instructions, parsing and updating parameters; iterating through the generation-evaluation-optimization process until the final 3D model is output.
[0063] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention. The scope of protection claimed by the appended claims and their equivalents is defined.
Claims
1.A 3D art model intelligent generation method based on multi-source data fusion, characterized in that, The method comprises the following steps: receiving multi-source input data, including a conceptual design drawing, a text description, and a reference 3D model; performing a feature extraction operation, extracting a visual style feature vector and a structural outline feature vector of the conceptual design drawing through an image analysis step, extracting a semantic attribute feature vector of the text description through a text analysis step, and extracting a topological structure feature matrix and a material parameter feature set of the reference 3D model through a model analysis step; performing a feature fusion operation, inputting the visual style feature vector, the structural outline feature vector, the semantic attribute feature vector, the topological structure feature matrix, and the material parameter feature set into a conditional generation model, the model fusing the features according to a dynamic weight distribution strategy and generating initial 3D model grid data and texture mapping; performing a closed-loop optimization operation, calculating a structural outline deviation value of the generated model grid and the input conceptual design drawing through a geometric error analysis step, and comparing the similarity of the color distribution of the generated model texture mapping and the conceptual design drawing through a style consistency analysis step; when the deviation value or the similarity does not reach a threshold value, triggering an optimization step to adjust the feature fusion weight and regenerate the model; if a user modification instruction is received, extracting semantic features to update the generated parameters through an instruction analysis step; iteratively performing the generation-evaluation-optimization operation until the final 3D model is output. 2.The 3D art model intelligent generation method based on multi-source data fusion according to claim 1, characterized in that, The dynamic weight distribution strategy specifically comprises: confidence scores are calculated for the five types of input features: the visual style feature vector confidence is based on the image clarity value of the conceptual design drawing and the proportion of the style saliency area to the total image area; the structural outline feature vector confidence is based on the percentage of continuous edge pixels in the total edge pixels in the edge detection result; the semantic attribute feature vector confidence is calculated by combining the frequency of entity keywords and the density of modifying words in the text description; the topological structure feature matrix confidence is determined by the grid closure index of the reference 3D model and the inverse relationship between the number of non-manifold edges; the material parameter feature set confidence is related to the material map resolution value and the physical parameter completeness index; a normalized exponential function is used to convert the five types of confidence scores into weight coefficients, so that each weight coefficient value is within the interval of zero to one and the sum is always one; finally, the features are fused through weighted summation operation, that is, each type of feature vector is multiplied by the corresponding weight coefficient and then added to generate a fused feature vector. 3.The method of claim 2, wherein, Before feature fusion, a dimension alignment operation is performed: the visual style feature vector and the structural outline feature vector are spliced into a joint image feature vector; the semantic attribute feature vector is mapped to the target dimension space through a fully connected neural network layer; the topological structure feature matrix is compressed through a graph convolution network to generate a topological feature vector of fixed dimension; the material parameter feature set is encoded into a material feature vector through a multi-layer perceptron; a three-level feature pyramid fusion structure is used: the first level fusion layer inputs the joint image feature vector and the topological feature vector into a cross-attention mechanism module and outputs a geometric enhancement feature vector; the second level fusion layer performs element-wise multiplication operation on the geometric enhancement feature vector and the material feature vector to generate a material binding feature vector; The third-level fusion layer fuses the material binding feature vector and the mapped semantic attribute feature vector through a gating mechanism, and outputs a uniform-dimensional condition tensor required by the conditional generation model. 4.The method of claim 1, wherein, The structure contour deviation value calculation process comprises: Projecting initial 3D model grid data to the plane corresponding to the perspective of the conceptual design drawing to generate a binary contour mask; Performing edge detection on the conceptual design drawing to generate a reference contour mask; The deviation value calculation adopts a standardized Hausdorff distance algorithm: calculating the average Euclidean distance of each pixel point in the generated contour mask to the nearest pixel point in the reference contour mask, and then calculating the average Euclidean distance of each pixel point in the reference contour mask to the nearest pixel point in the generated contour mask, and finally taking the arithmetic sum of the two averages as the deviation value. 5.The method of claim 1, wherein, The color distribution similarity is calculated as follows: Respectively extract the 256-level histogram data of the conceptual design drawing and the generated texture map in the L channel, a channel and b channel; Independently calculate the minimum value accumulation of the two histograms at each gray level for each color channel, and take the arithmetic average of the accumulation sums of the three channels as the final similarity; At the same time, the style feature similarity is calculated: input the pre-trained VGG-19 network of both sides to extract the specified layer feature map, calculate the Gram matrix respectively, and then calculate the square sum of the difference of all elements of the two matrices, and then divide by the product of the square of the number of feature maps and the square of the size of the feature map. 6.The method of claim 1, wherein, Threshold values are preset for geometric error and style consistency. When the geometric error or style consistency does not reach the threshold value, the optimization step performs the following operations: According to the structure contour deviation value, the fusion weight coefficient of the structure contour feature vector is increased by a preset proportion; according to the style feature similarity, the fusion weight coefficient of the visual style feature vector is increased by a preset proportion, and the other three types of feature weight coefficients of non-visual style and structure contour are compressed by the original proportion, and the compression factor is one minus the sum of the weight increase of the two types divided by the sum of the other three types of feature weight coefficients. The adjusted weight coefficients are re-input into the conditional generation model to perform model generation. 7.The method of claim 1, wherein, The instruction analysis step adopts a multi-level processing flow: Extract the semantic embedding vector of the user modification instruction through the pre-trained language model; Input the vector into the intent classifier to identify the operation type, which includes three types of geometric adjustment, material change and style transfer; The geometric adjustment branch outputs the vertex displacement vector field parameter, the material change branch outputs the material attribute modification amount containing 12-dimensional parameters, and the style transfer branch outputs the 512-dimensional style feature correction term; The final generated parameter set includes the aforementioned displacement vector field parameter, material attribute modification amount and style feature correction term, which is used to control model regeneration. 8.The method of claim 1, wherein, The conditional generation model comprises a parallel grid generation branch and a texture generation branch: The grid generation branch adopts a graph attention mechanism, processes the fused features through an eight-layer residual graph convolutional network, and outputs a vertex coordinate matrix and a face index matrix; The texture generation branch adopts a U-Net structure, and generates an RGBA texture map of a specified resolution through an encoder-decoder architecture; Both branches share a condition encoder and exchange information through a cross-modal attention mechanism at the intermediate feature layer. 9.The method of claim 1, wherein, When different input source features conflict, a conflict resolution process is started: Calculate the Euclidean distance between the conceptual design diagram structure profile feature vector and the reference model topology feature vector, and then divide by the larger norm value to obtain the structural difference degree; If the structural difference degree exceeds the warning value, the conflict arbitration mechanism is started, the conceptual design diagram features are preferentially adopted, and the reference model weight coefficient is reduced to one fifth of the original value; For conflicts between text and visual features, calculate the cosine similarity between the semantic feature vector and the visual feature vector. If the similarity is less than 0.6, the text semantic feature weight coefficient is modified to 0.5+0.5×original value. 10.A 3D artistic model intelligent generation system based on multi-source data fusion, characterized in that, The implementation process of the system includes the 3D art model intelligent generation method based on multi-source data fusion in any one of claims 1-9.
Citation Information
Patent Citations
3D content generation method and device, and storage medium
CN117475082A
Method for automatically reconstructing three-dimensional model of power transmission line iron tower from design drawing
CN117974929A