A method and system for establishing a three-dimensional scene based on virtual and real scripts

Abstract

The application relates to the technical field of three-dimensional modeling, and discloses a three-dimensional scene establishment method and system based on virtual and real scripts. The method comprises the following steps: performing deep semantic analysis according to a semantic description text, extracting emotional atmosphere and narrative logic features, obtaining a semantic feature vector, and matching and screening visual elements in a pre-stored visual element resource library to obtain a visual element list; generating spatial position and layout relationship of the visual elements according to the visual element list and a user interaction log, obtaining a layout parameter set, establishing a topological connection relationship, and generating a scene skeleton structure; performing mapping and synthesis of surface textures according to the scene skeleton structure to obtain a texture model; performing iterative adjustment of light and shadow parameters according to the texture model to obtain rendering configuration parameters; and performing three-dimensional scene drawing and pixel correction according to the rendering configuration parameters to obtain three-dimensional scene data. The method can generate a three-dimensional scene with high consistency of emotion and narrative logic according to semantic description.

Description

Technical Field

[0001] This invention relates to the field of 3D modeling technology, and in particular to a method and system for creating 3D scenes based on virtual and real-world scenarios. Background Technology

[0002] Currently, with the rapid development of digital content creation and virtual reality, 3D scene construction has become a crucial bridge connecting reality and virtuality. The current need lies in using 3D modeling technology to quickly transform complex semantic information from users—including emotions, logic, and cultural background—into context-appropriate immersive scenes. However, the depth and diversity of semantics make it difficult for existing methods to accurately capture and translate this information into scene parameters, often resulting in outputs that deviate from the intended atmosphere and narrative intent. Therefore, there is a need to develop technologies capable of deeply understanding semantics and flexibly driving 3D modeling to achieve a high degree of alignment between the scene and the user's intent.

[0003] In one existing technology, the user is first required to input a natural language description. The system then matches keywords (such as "trees" and "houses") from the description using a pre-set lexicon. Subsequently, the program retrieves corresponding standard model components from a pre-set 3D model resource library based on the keywords. Finally, the system arranges these components on a basic terrain according to simple spatial grammar rules (e.g., placing "houses" on flat ground and "trees" randomly distributed around them), completing 3D modeling and scene assembly. The entire process strictly follows a predefined logical chain, without parsing the implicit emotions, styles, or narrative relationships within the description. This method mechanically matches literal keywords and calls fixed models, completely ignoring the emotional inclinations, atmosphere, and narrative connections inherent in the description.

[0004] Therefore, existing technologies cannot generate 3D scenes that are highly consistent with emotional and narrative logic based on semantic descriptions. Summary of the Invention

[0005] This invention provides a method and system for creating three-dimensional scenes based on virtual and real-world scripts, so as to generate three-dimensional scenes with a high degree of consistency between emotion and narrative logic based on semantic description.

[0006] In a first aspect, to solve the aforementioned technical problems, the present invention provides a method for establishing a three-dimensional scene based on virtual and real-world scripts, comprising:

[0007] Obtain the user's semantic description text and user interaction logs;

[0008] Based on the semantic description text, deep semantic analysis is performed to extract emotional atmosphere and narrative logic features, resulting in a semantic feature vector.

[0009] Based on the semantic feature vector, the visual elements in the pre-stored visual element resource library are matched and filtered to obtain a list of visual elements;

[0010] Based on the list of visual elements and the user interaction log, the spatial position and layout relationship of the visual elements are generated in the three-dimensional virtual scene to obtain a layout parameter set;

[0011] Based on the set of layout parameters, establish topological connections and generate a scene skeleton structure;

[0012] Based on the scene skeleton structure, surface textures are mapped and synthesized to obtain a texture model;

[0013] Based on the texture model, the lighting and shadow parameters are iteratively adjusted to obtain the rendering configuration parameters;

[0014] Based on the rendering configuration parameters, 3D scene rendering and pixel correction are performed to obtain 3D scene data.

[0015] Secondly, the present invention provides a three-dimensional scene creation system based on virtual and real-world scripts, comprising:

[0016] The data acquisition module is used to obtain users' semantic description text and user interaction logs;

[0017] The semantic feature analysis module is used to perform deep semantic parsing based on the semantic description text, extract emotional atmosphere and narrative logic features, and obtain semantic feature vectors.

[0018] The visual element analysis module is used to match and filter visual elements in a pre-stored visual element resource library based on the semantic feature vector to obtain a list of visual elements.

[0019] The layout parameter analysis module is used to generate the spatial position and layout relationship of the visual elements in the three-dimensional virtual scene based on the visual element list and the user interaction log, and obtain the layout parameter set.

[0020] The scene skeleton construction module is used to establish topological connection relationships and generate a scene skeleton structure based on the layout parameter set.

[0021] The texture model construction module is used to map and synthesize surface textures based on the scene skeleton structure to obtain a texture model;

[0022] The rendering configuration module is used to iteratively adjust the lighting and shadow parameters according to the texture model to obtain the rendering configuration parameters;

[0023] The 3D scene generation module is used to draw and correct pixels of the 3D scene according to the rendering configuration parameters to obtain 3D scene data.

[0024] Compared with the prior art, the present invention has the following beneficial effects:

[0025] (1) This invention extracts emotional atmosphere and narrative logic features through deep semantic analysis, and combines intelligent matching and filtering of pre-stored visual element resource library to achieve high-precision conversion of semantics to visual elements, thereby improving the consistency of three-dimensional scene and user description in terms of emotion and logic.

[0026] (2) Based on the list of visual elements and user interaction logs, this invention performs spatial probability distribution modeling through latent semantic analysis and conditional generative adversarial networks, and optimizes the layout by combining dynamic adaptive factors, thereby realizing intelligent generation and adaptive adjustment of the position and layout relationship of visual elements in the three-dimensional scene, enhancing the spatial rationality and narrative expressiveness of the scene.

[0027] (3) This invention establishes topological connection relationships through Delaunay triangulation, and locates interactive nodes by combining interactive event knowledge base and Bayesian inference, thereby enhancing the narrative logic of the scene skeleton and dynamically reconstructing the topology, and improving the structural integrity and logical strength of the three-dimensional scene in terms of interactive support and narrative coherence.

[0028] (4) Based on the scene skeleton structure, the present invention performs surface texture mapping and lighting parameter iteration optimization, combines visual saliency assessment and drama enhancement matrix fusion to achieve narrative rendering of texture and lighting, and performs pixel-level semantic correction through generative adversarial network to improve the visual realism, emotional tension and user intent fit of the three-dimensional scene. Attached Figure Description

[0029] Figure 1 This is a schematic diagram of the method for creating a three-dimensional scene based on virtual and real-world scripts provided in the first embodiment of the present invention;

[0030] Figure 2 This is a schematic diagram of the structure of a three-dimensional scene creation system based on virtual and real-world scripts provided in the second embodiment of the present invention. Detailed Implementation

[0031] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] Reference Figure 1 The first embodiment of the present invention provides a method for creating a three-dimensional scene based on virtual and real-world scripts, including the following steps:

[0033] S11, Obtain the user's semantic description text and user interaction logs;

[0034] S12, Based on the semantic description text, perform deep semantic analysis to extract emotional atmosphere and narrative logic features, and obtain semantic feature vectors;

[0035] S13, Based on the semantic feature vector, match and filter with the visual elements in the pre-stored visual element resource library to obtain a list of visual elements;

[0036] S14. Based on the list of visual elements and the user interaction log, generate the spatial position and layout relationship of the visual elements in the three-dimensional virtual scene to obtain a layout parameter set.

[0037] S15, Based on the layout parameter set, establish topological connection relationships and generate a scene skeleton structure;

[0038] S16. Based on the scene skeleton structure, perform surface texture mapping and synthesis to obtain a texture model;

[0039] S17. Based on the texture model, the lighting and shadow parameters are iteratively adjusted to obtain the rendering configuration parameters;

[0040] S18, according to the rendering configuration parameters, perform 3D scene rendering and pixel correction to obtain 3D scene data.

[0041] In step S11, the user's semantic description text and user interaction logs are obtained.

[0042] Specifically, the semantic description text data originates from the natural language string content submitted by the user in a dedicated multi-line text input box on the front-end interface, and this content is directly assigned to the semantic description text variable. The user interaction log data comes from the user's historical operation behavior continuously recorded by the system backend. These logs are stored in a pre-built lightweight document-oriented database. Each log document contains a timestamp field, an operation type field, an operation object field, and a parameter field. The value of the operation type field comes from a preset operation type enumeration list (e.g., including model selection, parameter adjustment, etc.). The system retrieves all log records for the corresponding user from the database and loads them into the user interaction log list variable.

[0043] In step S12, deep semantic analysis is performed based on the semantic description text to extract emotional atmosphere and narrative logic features, thereby obtaining a semantic feature vector.

[0044] In one specific implementation, the step of performing deep semantic analysis based on the semantic description text to extract emotional atmosphere and narrative logic features, and obtaining a semantic feature vector, includes:

[0045] Based on the semantic description text, the Stanford Dependency Parser is used to parse the syntactic structure, extract the subject-verb-object structure in the text as logical nodes, obtain the logical node sequence, and calculate the cosine similarity between adjacent nodes as the logical association degree.

[0046] Based on the logical node sequence, the logical nodes are assigned sentiment polarity values ​​using the CNKI sentiment dictionary to obtain a sentiment polarity value sequence. The sentiment polarity values ​​are then arranged according to the order of the corresponding logical nodes in the original text to form a discrete sentiment value sequence.

[0047] The discrete sentiment value sequence is fitted using a cubic spline interpolation algorithm to generate a sentiment intensity variation curve and the arithmetic mean of the sentiment polarity value sequence is calculated to obtain the sentiment mean of the sequence.

[0048] The absolute difference between the emotional polarity value and the mean emotional value of the sequence is calculated to obtain the degree of emotional deviation. Logical nodes whose degree of emotional deviation exceeds the preset emotional deviation threshold are selected to obtain a set of core logical nodes. The text of the core logical nodes is encoded into a dense vector of fixed dimension using a pre-trained BERT model.

[0049] The mean, standard deviation, range, and number of positive and negative changes in the slope of the emotion polarity value are extracted from the emotion intensity change curve to obtain the emotion statistical characteristics;

[0050] The dense vector is subjected to mean pooling to obtain a comprehensive node vector, and the comprehensive node vector is concatenated with the sentiment statistical features to obtain a semantic feature vector.

[0051] Specifically, firstly, the system calls the pre-integrated Stanford dependency parser to process the semantic description text. The core of this parser is a statistical model based on a maximum entropy classifier. Its training data comes from the University of Pennsylvania Treebank, a corpus containing a large number of English sentences and their manually annotated grammatical dependency trees. During training, the algorithm extracts various features such as vocabulary, part-of-speech tagging, and position from the annotated data, and learns to predict the conditional probability of a specific dependency relationship between any two words by iteratively optimizing the parameters of the maximum entropy model. The system loads this trained model file during initialization. When parsing the semantic description text, the parser first performs word segmentation and part-of-speech tagging, then uses the trained classifier to calculate the probability of various dependency relationships between all word pairs, and finally uses a dynamic programming algorithm to search for the dependency tree with the highest global probability. The system identifies all subject-verb-object structures from this tree according to preset grammatical rules. Each structure is defined as a logical node, and logical node sequence variables are formed according to the original text order.

[0052] Subsequently, the system loads a pre-trained Word2Vec word vector model to quantify the semantic associations between logical nodes. This model is trained using a skip-word model architecture, with training data consisting of full-text Chinese Wikipedia entries and Chinese news texts within a specific time period. Key training parameters include a word vector dimension of 300, a context window size of 5, and a negative sampling count of 15. The training process uses stochastic gradient descent to optimize an objective function that maximizes the co-occurrence probability of the center word and its context words. After training, each word is mapped to a 300-dimensional real-valued vector. The system loads the model's word vector lookup table during initialization. When used, the system inputs the text of each node in the logical node sequence, and the model queries and returns the corresponding word vector. Next, the system calculates the cosine similarity between each pair of adjacent node word vectors in the sequence, i.e., the ratio of the sum of the products of each dimension of the two vectors to the product of their magnitudes; the resulting series of values ​​are recorded as the logical association variable.

[0053] The sentiment deviation threshold is set based on a clearly defined historical data statistical procedure. During the system development phase, a sample set containing 10,000 semantically descriptive texts and their manually annotated core logical nodes is collected. For each text in the sample set, the arithmetic mean of the sentiment values ​​of all its logical nodes is calculated, and then the absolute difference between each node's sentiment value and this mean is calculated. All absolute differences from all texts are collected, forming a set containing tens of thousands of values. This set is sorted in ascending order, and the value at the 75th percentile after sorting is determined as the sentiment deviation threshold. This fixed value is written to the system configuration file for runtime reading.

[0054] Simultaneously, the system reads the CNKI sentiment dictionary file and assigns sentiment polarity values ​​to the logical nodes. This dictionary is a pre-built database that stores the mapping relationship between Chinese words and a continuous numerical sentiment polarity value in tabular form. The system loads this into an in-memory hash table upon startup. The system traverses the sequence of logical nodes, searches for and matches the core words contained in each node in the hash table, and assigns the matched sentiment polarity value to that node, thus generating a sequence of sentiment polarity value variables that correspond one-to-one with the logical node sequence. Arranging these polarity values ​​in the original textual order of the nodes yields the discrete sentiment value sequence variables.

[0055] The system employs a cubic spline interpolation algorithm to fit the discrete sequence, generating a continuous curve representing the change in emotional intensity. Specifically, the algorithm treats each data point in the discrete sequence as a two-dimensional plane point. It solves a tridiagonal linear equation system to determine a unique cubic polynomial for every two adjacent data points. All piecewise polynomials must have equal function values ​​at connection points and continuous first and second derivatives, while the second derivative is set to zero at the endpoints of the sequence. Solving this system of equations yields a series of piecewise cubic polynomials that collectively define a smooth overall curve representing the change in emotional intensity.

[0056] Based on the original sentiment polarity value sequence, the arithmetic mean of all its values ​​is calculated to obtain the sentiment mean variable of the sequence. The absolute difference between each sentiment polarity value in the sequence and the sentiment mean of the sequence is calculated to obtain the sentiment deviation degree variable corresponding to each node. The system compares each sentiment deviation degree with the sentiment deviation threshold read from the configuration file, and filters out all logical nodes that exceed the threshold, forming the core logical node set variable.

[0057] Next, the system invokes a pre-trained BERT model to perform deep semantic encoding on the core logic nodes. This model employs a Transformer encoder architecture, containing 12 Transformer blocks, each with a 12-head self-attention mechanism sublayer and a feedforward neural network sublayer. The model is pre-trained using Chinese Wikipedia, Baidu Baike, and news corpora. Training tasks include masked language modeling and next-sentence prediction, using the Adam optimizer for large-scale distributed training. The pre-trained model parameters are loaded during system initialization. During use, the text of each node in the core logic node set is input into the model. The model extracts contextual features through a multi-layer Transformer encoder, ultimately outputting a dense 768-dimensional vector.

[0058] Four statistical features are extracted from the emotion intensity change curve function: the arithmetic mean of all emotion polarity values ​​represented by the curve is used as the emotion mean feature; the standard deviation of these values ​​is used as the emotion fluctuation feature; the difference between the maximum and minimum values ​​is used as the emotion range feature; and the number of times the sign of the first derivative of the statistical curve changes is used as the emotion turning point feature. These four values ​​constitute the statistical feature variables of emotion.

[0059] Finally, mean pooling is performed on all 768-dimensional dense vectors obtained by BERT encoding the core logical node set, i.e., the arithmetic mean of these vectors in each dimension is calculated to obtain the comprehensive node vector variable. This comprehensive node vector is then concatenated with the sentiment statistical feature variable, i.e., all elements of the two vectors are sequentially joined into a longer one-dimensional real vector, which is the final semantic feature vector variable. This vector comprehensively encodes the deep semantics, key narrative structure, and dynamic changes in sentiment of the text, providing a core basis for subsequent high-precision matching from the visual element resource library, and directly serving the key step in generating a three-dimensional scene with a high degree of consistency between sentiment and narrative logic in the invention's objective.

[0060] In step S13, the semantic feature vector is matched and filtered with the visual elements in the pre-stored visual element resource library to obtain a list of visual elements.

[0061] In one specific implementation, the step of matching and filtering visual elements in a pre-stored visual element resource library based on the semantic feature vector to obtain a list of visual elements includes:

[0062] Based on the semantic feature vector, the feature space is decoupled by principal component analysis to obtain the continuous emotional component and the discrete logical component.

[0063] Based on the continuous emotion components, the corresponding visual attribute label set and scene semantic label are mapped out through a multilayer perceptron classification model;

[0064] Based on the set of visual attribute labels and the logical discrete components, a hybrid retrieval vector is constructed through vector concatenation operations;

[0065] The similarity between the hybrid retrieval vector and the visual elements in the pre-stored visual element resource library is calculated using the cosine similarity algorithm to obtain a semantic matching confidence set.

[0066] Based on the set of semantic matching confidence scores, visual elements that exceed a preset semantic matching confidence threshold are selected, and the selected visual elements are sorted in descending order of semantic matching confidence scores to generate a list of visual elements containing semantic matching confidence scores and semantic types.

[0067] Specifically, firstly, the system decouples the semantic feature vectors from their feature space. This is achieved using principal component analysis (PCA). The algorithm takes the semantic feature vector as input, calculates the covariance matrix between each dimension of the vector, and then calculates the eigenvalues ​​and corresponding eigenvectors of this covariance matrix. The system pre-sets a variance contribution rate threshold of 90%. This threshold is based on the observation that after performing PCA on a large number of historically accumulated semantic feature vector samples, the cumulative variance contribution rate of the first few principal components first reaches 90%, indicating that the number of principal components can stably cover the main variation information of the samples. Eigenvalues ​​are cumulatively selected until their sum exceeds the threshold. The subspace formed by the directions of the first k selected principal components is defined as the continuous emotional component variable, primarily carrying the continuously changing emotional intensity information in the semantics; the projected components of the remaining dimensions are defined as the discrete logical component variable, primarily carrying discrete logical structure information.

[0068] Next, the system uses a pre-trained multilayer perceptron classification model to process the continuous sentiment component. This model is a three-layer fully connected neural network. The number of nodes in the input layer is the same as the dimension of the continuous sentiment component. The hidden layer contains 128 nodes using the ReLU activation function. The number of nodes in the output layer corresponds to the total number of all preset visual attribute labels and scene semantic labels. The Softmax activation function is used to convert the network's final linear output into a probability distribution for each label. The training data for this model comes from a constructed labeled dataset, where each data point contains a continuous sentiment component extracted from historical semantic feature vector samples, and a corresponding set of visual attribute labels and scene semantic labels jointly annotated by automated rules and the previous base model. The training process uses the cross-entropy loss function and the Adam optimizer for iterative optimization until the loss function converges. The pre-trained model parameter file is loaded during system initialization. When in use, the continuous sentiment component is input into the model, which calculates a probability distribution through forward propagation. The system sets a probability threshold, which is selected by evaluating the model using an independent validation set after training, choosing the probability value that maximizes the product of precision and recall. The system selects all labels whose output probability values ​​exceed the set threshold to form a set of visual attribute labels, and selects the label with the highest probability as the scene semantic label variable.

[0069] Next, the system constructs a hybrid retrieval vector for searching. Each label in the visual attribute label set is associated with a pre-trained word vector, which comes from the same Word2Vec model as in step S12. The system performs a weighted average of the word vectors of all selected labels, with the weights being the probability values ​​output by the multilayer perceptron model for the corresponding labels, resulting in a label semantic vector. This label semantic vector is then concatenated with the logical discrete component variables, i.e., all elements of the two vectors are sequentially connected to form a hybrid retrieval vector variable. This vector integrates the emotion-driven visual attributes with the original narrative logic information.

[0070] The semantic matching confidence threshold is determined based on historical data statistics. During the system development phase, a test set is constructed, containing a large number of semantic feature vectors and a list of matching visual elements determined by automated rules. For each test sample, the system calculates the similarity between its mixed retrieval vector and each visual element in the resource library, and records all similarity values. The similarity values ​​corresponding to all samples judged as correct matches by automated rules are collected to form a numerical distribution. The 85th percentile value of this distribution is set as the semantic matching confidence threshold and fixed in the system configuration file.

[0071] The visual element resource library is a pre-built database. Each visual element record contains a unique identifier, a pointer to a 3D model file, its semantic type, and a feature vector. This feature vector is a global feature extracted from representative rendered images of the element using a pre-trained convolutional neural network model during database loading. The resource library is loaded into memory during system initialization, and an index of the feature vectors is created to accelerate retrieval. During matching, the system calculates the similarity between the mixed retrieval vector and the feature vector of each visual element in the resource library using a cosine similarity algorithm. Cosine similarity is calculated as the ratio of the sum of the products of each dimension of the two vectors to the product of the magnitudes of the two vectors. All calculation results constitute the semantic matching confidence set variable.

[0072] Finally, the system performs filtering and sorting. The system compares each confidence value in the semantic matching confidence set with the semantic matching confidence threshold read from the configuration file. All visual element records with confidence scores exceeding the threshold are filtered out. These records are then sorted in descending order according to their corresponding semantic matching confidence scores. For each record, its visual element identifier, semantic matching confidence score, and semantic type are extracted, forming a visual element list variable output. This list provides a set of candidate visual elements that have undergone semantic filtering and priority sorting for subsequent spatial layout, supporting the requirement of high-precision matching between visual elements and semantic descriptions.

[0073] In step S14, based on the list of visual elements and the user interaction log, the spatial position and layout relationship of the visual elements are generated in the three-dimensional virtual scene to obtain a layout parameter set.

[0074] In one specific implementation, the step of generating and optimizing the spatial positions and layout relationships of the visual elements in a 3D virtual scene based on the visual element list and the user interaction log to obtain a layout parameter set includes:

[0075] Based on the visual element list and the user interaction log, user intent features are extracted through a latent semantic analysis model, and the feature values ​​of each dimension in the user intent features are weighted and averaged according to the preset intent weights to obtain a dynamic adaptation factor.

[0076] Based on the list of visual elements, spatial location mapping is performed according to the semantic type and the semantic matching confidence to obtain a set of layout coordinates;

[0077] Based on the set of layout coordinates and the dynamic adaptation factor, spatial probability distribution modeling is performed using a conditional generative adversarial network to generate a probability distribution layout heatmap containing pixel values.

[0078] Based on the probability distribution layout heatmap, the pixel values ​​are sampled using a mean-shift clustering algorithm to generate a set of layout variants.

[0079] Based on the set of layout variants, the centroid coordinates are obtained by normalizing the central moments, and the Euclidean distance between the centroid coordinates and the geometric center of the image is calculated to obtain the visual balance.

[0080] Based on the set of layout variants, layout variants with visual balance less than or equal to a preset visual balance threshold are selected to obtain a candidate layout set.

[0081] Based on the candidate layout set, the geometric transformation matrix of the candidate layout is calculated through affine transformation to obtain the layout parameter set.

[0082] Specifically, firstly, the system extracts user intent features from user interaction logs using a pre-trained latent semantic analysis model. This model is built upon a large amount of user action sequence data. The specific implementation involves treating all action types from historical user interaction logs as vocabulary, constructing an action frequency vector for each user session, forming a large user-action matrix. This matrix is ​​then subjected to singular value decomposition, decomposing it into the product of three matrices, one of which contains singular values. By retaining the largest few singular values ​​and their corresponding left and right singular vectors, dimensionality reduction is achieved, yielding the basis vectors of the user latent intent space. During system initialization, this dimensionality-reduced right singular vector matrix is ​​loaded as a projection model. When used, the current user interaction log variable is transformed into an action frequency vector and multiplied by this projection model to obtain a low-dimensional dense vector, i.e., the user intent feature variable.

[0083] The semantic type-spatial region mapping rules are predefined based on statistical analysis of historical scene data. The specific rule definition process involves collecting a large amount of constructed 3D scene data and statistically analyzing the coordinate distribution of visual elements of each semantic type in the normalized scene space. For example, elements of the "sky" type generally have their height coordinates located in the upper region of space, while those of the "ground" type are concentrated in the lower region. For each type, the mean and standard deviation of its coordinates in each dimension are calculated, forming a spatial region range centered on the mean and bounded by a certain multiple of the standard deviation. This correspondence is fixed in a lookup table. The system queries this table to obtain the basic spatial region range for each element in the visual element list based on its semantic type.

[0084] Next, the system performs initial spatial mapping based on the list of visual elements. Within their respective base regions, an initial 3D coordinate system is generated for each visual element. During generation, the semantic matching confidence of the element is converted into an offset weight, which is superimposed on the center coordinates of the base region by a random offset following a Gaussian distribution. The variance of the offset is inversely proportional to the confidence, making the positions of elements with higher confidence more stable. All coordinates constitute the layout coordinate set variable.

[0085] Subsequently, the system inputs the set of layout coordinates and the dynamic adaptation factor into a pre-trained conditional generative adversarial network (GAN). The generator of this network is a cascade of a fully connected neural network and a transposed convolutional network. Specifically, the layout coordinate set is first flattened into a one-dimensional vector, concatenated with the dynamic adaptation factor, and then input into the fully connected layer. The output of the fully connected layer is reshaped into a two-dimensional feature map, which is then upsampled through multiple transposed convolutional layers, ultimately outputting a two-dimensional probability distribution layout heatmap. The discriminator is a convolutional neural network that receives either real or generated heatmaps, and through multiple convolutions and downsampling, outputs a binary scalar representing the probability that the input is the true value. This network is trained adversarially using a dataset containing real, high-performing layouts and their heatmap representations. After training, the system uses the generator to generate probability distribution layout heatmap variables based on the current input.

[0086] Then, the system employs a mean-shift clustering algorithm to perform multi-peak sampling on the heatmap. This algorithm randomly selects an initial point from the pixels of the heatmap, iteratively calculates the probability density-weighted average position of all pixels within a specified radius around each point, and moves the point to this average position until it converges to a local probability density peak. The positions of all converged peak points are recorded, and each peak position represents a candidate placement center for a visual element.

[0087] The visual balance threshold is set based on historical data analysis. A large number of manually labeled 2D layout samples are collected, and the visual balance of each sample is calculated. Specifically, the normalized central moment calculation is a key step in calculating the centroid coordinates. For a layout composed of multiple coordinate points, its zeroth moment is the number of points, and its first moment is the sum of all point coordinate values. The centroid coordinates are the arithmetic mean of all point coordinate values, which is equivalent to dividing the first moment by the zeroth moment. This calculation process is the normalized central moment calculation, and the result, the centroid coordinates, is a 2D vector. The Euclidean distance between these centroid coordinates and the geometric center of the image is then calculated to obtain the visual balance; where the geometric center of the image is the center of the 3D bounding box defined by all coordinate points in the layout variant set. The distribution of visual balance values ​​for all samples is statistically analyzed, and the 80th percentile of this distribution is set as the visual balance threshold.

[0088] The system calculates and filters the visual balance of each variant in the layout variant set to form a candidate layout set variable.

[0089] Finally, the system calculates the affine transformation parameters for each layout variant in the candidate layout set. The system selects the four corner points of the layout and their target positions on the 3D spatial base plane, establishing a coordinate correspondence. The affine transformation matrix describes a linear mapping from a 2D point to a 3D point, plus translation. For each pair of corresponding points, three linear equations can be listed. Four pairs of points constitute an overdetermined system of linear equations. Solving this system of equations using the least squares method yields the optimal affine transformation matrix. The geometric transformation matrices corresponding to all candidate layout variants are summarized to form the final layout parameter set output. This parameter set encodes multiple possible spatial layout schemes that conform to visual balance and user intent.

[0090] This step, which integrates user intent understanding and generative models, is the core of achieving scene spatial rationality and narrative expressiveness.

[0091] In step S15, a topological connection relationship is established based on the layout parameter set to generate a scene skeleton structure.

[0092] In one specific implementation, the step of establishing topological connection relationships and generating a scene skeleton structure based on the layout parameter set includes:

[0093] Based on the set of layout parameters, the topological connection relationship between the spatial coordinates of the layout parameters is established by the Delaunay triangulation algorithm to generate the initial scene skeleton structure.

[0094] Based on the initial scene skeleton structure, the theoretical maximum sum of edge weights is calculated, and the actual sum of weights of all connected edges is calculated. The sum of weights is then compared with the theoretical maximum sum of edge weights to obtain the narrative logic strength. The weight value is the cosine similarity between the semantic feature vectors of the two visual elements connected by the corresponding edge.

[0095] When the narrative logic strength exceeds a preset logic strength threshold, the event trigger probability of the potential location is calculated using a Bayesian inference algorithm based on the scene semantic tags and the pre-stored interactive event knowledge base, and the potential location with the highest event trigger probability is selected as the interactive node.

[0096] Based on the interaction nodes, the initial scene skeleton structure is reconstructed through node insertion and edge reconnection operations in the graph data structure to obtain the scene skeleton structure.

[0097] When the narrative logic strength does not exceed the preset logic strength threshold, the initial scene skeleton structure is used as the final scene skeleton structure.

[0098] Specifically, firstly, the system constructs an initial topological connection based on the layout parameter set. The layout parameter set contains multiple candidate layout schemes, each represented by the 3D coordinates of a set of visual elements. The system selects the candidate scheme with the highest sum of semantic matching confidence and extracts the coordinates of all its visual elements. The Delaunay triangulation algorithm is then applied to these spatial coordinate points. The algorithm's implementation involves first constructing a convex hull containing all points, and then inserting points one by one. For each point to be inserted, all existing triangles containing that point within its circumcircle are found, these triangles are deleted to form a hole, and the point is connected to all points on the hole's boundary to form a new triangle. This process is repeated until all points are inserted, ultimately generating a mesh composed of triangles. This triangular mesh defines the proximity connections between visual elements and is represented as an initial scene skeleton structure variable. It is a graph structure where nodes are visual elements and edges are connections generated by triangulation.

[0099] Next, the system calculates the narrative logic strength of the initial scene skeleton structure. The weight of each connecting edge is defined as the cosine similarity between the semantic feature vectors of the two visual elements connected by that edge. The sum of all edge weights is calculated to obtain the actual total edge weight. The theoretical maximum total edge weight is defined as the sum of the weights of all possible edges assuming all visual elements are connected pairwise; for a set containing n visual elements, its theoretical maximum total edge weight is defined as the sum of the edge weights between all possible n(n-1) / 2 pairs of visual elements in the set, i.e., the sum of the cosine similarities between the semantic feature vectors of the two elements. The narrative logic strength variable is the ratio of the actual total edge weight to the theoretical maximum total edge weight. The determination of the logic strength threshold is based on historical data statistics. A large number of 3D scenes that have been manually evaluated as having "clear narrative logic" are collected, and the narrative logic strength values ​​of their initial scene skeletons are calculated to form a distribution. The 70th percentile value of this distribution is set as the logic strength threshold and written into the system configuration file.

[0100] The system compares the logic strength threshold read from the configuration file with the currently calculated narrative logic strength variable. If the narrative logic strength does not exceed the threshold, the system directly outputs the initial scene skeleton structure variable as the final scene skeleton structure variable.

[0101] If the narrative logic strength exceeds the logic strength threshold, it indicates that the current layout has the potential to support complex narrative interactions, and the system will perform skeleton enhancement. The enhancement process is based on the scene semantic label variables obtained from step S13 and a pre-stored interaction event knowledge base. This knowledge base is a graph database, constructed from data derived from the analysis of a large number of scripts, game levels, and narrative texts, extracting common interaction patterns, event types, and their typical scene contexts and spatial location preferences. Each rule in the knowledge base records the event type, related scene semantic labels, and a spatial probability distribution describing typical occurrence locations.

[0102] The system uses a Bayesian inference algorithm to calculate the event trigger probability of potential locations in the scene. Specifically, it uses scene semantic labels as evidence to query the knowledge base for all associated event types and their prior probabilities. For each triangular facet in the initial scene skeleton structure, its centroid is calculated as a potential location. For each potential location and each event type, the likelihood probability of observing the event at that location is calculated based on the spatial probability distribution model of that event type in the knowledge base. According to Bayes' theorem, the prior probability is multiplied by the likelihood probability and normalized to obtain the posterior probability of each potential location for each event type. The system selects the maximum value among all posterior probabilities and marks the potential location with this maximum value as an interaction node.

[0103] Then, the system performs topological reconstruction of the initial scene skeleton structure in the graph data structure. The reconstruction operation includes node insertion and edge reconnection. First, interactive nodes are inserted as new nodes into the skeleton graph. Then, all triangles in the graph are traversed. If an interactive node is located inside the circumcircle of a triangle, the three edges of that triangle are deleted, and the interactive node is connected to the three vertices of that triangle to form three new triangles. This process continues until the empty circumcircle property of Delaunay triangulation is satisfied. The resulting enhanced graph structure is the final scene skeleton structure variable. This structure, while maintaining spatial proximity, introduces narrative-driven interactive hotspots, making the generated 3D scene not only structurally stable but also providing spatial anchors for potential narrative development, directly serving the invention's objective of improving scene narrative coherence and interactive support.

[0104] This step enhances the internal coherence and interactive potential of the scene, and is key to building a coherent narrative space.

[0105] In step S16, surface textures are mapped and synthesized according to the scene skeleton structure to obtain a texture model.

[0106] In one specific implementation, the step of mapping and synthesizing surface textures based on the scene skeleton structure to obtain a texture model includes:

[0107] Based on the scene skeleton structure, a three-dimensional surface mesh is reconstructed using the traveling cube algorithm to obtain a three-dimensional surface mesh.

[0108] Based on the three-dimensional surface mesh, the corresponding texture coordinate mapping parameters are calculated using the least squares conformal mapping algorithm;

[0109] Based on the texture coordinate mapping parameters and the preset material parameter database, a basic material property layer containing diffuse reflection coefficient, normal vector and roughness value is generated for the three-dimensional surface mesh through the pre-constructed micro-surface model;

[0110] For the diffuse reflection coefficient, the color value of each pixel is converted into the corresponding grayscale brightness value using a preset standard brightness conversion formula, thereby obtaining a grayscale image;

[0111] The contrast ratio is obtained by calculating the standard deviation of all grayscale brightness values ​​in the grayscale image;

[0112] When the brightness-dark contrast is lower than a preset contrast threshold, a virtual point light source is deployed in the direction of the normal vector, and the radiant flux distribution is calculated using a photon mapping algorithm to generate a local light source enhancement matrix.

[0113] Based on the local light source enhancement matrix, a light and shadow baking is performed on the three-dimensional surface mesh using a screen space ambient light occlusion algorithm to generate a light and shadow texture map.

[0114] The light and shadow texture map and the basic material attribute layer are fused pixel by pixel using the alpha mixing algorithm to obtain the texture model.

[0115] Specifically, firstly, the system reconstructs the 3D surface based on the scene skeleton structure. The scene skeleton structure is a graph composed of nodes and edges, where nodes contain the 3D coordinates of visual elements. The system defines a voxel space centered on each node and assigns a signed distance function value to each voxel based on the connectivity between nodes. This value represents the distance from the voxel center to the nearest surface. The specific implementation of the moving cube algorithm is as follows: iterates through all voxels and checks the sign of the signed distance function values ​​at their eight corner points. For each edge, if the signs at both ends are different, a contour point is calculated on that edge using linear interpolation. Based on a predefined lookup table containing 256 cases, the contour points calculated within each voxel are connected to form triangular patches. Finally, the set of triangular patches generated from all voxels constitutes a closed 3D surface mesh variable.

[0116] Next, the system calculates the ultraviolet texture coordinates for the 3D surface mesh. This calculation employs a least-squares conformal mapping algorithm. Specifically, it maps the vertices of the 3D mesh to a 2D parametric plane. The algorithm first fixes the mesh boundary onto a 2D convex polygon, then optimizes the 2D coordinates of the internal vertices by solving a large system of sparse linear equations. The goal of this system is to make each 3D triangle as similar as possible to its corresponding 2D parameterized triangle, i.e., to minimize the angular distortion of all triangles. The 2D coordinates corresponding to each vertex obtained from the solution are the texture coordinate mapping parameter variables.

[0117] Then, the system combines the material parameter database to generate a basic material attribute layer. The material parameter database is a pre-built relational database whose data comes from physical measurements and scans of real-world material samples. Each record contains a material name and its corresponding diffuse color value, normal map vector, and roughness value. Based on the semantic type in the visual element list, the system assigns a basic material name to each facet of the 3D surface mesh. The micro-surface model is a physically based shading model, constructed by defining a function describing the distribution of microscopic surface normals. When used by the system, based on the texture coordinate mapping parameters, it queries the database for the diffuse color of the corresponding material as the diffuse coefficient, the normal map data as the normal vector, and the roughness scalar as the roughness value. This information is then filled into each texture pixel of the 3D surface mesh, collectively forming the variables of the basic material attribute layer.

[0118] Subsequently, the system evaluates and enhances the scene's contrast. A standard luminance conversion formula is applied to the diffuse reflection coefficient in the base material attribute layer. This formula calculates the weighted sum of the red, green, and blue components of each pixel's color value with a set of fixed weights to obtain the pixel's grayscale luminance value, thus converting the diffuse reflection coefficient map into a grayscale image variable. The standard deviation of all pixel luminance values ​​in this grayscale image is calculated to obtain the contrast ratio variable. The contrast threshold is determined based on historical data statistics. A large number of scene texture samples labeled "visually clear" are collected, and the contrast ratio of each sample is calculated to form a distribution. The 25th percentile value of this distribution is set as the contrast threshold and written to the configuration file. It should be noted that choosing a lower 25th percentile value as the threshold aims to adopt a more sensitive enhancement strategy; that is, as long as the scene's inherent material contrast is lower than the level of the worse visually clear samples, light and shadow enhancement is triggered to ensure that the final output visual performance has a minimum guarantee.

[0119] The system compares the contrast threshold read from the configuration file with the current brightness contrast variable. If the brightness contrast is lower than the threshold, the system initiates lighting enhancement. The enhancement process first deploys a series of virtual point lights in the direction of the normal vector. Then, it uses a photon mapping algorithm to calculate the radiant flux distribution. The algorithm is implemented in two stages. In the first stage, photons are emitted from the virtual point lights into the scene, and their positions and energy are stored after interacting with the surfaces. In the second stage, during rendering, for each surface point, the energy of all photons within a certain radius is collected and weighted to generate a local light enhancement matrix variable representing the additional illumination intensity.

[0120] The system performs lighting and shadow baking on a 3D surface mesh using a screen-space ambient occlusion algorithm based on the local light enhancement matrix. This algorithm utilizes the depth buffer of the current viewpoint, sampling multiple points in the surrounding hemisphere for each pixel, checking whether these points are occluded by scene geometry, and calculating the ambient occlusion factor based on the proportion of occluded sampling points. This factor is then multiplied by the local light enhancement matrix to generate a lighting and shadow texture map variable representing detailed shadows.

[0121] Finally, the system uses the alpha fusion algorithm to perform pixel-by-pixel fusion of the lighting texture map and the basic material attribute layer. This operation weights and sums the color values ​​of the basic material attribute layer with the corresponding brightness values ​​of the lighting texture map according to a fixed transparency coefficient, generating the final color and material attributes for each pixel. The fusion result becomes the texture model variable. This model not only includes the physical material attributes of the object's surface but also incorporates lighting details adjusted based on emotional atmosphere, significantly enhancing the visual realism and emotional tension of the 3D scene, directly serving the invention's objective of achieving narrative rendering of textures and lighting.

[0122] This step transforms the geometric skeleton into visualized surface properties, which is a key step in achieving a visually immersive 3D scene.

[0123] In step S17, the lighting and shadow parameters are iteratively adjusted according to the texture model to obtain the rendering configuration parameters.

[0124] In one specific implementation, the step of iteratively adjusting the lighting and shadow parameters according to the texture model to obtain the rendering configuration parameters includes:

[0125] Based on the texture model, color, normal, and specular intensity data are extracted through channel separation operations and combined to generate a rendering input data stream.

[0126] Based on the rendering input data stream, the contrast and edge density of each region are calculated using the Itti visual saliency model, and a weighted sum is performed to obtain the visual tension index.

[0127] When the visual tension index is lower than the preset visual tension threshold, the difference between the visual tension index and the visual tension threshold is converted into a light source direction offset angle and color temperature adjustment value according to the preset mapping rule, and a light and shadow parameter adjustment vector is generated.

[0128] The vector is adjusted according to the light and shadow parameters, and a convolution operation is performed using the cook-torrance reflectivity equation to synthesize a drama enhancement matrix.

[0129] The drama enhancement matrix and the rendering input data stream are weighted and fused using the alpha mixing algorithm to obtain the rendering configuration parameters.

[0130] When the visual tension index is lower than the preset visual tension threshold, the rendering input data stream is directly used as the rendering configuration parameter.

[0131] Specifically, first, the system performs channel separation on the texture model to extract basic rendering data. The texture model contains multiple image layers. The system reads RGB channel data storing color information as color data, channel data storing surface orientation information as normal data, and channel data storing material reflection characteristics as specular intensity data. The system combines these data according to a fixed memory layout to form a multi-channel rendering input data stream variable, which serves as the basis for subsequent evaluation and adjustments.

[0132] Next, the system uses a pre-built Itti visual saliency model to analyze the rendered input data stream. The model's computation process simulates early human visual attention mechanisms. Specifically, it first constructs Gaussian pyramids for the input color, brightness, and orientation features, performing Gaussian blurring and downsampling at multiple scales. Then, by calculating the differences between features at adjacent scales within the pyramid, a series of feature maps reflecting local contrast are obtained. Edge density is calculated by applying a multi-directional Gabor filter bank to the brightness channel and summing the amplitudes of the filtering responses in each region. The system then weights and sums the contrast and edge density feature values ​​of each region according to a set of preset weights. These weights are optimized through regression analysis on a large amount of user eye-tracking data, aiming to maximize the correlation between the weighted result and the user's actual gaze point distribution. The weighted sum is the visual tension index variable.

[0133] The visual tension threshold is determined based on historical data statistics. During the system development phase, a large number of scene rendering images rated as having "sufficient visual tension" by professional visual artists are collected, and their visual tension indices are calculated to form a numerical distribution. The 30th percentile value of this distribution is set as the visual tension threshold and written to the system configuration file. The system compares the visual tension threshold read from the configuration file with the currently calculated visual tension index variable.

[0134] If the visual tension index is not lower than the threshold, the system will directly output the rendering input data stream variables as rendering configuration parameter variables.

[0135] If the visual tension index falls below the threshold, the system initiates an iterative adjustment process for lighting parameters. Based on a preset mapping rule, the system converts the difference between the visual tension index and the visual tension threshold into specific lighting adjustment parameters. This mapping rule is defined as a piecewise linear function, where the difference is mapped to two parts: one part is the offset angle of the light source direction in the horizontal and vertical directions, and the other part is the adjustment value of the light source color temperature. These two values ​​together constitute the lighting parameter adjustment vector variable.

[0136] Subsequently, the system adjusts the vector based on lighting parameters and synthesizes dramatic enhancement effects using the Cook-Torrance reflectivity equation. The Cook-Torrance equation is a physically based microsurface reflection model constructed based on geometric attenuation factors, normal distribution functions, and Fresnel equations. Specifically, the system takes the current surface normal data, specular intensity data, and adjusted light source direction and color temperature as input to calculate the specular highlight intensity distribution for each surface point in the scene under the new lighting conditions. This calculation is performed in parallel on the GPU, and the result is stored as a dramatic enhancement matrix variable with the same resolution as the original data stream. This matrix primarily enhances the contrast between highlights and shadows.

[0137] Finally, the system fuses the dramatic enhancement matrix with the original rendering input data stream using the Alpha Blending algorithm. This algorithm assigns a fixed blending coefficient to each pixel, weighting and adding the adjusted value of that pixel in the dramatic enhancement matrix to the corresponding color value in the original data stream, thus obtaining the adjusted final color, normal, and specular data. The fused result becomes the rendering configuration parameter variable. This step, through the scientific and automated enhancement of lighting and shadows, significantly improves the visual expressiveness and emotional saturation of the scene, ensuring that the generated 3D scene effectively conveys the intended narrative atmosphere, directly serving the invention's objective of achieving narrative rendering of lighting and shadows and enhancing visual tension.

[0138] This step ensures that the generated 3D scene has strong visual appeal and emotional impact by calculating visual saliency and dynamically adjusting lighting parameters.

[0139] In step S18, the three-dimensional scene is drawn and pixel is corrected according to the rendering configuration parameters to obtain three-dimensional scene data.

[0140] In one specific implementation, the step of performing 3D scene rendering and pixel correction based on the rendering configuration parameters to obtain 3D scene data includes:

[0141] Based on the rendering configuration parameters, a rasterization instruction set is constructed through the graphics processing interface to drive the graphics processing unit to perform real-time rendering and obtain real-time rendering data.

[0142] The real-time rendering data is input into a generative adversarial network, and the visual semantic features of the image are extracted through the encoder.

[0143] Calculate the cosine similarity between the visual semantic features and the user intent features to obtain the feature similarity;

[0144] Based on the feature similarity, the gradient of the real-time rendering data is calculated using the backpropagation algorithm to generate a feedback gradient field;

[0145] The generator of the generative adversarial network is driven by the feedback gradient field to synthesize a pixel correction mask;

[0146] Based on the pixel correction mask and the real-time rendering data, pixel-by-pixel fusion is performed using the alpha fusion algorithm to obtain 3D scene data.

[0147] Specifically, first, the system drives the graphics processing unit to perform real-time rendering based on rendering configuration parameters. These parameters include color data, normal data, specular intensity data, and lighting parameters. The system calls the graphics processing interface to construct a rasterization instruction set based on these parameters. This process specifically includes setting vertex and index buffers to define geometry, binding textures containing color, normal, and other data, configuring shader programs to execute Cook-Torrance lighting model calculations, and setting the viewport and projection matrix. The graphics processing unit executes these instructions, using pixel shaders to calculate the final color of each screen pixel in parallel, generating a two-dimensional image. This image data is stored as real-time rendering data variables.

[0148] Subsequently, the system inputs the real-time rendered data into a pre-built generative adversarial network (GAN) for semantic evaluation and correction. This GAN consists of a generator with an encoder-decoder structure and a discriminator. The network's training data comes from a publicly available large-scale 3D scene dataset, which contains numerous high-quality scene models and their multi-angle rendered images, each image associated with its standardized semantic description vector. The training process is as follows: In each training round, a batch of real rendered images and their semantic vectors are sampled from the dataset. The generator's input is a version of the real image superimposed with Gaussian noise and its corresponding semantic vector; its goal is to output a denoised image with enhanced semantic consistency. The discriminator's input is either a real image or the image output by the generator; its goal is to determine the authenticity of the input image. Training uses adversarial loss and semantic vector-based reconstruction loss for optimization, updating the parameters of the generator and discriminator alternately for tens of thousands of iterations until the loss function converges. After training, the system fixes the network parameters. In this step, the network is used in two parts. First, the real-time rendered data is fed into the encoder part of the GAN. The encoder consists of multiple convolutional and pooling layers. It downsamples the input image and extracts abstract features through convolution and nonlinear activation operations, and finally outputs a low-dimensional dense vector, which is defined as a visual semantic feature variable.

[0149] Next, the system calculates the semantic consistency between the visual semantic feature variable and the user intent feature variable obtained from step S14. The specific calculation method is cosine similarity, which is the ratio of the sum of the products of each dimension of the two feature vectors to the product of the magnitudes of the two vectors, resulting in a scalar value between negative one and positive one. This value is defined as the feature similarity variable.

[0150] The system presets a feature similarity threshold to determine whether pixel correction needs to be initiated. This threshold is determined based on the network's performance on an independent validation set during the training phase. On the validation set, the cosine similarity between the visual semantic features of all samples and their corresponding ground truth semantic vectors is calculated, and their distribution is statistically analyzed. This ensures that the correction initiation mechanism achieves an optimal balance on the validation set, minimizing unnecessary corrections while ensuring that all rendering results that significantly deviate from the intent receive the corresponding similarity cutoff point for correction. This cutoff point is then set as the feature similarity threshold. Specifically, by searching a grid for different percentile values ​​(such as the 5th, 10th, and 15th percentiles), the percentile value that maximizes the overall improvement in the final matching degree between the corrected image and the intent vector is selected. The final threshold, for example, corresponding to the 10th percentile, is fixed in the system configuration.

[0151] The system compares the feature similarity threshold read from the configuration file with the currently calculated feature similarity variable. If the feature similarity is not lower than the threshold, the system directly outputs the real-time rendered data variable as the final 3D scene data variable.

[0152] If the feature similarity is below the threshold, the system initiates a pixel correction process. This correction process generates a guidance signal based on the backpropagation algorithm. The system uses feature similarity as a loss value and calculates the gradient of this loss with respect to the color value of each pixel in the real-time rendering data variables using the backpropagation algorithm. This calculation process starts from the encoder output layer of the network and uses the chain rule to backpropagate the gradient layer by layer until it reaches the input layer of the network, ultimately obtaining a gradient tensor of the same size as the real-time rendering data. This tensor is defined as the feedback gradient field variable.

[0153] The system then uses this feedback gradient field to drive the decoder part of the generator in the generative adversarial network. The decoder consists of multiple transposed convolutional layers. The system concatenates the visual semantic feature variables extracted by the encoder with the feedback gradient field as the input to the decoder. The decoder upsamples the input and reconstructs it into a two-dimensional image through a series of transposed convolutions and nonlinear activation operations; this image serves as the pixel-corrected mask variable.

[0154] The fusion coefficients used in the Alpha Hybrid algorithm are determined based on historical data statistics and an objective image semantic similarity evaluation model. This evaluation model is a cross-modal matching model pre-trained on a large-scale public image-text pairing dataset. The model employs a dual-tower neural network architecture, where the image encoder uses a pre-trained ResNet network, and the text encoder uses a pre-trained BERT model. The training objective is to maximize the cosine similarity of the encoded vectors of matched image-text pairs in a shared semantic space. After training, the model can compute an objective matching score for any image and semantic description vector; this score represents the cosine similarity between the two encoded vectors.

[0155] During system development, a separate test set was used, containing scenarios where the rendered results deviated from the target semantics to varying degrees. For each test scenario, the system attempted multiple different fusion coefficients for pixel correction and used the aforementioned cross-modal matching model to calculate the matching score between the corrected image and the target semantic description vector. The distribution of the fusion coefficients that achieved the highest matching score across all test scenarios was statistically analyzed, and the 95th percentile of this distribution was set as the final fixed fusion coefficient. This high percentile selection ensured that the correction intensity was sufficient and not excessive in most cases.

[0156] Finally, the system fuses the pixel correction mask with the real-time rendering data using the alpha fusion algorithm. This operation calculates the final color for each pixel, weighting the original color value in the real-time rendering data with the corresponding adjustment value in the pixel correction mask according to a fixed fusion coefficient read from the configuration file. The fused image data is the final 3D scene data variable output. This step, through an objective and quantifiable semantic evaluation loop and pixel-level optimization, ensures that the generated 3D scene visually closely matches the user's deep intent, marking the final completion of the entire semantically description-based 3D scene creation process.

[0157] Reference Figure 2 The second embodiment of the present invention provides a three-dimensional scene creation system based on virtual and real-world scripts, including:

[0158] The data acquisition module is used to obtain users' semantic description text and user interaction logs;

[0159] The semantic feature analysis module is used to perform deep semantic parsing based on the semantic description text, extract emotional atmosphere and narrative logic features, and obtain semantic feature vectors.

[0160] The visual element analysis module is used to match and filter visual elements in a pre-stored visual element resource library based on the semantic feature vector to obtain a list of visual elements.

[0161] The layout parameter analysis module is used to generate the spatial position and layout relationship of the visual elements in the three-dimensional virtual scene based on the visual element list and the user interaction log, and obtain the layout parameter set.

[0162] The scene skeleton construction module is used to establish topological connection relationships and generate a scene skeleton structure based on the layout parameter set.

[0163] The texture model construction module is used to map and synthesize surface textures based on the scene skeleton structure to obtain a texture model;

[0164] The rendering configuration module is used to iteratively adjust the lighting and shadow parameters according to the texture model to obtain the rendering configuration parameters;

[0165] The 3D scene generation module is used to draw and correct pixels of the 3D scene according to the rendering configuration parameters to obtain 3D scene data.

[0166] It should be noted that the three-dimensional scene creation system based on virtual and real scripts provided in this embodiment of the invention is used to execute all the process steps of the three-dimensional scene creation method based on virtual and real scripts in the above embodiment. The working principles and beneficial effects of the two are one-to-one, so they will not be described again.

[0167] It should be noted that the system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the system embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0168] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention for those skilled in the art.

Claims

1. A method for creating a 3D scene based on virtual and real-world scripts, characterized in that, include: Obtain the user's semantic description text and user interaction logs; Based on the semantic description text, deep semantic analysis is performed to extract emotional atmosphere and narrative logic features, resulting in a semantic feature vector. Based on the semantic feature vector, the visual elements in the pre-stored visual element resource library are matched and filtered to obtain a list of visual elements; Based on the visual element list and the user interaction log, user intent features are extracted using a latent semantic analysis model. A weighted average of the feature values ​​in each dimension of the user intent features is calculated according to a preset intent weight to obtain a dynamic adaptation factor. Based on the visual element list, spatial location mapping is performed according to semantic type and semantic matching confidence to obtain a set of layout coordinates. Based on the set of layout coordinates and the dynamic adaptation factor, a conditional generative adversarial network is used to model the spatial probability distribution, generating a probability distribution layout heatmap containing pixel values. Based on the probability distribution layout heatmap, the pixel values ​​are sampled using a mean-shift clustering algorithm to generate a set of layout variants. Based on the set of layout variants, the centroid coordinates are obtained through normalized central moments, and the Euclidean distance between the centroid coordinates and the geometric center of the image is calculated to obtain visual balance. Based on the set of layout variants, layout variants with visual balance less than or equal to a preset visual balance threshold are selected to obtain a candidate layout set. Based on the candidate layout set, the geometric transformation matrix of the candidate layouts is calculated through affine transformation to obtain a set of layout parameters. Based on the set of layout parameters, the topological connection relationship between the spatial coordinates of the layout parameters is established by the Delaunay triangulation algorithm to generate the initial scene skeleton structure. Based on the initial scene skeleton structure, the theoretical maximum sum of edge weights is calculated, and the sum of the actual weights of all connected edges is calculated. This sum is then compared to the theoretical maximum sum of edge weights to obtain the narrative logic strength. The weight value is the cosine similarity between the semantic feature vectors of the two visual elements connected by the corresponding edge. When the narrative logic strength exceeds a preset logic strength threshold, the event trigger probability of potential locations is calculated using a Bayesian inference algorithm based on scene semantic tags and a pre-stored interaction event knowledge base. The potential location with the highest event trigger probability is selected as the interaction node. Based on the interaction node, the initial scene skeleton structure is topologically reconstructed using node insertion and edge reconnection operations in the graph data structure to obtain the scene skeleton structure. When the narrative logic strength does not exceed the preset logic strength threshold, the initial scene skeleton structure is used as the final scene skeleton structure. Based on the scene skeleton structure, surface textures are mapped and synthesized to obtain a texture model; Based on the texture model, the lighting and shadow parameters are iteratively adjusted to obtain the rendering configuration parameters; Based on the rendering configuration parameters, 3D scene rendering and pixel correction are performed to obtain 3D scene data.

2. The method for establishing a three-dimensional scene based on virtual and real-world scripts according to claim 1, characterized in that, The step involves performing deep semantic analysis on the semantic description text to extract emotional atmosphere and narrative logic features, resulting in a semantic feature vector, including: Based on the semantic description text, the Stanford Dependency Parser is used to parse the syntactic structure, extract the subject-verb-object structure in the text as logical nodes, obtain the logical node sequence, and calculate the cosine similarity between adjacent nodes as the logical association degree. Based on the logical node sequence, the logical nodes are assigned sentiment polarity values ​​using the CNKI sentiment dictionary to obtain a sentiment polarity value sequence. The sentiment polarity values ​​are then arranged according to the order of the corresponding logical nodes in the original text to form a discrete sentiment value sequence. The discrete sentiment value sequence is fitted using a cubic spline interpolation algorithm to generate a sentiment intensity variation curve and the arithmetic mean of the sentiment polarity value sequence is calculated to obtain the sentiment mean of the sequence. The absolute difference between the emotional polarity value and the mean emotional value of the sequence is calculated to obtain the degree of emotional deviation. Logical nodes whose degree of emotional deviation exceeds the preset emotional deviation threshold are selected to obtain a set of core logical nodes. The text of the core logical nodes is encoded into a dense vector of fixed dimension using a pre-trained BERT model. The mean, standard deviation, range, and number of positive and negative changes in the slope of the emotion polarity value are extracted from the emotion intensity change curve to obtain the emotion statistical characteristics; The dense vector is subjected to mean pooling to obtain a comprehensive node vector, and the comprehensive node vector is concatenated with the sentiment statistical features to obtain a semantic feature vector.

3. The method for establishing a three-dimensional scene based on virtual and real-world scripts according to claim 1, characterized in that, The step of matching and filtering visual elements in a pre-stored visual element resource library based on the semantic feature vector to obtain a list of visual elements includes: Based on the semantic feature vector, the feature space is decoupled by principal component analysis to obtain the continuous emotional component and the discrete logical component. Based on the continuous emotion components, the corresponding visual attribute label set and scene semantic label are mapped out through a multilayer perceptron classification model; Based on the set of visual attribute labels and the logical discrete components, a hybrid retrieval vector is constructed through vector concatenation operations; The similarity between the hybrid retrieval vector and the visual elements in the pre-stored visual element resource library is calculated using the cosine similarity algorithm to obtain a semantic matching confidence set. Based on the set of semantic matching confidence scores, visual elements that exceed a preset semantic matching confidence threshold are selected, and the selected visual elements are sorted in descending order of semantic matching confidence scores to generate a list of visual elements containing semantic matching confidence scores and semantic types.

4. The method for establishing a three-dimensional scene based on virtual and real-world scripts according to claim 3, characterized in that, The step of mapping and synthesizing surface textures based on the scene skeleton structure to obtain a texture model includes: Based on the scene skeleton structure, a three-dimensional surface mesh is reconstructed using the traveling cube algorithm to obtain a three-dimensional surface mesh. Based on the three-dimensional surface mesh, the corresponding texture coordinate mapping parameters are calculated using the least squares conformal mapping algorithm; Based on the texture coordinate mapping parameters and the preset material parameter database, a basic material property layer containing diffuse reflection coefficient, normal vector and roughness value is generated for the three-dimensional surface mesh through the pre-constructed micro-surface model; For the diffuse reflection coefficient, the color value of each pixel is converted into the corresponding grayscale brightness value using a preset standard brightness conversion formula, thereby obtaining a grayscale image; The contrast ratio is obtained by calculating the standard deviation of all grayscale brightness values ​​in the grayscale image; When the brightness-dark contrast is lower than a preset contrast threshold, a virtual point light source is deployed in the direction of the normal vector, and the radiant flux distribution is calculated using a photon mapping algorithm to generate a local light source enhancement matrix. Based on the local light source enhancement matrix, a light and shadow baking is performed on the three-dimensional surface mesh using a screen space ambient light occlusion algorithm to generate a light and shadow texture map. The light and shadow texture map and the basic material attribute layer are fused pixel by pixel using the alpha mixing algorithm to obtain the texture model.

5. The method for establishing a three-dimensional scene based on virtual and real-world scripts according to claim 1, characterized in that, The step of iteratively adjusting the lighting and shadow parameters based on the texture model to obtain rendering configuration parameters includes: Based on the texture model, color, normal, and specular intensity data are extracted through channel separation operations and combined to generate a rendering input data stream. Based on the rendering input data stream, the contrast and edge density of each region are calculated using the Itti visual saliency model, and a weighted sum is performed to obtain the visual tension index. When the visual tension index is lower than the preset visual tension threshold, the difference between the visual tension index and the visual tension threshold is converted into a light source direction offset angle and color temperature adjustment value according to the preset mapping rule, and a light and shadow parameter adjustment vector is generated. The vector is adjusted according to the light and shadow parameters, and a convolution operation is performed using the cook-torrance reflectivity equation to synthesize a drama enhancement matrix. The drama enhancement matrix and the rendering input data stream are weighted and fused using the alpha mixing algorithm to obtain the rendering configuration parameters. When the visual tension index is lower than the preset visual tension threshold, the rendering input data stream is directly used as the rendering configuration parameter.

6. The method for establishing a three-dimensional scene based on virtual and real-world scripts according to claim 4, characterized in that, The step of rendering and pixel correction of the 3D scene according to the rendering configuration parameters to obtain 3D scene data includes: Based on the rendering configuration parameters, a rasterization instruction set is constructed through the graphics processing interface to drive the graphics processing unit to perform real-time rendering and obtain real-time rendering data. The real-time rendering data is input into a generative adversarial network, and the visual semantic features of the image are extracted through the encoder. Calculate the cosine similarity between the visual semantic features and the user intent features to obtain the feature similarity; Based on the feature similarity, the gradient of the real-time rendering data is calculated using the backpropagation algorithm to generate a feedback gradient field; The generator of the generative adversarial network is driven by the feedback gradient field to synthesize a pixel correction mask; Based on the pixel correction mask and the real-time rendering data, pixel-by-pixel fusion is performed using the alpha fusion algorithm to obtain 3D scene data.

7. A three-dimensional scene creation system based on virtual and real-world scripts, characterized in that, include: The data acquisition module is used to obtain users' semantic description text and user interaction logs; The semantic feature analysis module is used to perform deep semantic parsing based on the semantic description text, extract emotional atmosphere and narrative logic features, and obtain semantic feature vectors. The visual element analysis module is used to match and filter visual elements in a pre-stored visual element resource library based on the semantic feature vector to obtain a list of visual elements. The layout parameter analysis module is used to extract user intent features based on the visual element list and the user interaction log using a latent semantic analysis model, and to calculate a dynamic adaptation factor by weighting the feature values ​​of each dimension of the user intent features according to a preset intent weight; to obtain a layout coordinate set by mapping spatial positions according to the visual element list based on semantic type and semantic matching confidence; to generate a probability distribution layout heatmap containing pixel values ​​by modeling spatial probability distribution using a conditional generative adversarial network based on the layout coordinate set and the dynamic adaptation factor; to generate a layout variant set by sampling the pixel values ​​using a mean-shift clustering algorithm based on the probability distribution layout heatmap; to obtain a centroid coordinate set by calculating the barycenter coordinates and the Euclidean distance between the centroid coordinates and the geometric center of the image based on the layout variant set; to select layout variants whose visual balance is less than or equal to a preset visual balance threshold based on the layout variant set; and to obtain a layout parameter set by calculating the geometric transformation matrix of the candidate layouts using affine transformation based on the candidate layout set. The scene skeleton construction module is used to establish the topological connection relationship between the spatial coordinates of the layout parameters based on the layout parameter set and to generate the initial scene skeleton structure by using the Delaunay triangulation algorithm. Based on the initial scene skeleton structure, the theoretical maximum sum of edge weights is calculated, and the sum of the actual weights of all connected edges is calculated. This sum is then compared to the theoretical maximum sum of edge weights to obtain the narrative logic strength. The weight value is the cosine similarity between the semantic feature vectors of the two visual elements connected by the corresponding edge. When the narrative logic strength exceeds a preset logic strength threshold, the event trigger probability of potential locations is calculated using a Bayesian inference algorithm based on scene semantic tags and a pre-stored interaction event knowledge base. The potential location with the highest event trigger probability is selected as the interaction node. Based on the interaction node, the initial scene skeleton structure is topologically reconstructed using node insertion and edge reconnection operations in the graph data structure to obtain the scene skeleton structure. When the narrative logic strength does not exceed the preset logic strength threshold, the initial scene skeleton structure is used as the final scene skeleton structure. The texture model construction module is used to map and synthesize surface textures based on the scene skeleton structure to obtain a texture model; The rendering configuration module is used to iteratively adjust the lighting and shadow parameters according to the texture model to obtain the rendering configuration parameters; The 3D scene generation module is used to draw and correct pixels of the 3D scene according to the rendering configuration parameters to obtain 3D scene data.

Images