Virtual reality content generation model training method and device based on AI large model

Abstract

The invention relates to the technical field of model training, and discloses a virtual reality content generation model training method and device based on an AI large model, and the method comprises the steps: obtaining virtual reality training data facing a building digital twin scene; preprocessing the virtual reality training data to obtain preprocessed data adapted to AI large model training; constructing an AI large model comprising a generator and a discriminator; training the AI large model based on the preprocessing to obtain a trained AI large model; and inputting a potential vector into the trained AI large model to enable the trained AI large model to output virtual reality content matched with the building digital twin scene. According to the method, the geometric structure data, the material and texture information, the environment sensor data and the dynamic scene data are combined, the high-dimensional joint feature expression is constructed, more comprehensive and accurate building virtual reality modeling is realized, and in addition, through the preprocessing steps of space-time calibration, normalization, noise filtering and the like, the real-time performance of the building is improved.

Description

Technical Field

[0001] This invention relates to the field of model training technology, and more specifically to a method and apparatus for training virtual reality content generation models based on large AI models. Background Technology

[0002] With the rapid development of digital twin technology in the fields of architecture, urban planning, and smart cities, virtual reality content generation has become an important component of building information modeling, intelligent building management, and urban digital transformation. The core objective of building digital twins is to construct digital models that are highly consistent with physical buildings to support applications such as visualization simulation, operation and maintenance optimization, disaster early warning, and energy-saving analysis. However, traditional methods for constructing virtual reality building data typically rely on manual modeling or rule-driven procedural generation. These methods have many shortcomings in terms of data acquisition cost, modeling accuracy, adaptability, and computational efficiency, making it difficult to meet the needs of practical applications.

[0003] In recent years, artificial intelligence has made significant progress in image synthesis, 3D modeling, and virtual reality content generation, providing an automated, efficient, and intelligent content generation solution for building digital twins.

[0004] For example, Chinese invention patent CN119598195A proposes a method for industrial load data augmentation based on recurrent convolutional generative adversarial networks (GANs). The method includes acquiring historical industrial load data and dividing it into training and testing sets; constructing a generator and discriminator based on a multilayer perceptron network (MLPF), forming a GAN model, and constructing loss functions for both the generator and the GAN model; optimizing the generator's parameters based on the training set to obtain an optimized generator; optimizing the discriminator's parameters using the GAN model's loss function based on the optimized generator's output data to obtain an optimized discriminator; combining the optimized generator and discriminator to form an optimized GAN model; and inputting the test set into the optimized GAN model to output the augmented data. This approach enhances the model's ability to process complex data, better captures the spatiotemporal relationship of load, and thus provides reliable data to ensure rational, economical, and efficient power grid planning.

[0005] However, existing technologies only focus on pixel-level similarity and cannot maintain the global topological structure, resulting in distorted geometric structure of the generated data. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention proposes a training method and apparatus for virtual reality content generation models based on large AI models, in order to solve the aforementioned technical problems.

[0007] Firstly, it provides a training method for virtual reality content generation models based on large AI models, including: Acquire virtual reality training data for building digital twin scenarios; The virtual reality training data is preprocessed to obtain preprocessed data adapted for training large AI models; Build a large AI model that includes a generator and a discriminator; The AI ​​large model is trained based on the preprocessing to obtain the trained AI large model; By inputting latent vectors into the trained AI model, the trained AI model outputs virtual reality content that matches the building's digital twin scene.

[0008] Furthermore, the virtual reality training data is preprocessed to obtain preprocessed data adapted for AI large-scale model training, including: Acquire multi-source heterogeneous building-related data; Multimodal alignment processing is performed on multi-source heterogeneous building-related data to obtain building-related data with consistent spatiotemporal dimensions; The multimodal aligned building-related data is normalized to obtain normalized building-related data. The normalized building-related data is subjected to noise filtering to obtain building-related data with outliers removed. High-dimensional feature vectors are constructed from building-related data after removing outliers to obtain preprocessed data suitable for training large AI models.

[0009] Furthermore, a large AI model including a generator and a discriminator is constructed, and the large AI model is trained based on the preprocessing to obtain a trained large AI model, including: Construct the network structure of the generator; The weight matrix and bias terms of the generator are initialized to obtain the initialized generator; Construct the network structure of the discriminator; The weight matrix and bias terms of the discriminator are initialized to obtain the initialized discriminator; Based on the preprocessed data, the parameters of the initialized generator and discriminator are optimized to obtain the trained AI large model.

[0010] Furthermore, based on the preprocessed data, the parameters of the initialized generator and discriminator are optimized, including: By statistically analyzing the mean and variance of the preprocessed data, the data distribution characteristics of the preprocessed data can be obtained. The noise level in the preprocessed data is detected to obtain the noise suppression factor; Based on the data distribution characteristics and noise suppression factor, the learning rates of the generator and the discriminator are dynamically adjusted to obtain a dynamic learning rate that adapts to the data distribution and noise level. Based on the adjusted learning rate, the weight matrices of the generator and the discriminator are updated to obtain the post-generator and discriminator with optimized parameters.

[0011] Furthermore, training the large AI model based on the preprocessing also includes: A topological constraint term is introduced into the damage function of the generator, wherein the topological constraint term includes a geometric constraint term and a material environment property constraint term; The geometric structure constraints are used to measure the similarity in geometric shape between the generated data output by the generator and the real building data, constrain the local domain topological relationship of the generated data, and obtain geometrically matched generated data. By using the material and environmental attribute constraints, the continuity of the generated data output by the generator with the real building data in terms of material texture and environmental attributes is measured, the spatial smoothness of the generated data is constrained, and generated data with continuous material and environmental attributes is obtained. By combining the generated data of geometric shape matching and the generated data of continuous material and environmental attributes, generated data with consistent topological structure and the corresponding intermediate state of AI large model training are obtained.

[0012] Furthermore, training the large AI model based on the preprocessing also includes: Calculate the gradient of the generator loss function with respect to its weight matrix, correct the direction of the gradient and remove noise to obtain the cleaned generator loss gradient; Calculate the gradient of the discriminator loss function with respect to its weight matrix, filter out the amplitude fluctuations of the gradient, and obtain the purified discriminator loss gradient. The parameters of the generator are updated based on the gradient loss of the purified generator to obtain the generator with updated parameters after gradient purification. The parameters of the discriminator are updated based on the gradient loss of the purified discriminator to obtain the discriminator with updated parameters after gradient purification.

[0013] Furthermore, training the AI ​​large model based on the preprocessed data also includes: The discriminator determines the authenticity of the generated data output by the generator, and based on the determination result and the feedback of the generator's loss function, high-quality generated data that meets the preset authenticity requirements are selected. An improved autoencoder model is constructed, which includes a multi-branch encoder and a multi-branch decoder for processing different types of data, to obtain an autoencoder model adapted for multimodal data augmentation. The high-quality generated data is input into the autoencoder model, and through latent space interpolation and latent space perturbation, diverse augmented data is obtained; The augmented data is added to the preprocessed data for iterative training of the AI ​​large model, resulting in an expanded preprocessed dataset containing the augmented data and an iteratively trained AI large model.

[0014] Furthermore, by inputting latent vectors into the trained AI model, the trained AI model outputs virtual reality content that matches the building's digital twin scene, including: The training process of the AI ​​large model is monitored. When the training meets the preset stopping iteration condition, the training of the AI ​​large model is determined to be completed, and the trained AI large model is obtained. Input vectors from the latent space into the generator of the trained large AI model; The generator processes the input latent space vector to obtain virtual reality content that includes architectural geometry, material textures, and environmental attributes. The output virtual reality content is applied to the visualization simulation, operation and maintenance decision support, or risk analysis scenarios of building digital twins to obtain the application results for the corresponding scenarios.

[0015] Secondly, a training device for a virtual reality content generation model based on an AI large-scale model is provided, based on any of the preceding methods for training a virtual reality content generation model based on an AI large-scale model, including: The acquisition module is configured to acquire virtual reality training data for building digital twin scenarios; The preprocessing module is configured to preprocess the virtual reality training data to obtain preprocessed data adapted for training large AI models. The building module is configured to build large AI models that include generators and discriminators; The training module is configured to train the AI ​​large model based on the preprocessing to obtain the trained AI large model; The output module is configured to input latent vectors into the trained AI model, causing the trained AI model to output virtual reality content that matches the building digital twin scene.

[0016] Thirdly, a terminal includes a processor, an input device, an output device, and a memory, wherein the processor, input device, output device, and memory are interconnected, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to invoke the program instructions to execute the virtual reality content generation model training method based on the AI ​​large model described in any of the preceding claims.

[0017] The invention employing the above technical solution has the following advantages: 1. This invention combines geometric structure data, material and texture information, environmental sensor data and dynamic scene data to construct a high-dimensional joint feature representation, thereby achieving more comprehensive and accurate virtual reality modeling of buildings. In addition, through preprocessing steps such as spatiotemporal calibration, normalization and noise filtering, the uniformity and consistency of data are ensured, thereby improving the generalization ability of the AI ​​large model.

[0018] 2. This invention adopts a generative adversarial network structure, including a generator and a discriminator. Through adversarial training, the generation quality is continuously optimized, making virtual reality data more realistic. In addition, by combining multimodal data input, the generative adversarial network can learn the global features of the architectural scene, rather than just single-dimensional information, thereby improving the spatial consistency and structural rationality of the generated data.

[0019] 3. This invention proposes parameter sensitivity adaptive adjustment, which dynamically adjusts the learning rate based on the statistical distribution of the data and the noise level; it automatically reduces the learning rate in dense data areas to prevent mode collapse; it increases the learning rate in sparse data areas to accelerate convergence; and it automatically reduces the gradient update amplitude in noisy data to ensure training stability.

[0020] 4. This invention proposes geometric structure constraints (to ensure the topological consistency of point cloud or mesh data) and material and environmental attribute constraints (to maintain the spatial consistency of texture, lighting, and environmental data). Through these constraints, the generated architectural virtual reality data is not only more realistic in local details, but also more stable and reasonable in overall topological structure.

[0021] 5. This invention proposes generator gradient cleanup (improving training stability through gradient direction correction and noise suppression) and discriminator gradient cleanup (retaining gradient direction information based on the sign function and removing useless gradient magnitude fluctuations), which can effectively improve the training stability of generative adversarial networks, reduce pattern collapse, and improve the realism of generated data.

[0022] 6. This invention employs a discriminator feedback mechanism to filter high-quality generated data and eliminate low-quality or noisy samples. In the early stages of training, the discriminator sets a high threshold to retain only high-quality data. As training progresses, the threshold is gradually reduced to ensure that the filtered data always meets the current model's capability level, thereby improving the quality of generated data. Attached Figure Description

[0023] To more clearly illustrate the specific embodiments of the present invention, the accompanying drawings used in the specific embodiments will be briefly described below. In all the drawings, the elements or parts are not necessarily drawn to scale.

[0024] Figure 1This is a flowchart of the virtual reality content generation model training method based on a large AI model according to the present invention. Figure 2 This is a comparison chart of training stability in the training method of the virtual reality content generation model based on the AI ​​large model of the present invention. Figure 3 This is an experimental diagram of a real building's circular staircase structure used in the training method for virtual reality content generation model based on a large AI model, as described in this invention. Figure 4 This is the result generated by the traditional method in the training method of virtual reality content generation model based on AI large model of the present invention. Figure 1 ; Figure 5 This is the result generated by the traditional method in the training method of virtual reality content generation model based on AI large model of the present invention. Figure 2 ; Figure 6 This is a comparison diagram of geometric structure consistency in the training method of virtual reality content generation model based on AI large model of the present invention; Figure 7 This is a comparison diagram of material texture continuity in the training method of virtual reality content generation model based on AI large model of the present invention; Figure 8 This is a comparison chart of the consistency of environmental sensor data in the training method of virtual reality content generation model based on AI large model of the present invention. Figure 9 This diagram illustrates the impact of the gradient purification mechanism on training stability in the AI-based large-scale virtual reality content generation model training method of this invention. Detailed Implementation

[0025] The embodiments of the technical solution of the present invention will now be described in detail with reference to the accompanying drawings. These embodiments are merely illustrative of the technical solution of the present invention and are therefore intended to limit the scope of protection of the present invention.

[0026] like Figures 1-9 As shown, the virtual reality content generation model training method based on AI large model of the present invention includes: Acquire virtual reality training data for building digital twin scenarios; Preprocess the virtual reality training data to obtain preprocessed data suitable for training large AI models; Build a large AI model that includes a generator and a discriminator; The AI ​​large model is trained based on preprocessing to obtain the trained AI large model; By inputting latent vectors into a trained AI model, the trained AI model can output virtual reality content that matches the building's digital twin scene.

[0027] In this embodiment, the virtual reality training data is preprocessed to obtain preprocessed data adapted for training large AI models, including: Acquire multi-source heterogeneous building-related data; Multimodal alignment processing is performed on multi-source heterogeneous building-related data to obtain building-related data with consistent spatiotemporal dimensions; The multimodal aligned building-related data is normalized to obtain normalized building-related data. The normalized building-related data is subjected to noise filtering to obtain building-related data with outliers removed. High-dimensional feature vectors are constructed from building-related data after removing outliers to obtain preprocessed data suitable for training large AI models.

[0028] Specifically, the preparation of training data for large AI models: The training data of this invention constructs virtual reality scenes for building digital twins, covering the collection, fusion and preprocessing of multi-source heterogeneous data; Data sources include: geometric structure data, which are obtained through laser scanning and 3D modeling tools (such as BIM software) to obtain point clouds, mesh models and CAD drawings of buildings, used to describe the spatial topology of buildings; Material and texture information is captured through high-resolution photography, satellite remote sensing, or video frames to capture details of building surfaces (such as concrete textures and glass reflectivity) and converted into digital parameters such as normal maps and diffuse maps. Environmental sensor data integrates real-time monitoring data such as temperature and humidity, light intensity, and vibration frequency inside and outside the building, as well as historical operation and maintenance records (such as equipment energy consumption and structural deformation monitoring). Dynamic scene data, including heat maps of pedestrian flow and time-series data of equipment operation status.

[0029] The acquired data is processed through multimodal alignment (such as spatiotemporal calibration and coordinate system unification), normalization (to eliminate dimensional differences), and noise filtering (such as sensor outlier removal based on wavelet transform) to construct a high-dimensional feature vector. For example, virtual reality data of a high-rise building can be integrated with its 3D point cloud model (geometric layer), exterior wall material parameters (attribute layer), air conditioning system temperature control timing (environment layer), and pedestrian flow distribution heat map (dynamic layer) to form a cross-dimensional joint feature representation, providing a complex and realistic input distribution basis for subsequent model training.

[0030] In this embodiment, a large AI model including a generator and a discriminator is constructed. Based on preprocessing, the large AI model is trained to obtain the trained large AI model, including: Construct the network structure of the generator; The weight matrix and bias terms of the generator are initialized to obtain the initialized generator; Construct the network structure for the discriminator; The weight matrix and bias terms of the discriminator are initialized to obtain the initialized discriminator; Based on the preprocessed data, the parameters of the initialized generator and discriminator are optimized to obtain the trained AI large model.

[0031] In this embodiment, based on the preprocessed data, the parameters of the initialized generator and discriminator are optimized, including: By statistically analyzing the mean and variance of the preprocessed data, the data distribution characteristics of the preprocessed data can be obtained. The noise level in the preprocessed data is detected to obtain the noise suppression factor; Based on the data distribution characteristics and noise suppression factor, the learning rates of the generator and discriminator are dynamically adjusted to obtain a dynamic learning rate that adapts to the data distribution and noise level. Based on the adjusted learning rate, update the weight matrices of the generator and discriminator to obtain the post-generator and discriminator with optimized parameters.

[0032] In this embodiment, training the large AI model based on preprocessing further includes: A topological constraint term is introduced into the damage function of the generator, which includes a geometric constraint term and a material environment property constraint term. By using geometric structure constraints, the similarity between the generated data output by the generator and the real building data in terms of geometric shape is measured, and the local domain topological relationship of the generated data is constrained to obtain geometrically matched generated data. By using material and environmental attribute constraints, the continuity of the generated data output by the generator with the real building data in terms of material texture and environmental attributes is measured, and the spatial smoothness of the generated data is constrained to obtain generated data with continuous material and environmental attributes. By combining generated data based on geometric shape matching and generated data with continuous material and environmental attributes, we obtain generated data with consistent topological structure and the corresponding intermediate state for training a large AI model.

[0033] In this embodiment, training the large AI model based on preprocessing further includes: Calculate the gradient of the generator loss function with respect to its weight matrix, correct the direction of the gradient and remove noise to obtain the cleaned generator loss gradient; Calculate the gradient of the discriminator loss function with respect to its weight matrix, filter out fluctuations in the magnitude of the gradient, and obtain the purified discriminator loss gradient. The parameters of the generator are updated based on the gradient loss of the purified generator, and the generator with updated parameters after gradient purification is obtained. The parameters of the discriminator are updated based on the gradient loss of the purified discriminator, resulting in a discriminator with updated parameters after gradient purification.

[0034] In this embodiment, training the large AI model based on preprocessed data further includes: The discriminator determines the authenticity of the generated data output by the generator, and based on the determination result and the feedback from the generator's loss function, high-quality generated data that meets the preset authenticity requirements are selected. An improved autoencoder model is constructed, which includes a multi-branch encoder and a multi-branch decoder for processing different types of data, resulting in an autoencoder model adapted for multimodal data augmentation. High-quality generated data is input into the autoencoder model, and through latent space interpolation and latent space perturbation, diverse augmented data is obtained; Augmented data is added to the preprocessed data for iterative training of the large AI model, resulting in an expanded preprocessed dataset containing augmented data and a large AI model after iterative training.

[0035] Specifically, large-scale AI model training: This invention constructs a virtual reality content generation model based on an AI big model, aiming to use deep learning technology to efficiently model and learn complex data in the digital twin environment of buildings; the AI ​​big model has powerful feature extraction and generalization capabilities, and can capture potential patterns from massive amounts of building virtual reality data to achieve high-quality content generation.

[0036] The AI ​​large model mainly consists of two core components: a generator and a discriminator. The former is used to generate realistic architectural virtual reality data, while the latter is used to evaluate the authenticity of the generated data, thereby continuously optimizing the model performance through adversarial training. To enhance the adaptability of large AI models, this invention combines multimodal data fusion technology to effectively integrate different types of data, such as geometric structure, material information, and environmental sensor data, enabling the model to more comprehensively understand and simulate the real characteristics of architectural scenes.

[0037] This invention uses generative adversarial networks as the large AI model, and the training process is as follows: The initial training process, targeting the virtual reality data characteristics of a digital twin environment, requires setting initial parameters for the generator and discriminator to enable the generative adversarial network (GAN) to initially adapt to high-dimensional and complex scene features. This includes designing the network structure and randomly initializing the weight matrix, so that it can approximate the distribution of the target data in the early stages of training. This is represented as follows: In the formula, For generator functions; The input data is specifically architectural virtual reality information; The weight matrix of the generator. For the generator's bias term, The activation function is Sigmoid. Preferably, The matrix parameters are randomly initialized to conform to a normal distribution.

[0038] The input data includes not only two-dimensional or three-dimensional geometric information, but also material, lighting, texture, environmental sensor data (such as temperature, humidity, vibration, etc.), and real-time monitoring data. For example, suppose a virtual reality platform for architecture needs to simulate a modern office building. The input data could include: geometric structure data, such as point cloud data and mesh data obtained through laser scanning or 3D modeling; texture and material information, such as texture mapping data converted from high-resolution photos or video frames, and the physical properties of materials (glass, concrete, etc.) for various parts of the building; and environmental data, such as real-time monitoring data of indoor and outdoor temperature, humidity, and lighting intensity. After preprocessing (such as normalization), this data forms a high-dimensional input vector, which is used for the initial training of the generator, enabling the generative adversarial network to capture the realistic features of the architectural scene from complex and heterogeneous data.

[0039] The generator's structure includes: an input layer that receives preprocessed (e.g., normalized) high-dimensional virtual reality data features; an intermediate hidden layer that uses multiple fully connected or convolutional layers to extract local and global features from the data layer by layer, stabilizing the training process with batch normalization and an activation function (Sigmoid activation function); and an output layer that outputs generated pseudo-data through a fully connected layer using a Sigmoid activation function, ensuring that the numerical range of the output data is consistent with the real data.

[0040] Furthermore, to enable adversarial training to possess basic discriminative capabilities, the discriminator needs to be initialized in the same way, as follows: In the formula, For the discriminator function, This is the weight matrix of the discriminator. This is the bias term for the discriminator. Preferably, The matrix parameters are randomly initialized to conform to a normal distribution. This is the LeakyReLU activation function.

[0041] The discriminator structure includes: an input layer that receives both real and generated data, ensuring consistency through a unified preprocessing procedure; an intermediate hidden layer that uses multiple fully connected or convolutional layers to extract deep features from the input data, employs Dropout to prevent overfitting, and uses the LeakyReLU activation function to preserve negative information, enhancing the model's sensitivity to subtle differences; and an output layer that uses a Sigmoid activation function to output a probability value reflecting the probability that the input sample belongs to the real data.

[0042] Traditional generative adversarial networks (GANs) typically use a fixed learning rate (e.g., 0.0001). In existing technologies, some improved GANs (such as adaptive GANs) adjust the learning rate, but without considering the statistical characteristics and noise effects of the data. During training, the learning rate adjustment mechanism usually relies on a fixed hyperparameter optimization strategy. Since the distribution of architectural virtual reality data can be extremely uneven, with some areas having dense data and others sparse data, a fixed learning rate may lead to excessively high learning rates in high-density areas, easily causing mode collapse, or excessively low learning rates in low-density areas, resulting in slow convergence and excessively long training cycles. Traditional methods cannot dynamically monitor the noise level of the data; when the data has significant noise, a fixed learning rate can lead to training instability. To address the potential imbalances, high-dimensionality, and noise effects in the distribution of architectural digital twin virtual reality data, this invention employs a parameter-sensitive adaptive adjustment method. By dynamically monitoring the statistical properties and noise levels of the data distribution, the learning rate and gradient update magnitude of the generator and discriminator are changed in real time, thereby ensuring a balance between convergence speed and stability in the early stages of training. This is expressed as: In the formula, To generate the learning rate for the adversarial network, For input data The characteristic mean, For input data The characteristic variance, Describing the L2 norm, Let the gradient of the generator's loss function be the weight. To prevent division by zero, the first smallest constant, To prevent division by zero by the second smallest constant; This is a noise suppression factor used to automatically reduce the learning rate when noise levels are high. Preferably, Set to 0.0001. Set to 0.0005.

[0043] The noise suppression factor is used to automatically reduce the learning rate when the data noise is high, preventing noise from interfering with parameter updates. It works by statistically analyzing the noise level in the input data, such as calculating the signal-to-noise ratio or local fluctuation index. When the noise component exceeds a preset threshold, the noise level is considered high. Furthermore, the value of the noise suppression factor is dynamically adjusted based on the detection results. For example, when the noise is higher than the preset threshold, the noise suppression factor is increased to reduce the learning rate, thereby reducing the gradient update step size; conversely, a higher learning rate is maintained.

[0044] Furthermore, the weights of the generator and discriminator are dynamically updated based on the learning rate, as follows: In the formula, For the first The weight matrix of the generator in the next iteration For the first The discriminant weight matrix for the next iteration; A positive integer representing the number of iterations; For the first The weight matrix of the generator in the next iteration For the first The discriminant weight matrix of the next iteration. The generator loss gradient after purification. Let the gradient of the generator's loss function be the weight. Let the gradient of the discriminator's loss function with respect to the weights be denoted as . The gradient of the purified discriminator loss. Let the loss function of the generator be... Let be the loss function of the discriminator.

[0045] To verify the effectiveness of the parameter sensitivity adaptive adjustment method of the present invention, the following steps were performed: Figure 2 The experiment shown.

[0046] By analyzing the improvement of training stability by the dynamic learning rate adjustment strategy, the advantages of the dynamic learning rate adjustment strategy over the fixed learning rate in training stability are verified. Experimental results show that the loss curve of the present invention converges faster (200 rounds vs. 300 rounds in the traditional method) and the fluctuation amplitude is reduced by about 40%, indicating that the dynamic learning rate effectively balances convergence speed and stability through noise suppression and gradient purification.

[0047] Adversarial training between the generator and discriminator is used to improve the realism of generated samples by allowing the generator and discriminator to alternately compete in order to fully learn the complex feature distribution of architectural virtual reality data. The generator aims to generate data that can confuse the discriminator, while the discriminator tries to distinguish between real and fake data. Through multiple rounds of adversarial training, the model gradually approximates the real distribution and can handle high-dimensional and diverse inputs. The loss function that constrains the generator during adversarial training is expressed as: In the formula, This represents the distribution of virtual reality data obtained from real-world architectural scenes. This represents a noise vector from the latent space. This represents the pseudo-data output by the generator under noisy input; It is a logarithmic function with a default base of 10; For the expected operation, This refers to the topological constraints for generating the data. Preferably, Sampling is performed using a Gaussian distribution to cover more potential space.

[0048] Furthermore, the discriminator's loss is the opposite of the generator's. By continuously improving its ability to distinguish between real and fake data, it forces the generator to output more realistic samples. This process is iterated until the generator can generate high-fidelity digital twin data of buildings. During adversarial training, a gradient penalty term is used to enhance training stability and avoid mode collapse. The loss function that constrains the discriminator is expressed as: In the formula, The gradient penalty coefficient controls the strength of the smoothing constraint. This is a linear interpolation between real and generated data. Data mixing enhances the discriminator's ability to distinguish intermediate states. The calculation method is as follows: Preferably, Set to 10.

[0049] In architectural digital twins, virtual reality data needs to satisfy topological consistency in attributes such as geometry, materials, and environmental sensors. Traditional generative adversarial networks (GANs) mainly focus on element-level similarity of data, rather than the consistency of global geometric structure. Especially in 3D data generation tasks (such as 3D GANs), some existing technologies use point cloud loss to measure point cloud similarity. However, these methods do not take into account the specific attributes of architectural virtual reality data (such as the spatial consistency of material and environmental data), and cannot maintain local topological relationships well, leading to possible local distortions or discontinuities in the generated 3D structure. This method uses a topological structure constraint term of the generated data to fuse geometric structure constraints and material and environmental attribute constraints, i.e. For the generated point cloud or mesh data, geometric structure constraints are used to measure the geometric similarity between the generated data and the real data, thereby constraining the local neighborhood topological relationships of the generated data, as expressed as: In the formula, Geometric constraints; These are geometric constraint weighting coefficients, controlling the strength of geometric consistency. Given the L2 norm, calculate the Euclidean distance between pairs of points; and Representing the point coordinates in the generated data and the real data respectively, defined As the first coordinate, This is the second coordinate. Preferably, Set it to 0.2.

[0050] Furthermore, to maintain the continuity of generated data in material texture and the spatial smoothness of environmental data, the calculation method for material and environmental attribute constraints is expressed as follows: In the formula, Environmental attribute constraints; These are the attribute constraint weight coefficients; For the horizontal gradient operator, For the vertical gradient operator; Indicates the location of the generated data. Material or environmental property values ​​(such as temperature, texture RGB values); It is a positive integer. It is a positive integer; This represents the L1 norm.

[0051] When architectural virtual reality data contains noise (such as sensor errors), the discriminator may mislead the generator, causing the model to learn an incorrect distribution. Traditional methods cannot remove noise information from the gradient, resulting in significant fluctuations and poor convergence during training. Conventional generative adversarial networks use standard gradient updates during training, but in high-dimensional and complex data environments (such as architectural virtual reality data), the gradient direction may be affected by noise, leading to erroneous updates. To enhance the quality assurance of pseudo-data in high-dimensional and noisy architectural virtual scenes, this invention adaptively cleans the gradient directions of the generator and discriminator during training. Adaptive cleansing refers to correcting the gradient direction and magnitude during gradient updates, removing the adverse effects introduced by noise and interference, ensuring that the update direction is closer to the gradient direction of the real data distribution. This ensures that the generator can more stably capture the key features of the real data in the face of the variable and noisy environment of virtual reality data. Removing noisy gradients can reduce erroneous updates, improve the detail reproduction and overall consistency of the generated data. The calculation method of the generator loss gradient after cleansing is expressed as follows: In the formula, The generator loss gradient after purification. As a gradient conflict purifier, It is an exponential function. Preferably, Set it to 0.1.

[0052] Furthermore, in virtual reality data, since the input samples may contain a large amount of noise and local anomalies, directly using the original gradient can easily lead to the discriminator overfitting to local noise. To ensure the stability of the discriminator when dealing with noise interference, this invention uses the same approach to clean the gradient of the discriminator. The discriminator's cleanup method uses a sign function to retain only the direction information of the gradient, thereby filtering out unstable amplitudes introduced by noise. This allows the discriminator to focus primarily on direction consistency rather than amplitude fluctuations during updates, enabling the discriminator to learn global features more stably, improving its ability to distinguish between real and pseudo data, and reducing training fluctuations caused by noise. The calculation method is expressed as follows: In the formula, The gradient of the purified discriminator loss. This represents the gradient purification coefficient. This represents the sign function, which removes the useless parts of the gradient. Preferably, Set it to 0.3.

[0053] To ensure that the virtual reality data generated during adversarial training accurately reflects the needs of the architectural scene, a data filtering mechanism is employed to discriminate and screen the data generated in each training round. Based on the discriminator's judgment results and the loss function feedback, high-quality samples are retained while a large amount of noisy or significantly flawed data is removed. This strengthens the learning of the true distribution in subsequent iterations, as expressed below: In the formula, To obtain a high-quality generated dataset through screening, To determine the threshold.

[0054] The decision threshold is dynamically adjusted based on the discriminator's ability to distinguish between real and generated architectural scene data. In the early stages of training, because the data generated by the generator is of low quality, the discriminator can easily distinguish between real and fake data. At this time, the decision threshold can be set to a relatively high value (such as 0.8), meaning that only generated data that the discriminator considers to have high realism will be retained. As training progresses, the generator's ability gradually increases, and the quality of generated data improves. At this time, in order to further filter out higher quality data, the decision threshold will be gradually reduced (such as reduced to 0.6 or 0.5), ensuring that high-quality samples that match the current model's ability level can be filtered out at different stages of training. This strengthens the learning of the real distribution in subsequent iterations, ultimately enabling the generator to stably output high-quality virtual reality data that matches the real architectural scene.

[0055] This step involves filtering the obtained high-quality generated dataset. The data augmentation of the samples was carried out by using an improved autoencoder model. This model uses multimodal latent space learning and perturbation mechanisms to diversify the expansion of architectural virtual reality data while maintaining topological consistency of geometric structure, material properties and environmental sensor data.

[0056] The improved autoencoder can not only reconstruct the input data, but also perform interpolation and noise injection in the latent space to generate new samples with high fidelity and diversity, thereby improving the robustness and generalization ability of subsequent model training.

[0057] Specifically, the improved autoencoder model consists of a multi-branch encoder and a decoder, which process geometric structure, material texture, and environmental sensor data, respectively. Let the input data... From the filter set ,in Includes geometric features (such as point cloud or mesh data), material features (such as texture mapping parameters) and environmental features (e.g., time-series data on temperature and humidity); encoder Mapping multimodal data to latent vectors decoder from Reconstructing data The model structure is as follows: encoder A multi-branch fully connected network is used, with each branch corresponding to a data type. The outputs are concatenated into a unified latent vector, represented as follows: In the formula, , and Sub-encoders for geometry, material, and environment, respectively. This represents a vector concatenation operation; The dimension is (Preferred, The sub-encoder structure includes an input layer, multiple hidden layers (using batch normalization and LeakyReLU activation functions), and an output layer, which extract high-dimensional features through nonlinear transformations.

[0058] decoder It also adopts a multi-branch structure, starting from the latent vector The reconstructed modal data are represented as follows: In the formula, , and This is the corresponding sub-decoder, whose output dimension is the same as the input data.

[0059] The data augmentation module is trained independently. To optimize the autoencoder training, a multimodal reconstruction loss is defined. The difference between the reconstructed data and the original data is measured and expressed as: In the formula, , and Weights are assigned to the loss of each modality to balance the contributions of different data types (preferably). , , ); Describing the L2 norm, This represents the L1 norm.

[0060] Optionally, to further enhance data diversity, this invention employs interpolation and perturbation mechanisms in the latent space, specifically: Latent space interpolation, randomly selecting two latent vectors and (From different samples), new vectors are generated through linear interpolation. , is represented as: In the formula, The interpolation coefficients follow a uniform distribution. The decoder will Mapped to augmented samples This sample transitions smoothly from the original sample in terms of geometry and attributes.

[0061] Latent spatial perturbation, for latent vector Add adaptive noise to generate a perturbation vector , is represented as: In the formula, The disturbance intensity coefficient (preferably, ), The noise vector follows a standard normal distribution. , This indicates element-wise multiplication. Decoder output. Slight variations are introduced to simulate fluctuations in real data.

[0062] To maintain the topological consistency of the augmented data, a constraint mechanism is adopted, defining the topological constraint loss. Including geometric constraints Material and environmental property constraints , is represented as: In the formula, Ensure that the generated geometric data has the same local neighborhood structure as the original data. and These represent the coordinates of points in the generated and real geometric data, respectively. Force spatial smoothness of material textures and environmental data. and For gradient operators in the horizontal and vertical directions, Indicates position The attribute value at the location (such as texture RGB or temperature). and Indices are positive integers; and For constraint weights (preferably, , ).

[0063] Furthermore, the total loss function of the autoencoder Combining reconstruction loss and topological constraint loss, it can be expressed as: Furthermore, minimize through backpropagation Train the autoencoder parameters.

[0064] After training, the selection set is... Interpolation and perturbation are applied to each sample to generate an augmented dataset. This collection contains the original samples and their enhanced versions for iterative training of large AI models.

[0065] Repeat the above steps iteratively until a preset stopping iteration condition is met, indicating that the model training is complete. The generator can output highly reliable and diverse virtual reality data when inputting latent vectors. This data matches real building scenes and has significant advantages in noise suppression and feature preservation. It can be applied to fields such as visualization simulation, operation and maintenance decision-making, and risk analysis in building digital twins. In one embodiment, the preset stopping iteration condition is reaching a preset maximum number of iterations. Preferably, the preset maximum number of iterations is set to 1000.

[0066] Furthermore, to verify the ability of the proposed geometry-attribute joint constraint mechanism to recreate building digital twin scenes, the following steps were performed: Figures 3-5 The experiment shown.

[0067] This experiment compares the results generated by traditional generative adversarial networks (GANs) with those generated by our proposed technique. Traditional methods, lacking topological constraints, result in point clouds with significant structural distortions and missing local details. In contrast, the architectural structures generated by our technique not only closely match the real-world geometry but also accurately preserve spatial topological relationships. The thermal mapping of the error distribution in the generated results shows that our technique significantly reduces geometric errors, demonstrating that the joint constraint mechanism can effectively capture multi-dimensional features. In this embodiment, by inputting latent vectors into the trained AI model, the trained AI model outputs virtual reality content that matches the building digital twin scene, including: Monitor the training process of the large AI model. When the training meets the preset stopping iteration conditions, determine that the training of the large AI model is complete and obtain the trained large AI model. Input vectors from the latent space into the generator of the trained large AI model; The generator processes the input latent space vector to obtain virtual reality content that includes architectural geometry, material textures, and environmental attributes. The output virtual reality content is applied to the visualization simulation, operation and maintenance decision support, or risk analysis scenarios of building digital twins to obtain the application results for the corresponding scenarios.

[0068] Specifically, large AI model generation: The trained generator can generate high-fidelity virtual reality content based on the input latent vectors; Specifically, the AI ​​big model utilizes the complex feature distribution of architectural virtual reality data learned during training to generate virtual reality data that highly matches the real architectural scene by inputting latent vectors. This generated data not only matches the real scene in geometric structure, but also maintains a high degree of realism in terms of material texture, lighting effects and environmental attributes. The generated virtual reality content can be used in various application scenarios of building digital twins, such as visualization simulation, operation and maintenance decision support, and risk analysis. Through the ability to generate large AI models, users can quickly obtain highly reliable and diverse virtual reality data, thereby supporting the construction and application of building digital twin systems.

[0069] In one embodiment, the impact of topological constraints on the quality of generated data is analyzed, with a focus on evaluating the improvement effect of topological constraints on the quality of generated data. Experiments compare three methods: a conventional generative adversarial network (GAN) method without topological constraints, a conventional GAN ​​method using only point cloud loss, and the complete topological constraint method of this invention. The comparisons are quantified from three dimensions: geometric consistency, material continuity, and environmental data consistency. Geometric consistency error is measured using Chamfer distance, with smaller values ​​indicating better geometric structure preservation. Material continuity is scored, with higher values ​​indicating better material texture continuity. Environmental data consistency is represented by correlation coefficients, with dimensionless coefficients; values ​​closer to one indicate higher consistency of environmental sensor data. In the experimental configuration, the conventional method without topological constraints only focuses on element-level similarity, the conventional point cloud loss method incorporates basic geometric constraints, while the method of this invention integrates geometric constraints and material environmental attribute constraints. The three sub-figures in the figure illustrate the performance of different methods on the three metrics, clearly showing that the method of this invention outperforms the conventional method in all metrics. Regarding geometric consistency error, the method of this invention has the lowest bar height, indicating that its generated geometric data is closest to the real data. Regarding material continuity scoring, the method of this invention has the highest bar height, indicating better performance in preserving material texture. Regarding environmental data consistency, the bar chart of this invention is closest to the perfect value, demonstrating its accurate modeling capability for environmental sensor data. The experimental results fully demonstrate that this invention effectively maintains the local topological relationships and spatial consistency of architectural virtual reality data through topological constraints.

[0070] In other embodiments, a virtual reality content generation model training apparatus based on an AI large model is provided, and a virtual reality content generation model training method based on an AI large model according to any of the preceding embodiments is provided, including: The acquisition module is configured to acquire virtual reality training data for building digital twin scenarios; The preprocessing module is configured to preprocess virtual reality training data to obtain preprocessed data adapted for training large AI models. The building module is configured to build large AI models that include generators and discriminators; The training module is configured to train a large AI model based on preprocessing to obtain a trained large AI model. The output module is configured to enable the trained AI model to output virtual reality content that matches the building's digital twin scene by inputting latent vectors into the trained AI model.

[0071] In other embodiments, a terminal includes a processor, an input device, an output device, and a memory, which are interconnected. The memory stores a computer program, which includes program instructions. The processor is configured to invoke the program instructions to execute any of the preceding AI-based large-scale virtual reality content generation model training methods.

[0072] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be covered within the scope of the claims and specification of the present invention.

Claims

1. A training method for a virtual reality content generation model based on a large AI model, characterized in that, include: Acquire virtual reality training data for building digital twin scenarios; The virtual reality training data is preprocessed to obtain preprocessed data adapted for training large AI models; Build a large AI model that includes a generator and a discriminator; The AI ​​large model is trained based on the preprocessing to obtain the trained AI large model; By inputting latent vectors into the trained AI model, the trained AI model outputs virtual reality content that matches the building's digital twin scene.

2. The virtual reality content generation model training method based on AI large model according to claim 1, characterized in that, The virtual reality training data is preprocessed to obtain preprocessed data adapted for training large AI models, including: Acquire multi-source heterogeneous building-related data; Multimodal alignment processing is performed on multi-source heterogeneous building-related data to obtain building-related data with consistent spatiotemporal dimensions; The multimodal aligned building-related data is normalized to obtain normalized building-related data. The normalized building-related data is subjected to noise filtering to obtain building-related data with outliers removed. High-dimensional feature vectors are constructed from building-related data after removing outliers to obtain preprocessed data suitable for training large AI models.

3. The virtual reality content generation model training method based on a large AI model according to claim 1, characterized in that, Construct a large AI model containing a generator and a discriminator, train the large AI model based on the preprocessing, and obtain the trained large AI model, including: Construct the network structure of the generator; The weight matrix and bias terms of the generator are initialized to obtain the initialized generator; Construct the network structure of the discriminator; The weight matrix and bias terms of the discriminator are initialized to obtain the initialized discriminator; Based on the preprocessed data, the parameters of the initialized generator and discriminator are optimized to obtain the trained AI large model.

4. The virtual reality content generation model training method based on a large AI model according to claim 3, characterized in that, Based on the preprocessed data, the parameters of the initialized generator and discriminator are optimized, including: By statistically analyzing the mean and variance of the preprocessed data, the data distribution characteristics of the preprocessed data can be obtained. The noise level in the preprocessed data is detected to obtain the noise suppression factor; Based on the data distribution characteristics and noise suppression factor, the learning rates of the generator and the discriminator are dynamically adjusted to obtain a dynamic learning rate that adapts to the data distribution and noise level. Based on the adjusted learning rate, the weight matrices of the generator and the discriminator are updated to obtain the post-generator and discriminator with optimized parameters.

5. The virtual reality content generation model training method based on a large AI model according to claim 3, characterized in that, Training the large AI model based on the preprocessing also includes: A topological constraint term is introduced into the damage function of the generator, wherein the topological constraint term includes a geometric constraint term and a material environment property constraint term; The geometric structure constraints are used to measure the similarity in geometric shape between the generated data output by the generator and the real building data, constrain the local domain topological relationship of the generated data, and obtain geometrically matched generated data. By using the material and environmental attribute constraints, the continuity of the generated data output by the generator with the real building data in terms of material texture and environmental attributes is measured, the spatial smoothness of the generated data is constrained, and generated data with continuous material and environmental attributes is obtained. By combining the generated data of geometric shape matching and the generated data of continuous material and environmental attributes, generated data with consistent topological structure and the corresponding intermediate state of AI large model training are obtained.

6. The virtual reality content generation model training method based on a large AI model according to claim 3, characterized in that, Training the large AI model based on the preprocessing also includes: Calculate the gradient of the generator loss function with respect to its weight matrix, correct the direction of the gradient and remove noise to obtain the cleaned generator loss gradient; Calculate the gradient of the discriminator loss function with respect to its weight matrix, filter out the amplitude fluctuations of the gradient, and obtain the purified discriminator loss gradient. The parameters of the generator are updated based on the gradient loss of the purified generator to obtain the generator with updated parameters after gradient purification. The parameters of the discriminator are updated based on the gradient loss of the purified discriminator to obtain the discriminator with updated parameters after gradient purification.

7. The virtual reality content generation model training method based on a large AI model according to claim 1, characterized in that, Training the AI ​​large model based on the preprocessed data also includes: The discriminator determines the authenticity of the generated data output by the generator, and based on the determination result and the feedback of the generator's loss function, high-quality generated data that meets the preset authenticity requirements are selected. An improved autoencoder model is constructed, which includes a multi-branch encoder and a multi-branch decoder for processing different types of data, to obtain an autoencoder model adapted for multimodal data augmentation. The high-quality generated data is input into the autoencoder model, and through latent space interpolation and latent space perturbation, diverse augmented data is obtained; The augmented data is added to the preprocessed data for iterative training of the AI ​​large model, resulting in an expanded preprocessed dataset containing the augmented data and an iteratively trained AI large model.

8. The virtual reality content generation model training method based on a large AI model according to claim 1, characterized in that, By inputting latent vectors into the trained AI model, the trained AI model outputs virtual reality content that matches the building's digital twin scene, including: The training process of the AI ​​large model is monitored. When the training meets the preset stopping iteration condition, the training of the AI ​​large model is determined to be completed, and the trained AI large model is obtained. Input vectors from the latent space into the generator of the trained large AI model; The generator processes the input latent space vector to obtain virtual reality content that includes architectural geometry, material textures, and environmental attributes. The output virtual reality content is applied to the visualization simulation, operation and maintenance decision support, or risk analysis scenarios of building digital twins to obtain the application results for the corresponding scenarios.

9. A training device for a virtual reality content generation model based on a large AI model, characterized in that, The virtual reality content generation model training method based on any one of claims 1 to 8 includes: The acquisition module is configured to acquire virtual reality training data for building digital twin scenarios; The preprocessing module is configured to preprocess the virtual reality training data to obtain preprocessed data adapted for training large AI models. The building module is configured to build large AI models that include generators and discriminators; The training module is configured to train the AI ​​large model based on the preprocessing to obtain the trained AI large model; The output module is configured to input latent vectors into the trained AI model, causing the trained AI model to output virtual reality content that matches the building digital twin scene.

10. A terminal, characterized in that, The system includes a processor, an input device, an output device, and a memory, which are interconnected. The memory is used to store a computer program, which includes program instructions. The processor is configured to invoke the program instructions to execute the virtual reality content generation model training method based on an AI large model as described in any one of claims 1 to 8.

Images