VR content generation method based on intelligent modeling model

Abstract

The invention discloses a VR content generation method based on an intelligent modeling model. Establishing a specific intelligent modeling model comprising an intermediate conversion module, and inputting virtual reality VR scene content data into an intelligent modeling network for training; and obtaining image data under a to-be-generated VR scene, inputting the image data into the trained model, outputting environment data of the VR scene after processing, performing three-dimensional modeling to obtain a three-dimensional model of the VR scene, and finally realizing VR scene content generation. According to the method disclosed by the invention, the steps of spending a lot of time for conception and design in a traditional modeling method are reduced, and the modeling efficiency is improved; according to the method, a personalized modeling thought is provided, and the personalized design capability of modeling is improved and the modeling content is enriched by combining the customized modeling parameters of the intelligent modeling model.

Description

Technical Field

[0001] The present invention relates to a method for generating VR content, and relates to the technical fields of artificial intelligence (AI) and virtual reality (VR), and in particular to a method for generating VR content based on an intelligent modeling model. Background Art

[0002] Since its birth, virtual reality (VR) technology has shown great application potential in entertainment, education, medical care, industrial design and other fields with its immersive experience. With the rapid development of technologies such as 5G, cloud computing, and edge computing, the performance of VR devices has been continuously improved, and the user experience has also been significantly improved. However, despite the significant progress made in content production and display, VR technology still faces many challenges, especially in content creation efficiency, quality and personalization.

[0003] The production of traditional VR content often requires a lot of manpower, material resources and time. From scene modeling, character design to animation production, each link requires professional technicians to perform delicate operations. At the same time, the production quality of VR content is affected by multiple factors such as the producer's technical level, creative ability and aesthetic concepts. The quality of VR content on the market varies. Although some content has stunning visual effects, it lacks depth and connotation; while some content is too simple and rough to provide high-quality immersive experience. Moreover, due to the long production cycle and high cost of VR content, it is difficult to achieve large-scale personalized customization, which to a certain extent limits the widespread application and in-depth development of VR technology. The existing technology has problems such as low production efficiency, uneven content quality and insufficient personalization. Summary of the invention

[0004] In order to solve the problems existing in the background technology, the present invention provides a VR content generation method based on an intelligent modeling model. The method of the present invention uses an intelligent modeling model combined with data analysis to learn a large amount of VR content, generate high-quality and diverse VR scenes and characters, and can be applied in virtual reality.

[0005] The technical solution adopted by the present invention is:

[0006] The VR content generation method based on the intelligent modeling model of the present invention comprises:

[0007] S1: Establish a specific intelligent modeling model including an intermediate conversion module.

[0008] S2: Inputting a number of virtual reality (VR) scene content data at the current moment and the previous moment into the intelligent modeling network for training to obtain a trained intelligent modeling model.

[0009] S3: Acquire a number of image data of the virtual reality VR scene to be generated and input them into the trained intelligent modeling model for processing. After processing, the trained intelligent modeling model outputs the environmental modeling parameters of the virtual reality VR scene to be generated, and obtains the three-dimensional model of the virtual reality VR scene after three-dimensional modeling, thereby finally realizing the generation of virtual reality VR scene content.

[0010] When performing 3D modeling, first enter the required parameters in the modeling software interface based on the data generated by the model and the virtual environment representation. These parameters define the basic framework and characteristics of the scene. Subsequently, the internal advanced algorithm calculates and optimizes based on these parameters, and gradually builds a 3D model of the scene. During this process, the software can intelligently handle the occlusion relationship between objects, the reflection and refraction effects of light, and the surface texture of the material, ensuring that the final model is both in line with physical laws and has a high degree of visual realism.

[0011] For complex scenes, such as VR environments with a lot of details and dynamic elements, the modeling software also supports modular design and parametric adjustment. Parameters can be adjusted at any time during the modeling process, such as changing the layout of the scene, adjusting the size and position of objects, changing material textures, and adjusting lighting effects, without having to rebuild from scratch, which greatly improves work efficiency and flexibility. It has advanced customization functions such as intelligent color matching, style migration, and dynamic element addition. Through these functions, personal style or specific themes can be integrated into VR scenes to create a unique personalized space. This method not only simplifies the cumbersome process of traditional modeling, but also makes it possible to create high-quality, personalized VR scenes.

[0012] In the step S1, the intelligent modeling model includes a generation network, an intermediate conversion module and a modeling network connected in sequence. The input of the intelligent modeling model is processed by the generation network to generate two-dimensional content, which is then input into the intermediate conversion module to convert the two-dimensional content into pre-processed point cloud content, and finally input into the modeling network for processing and output of environmental modeling parameters.

[0013] The current modeling algorithms basically use point cloud data or neural network fields to obtain scene information and complete the modeling process according to the information content. The main purpose is to increase the degree of scene restoration. The present invention proposes a new modeling idea. The main purpose is not to restore the scene, but to create it according to the actual scene and generate a related but personalized modeling algorithm.

[0014] The intermediate conversion module includes a convolutional neural network, a residual network ResNet (ResidualNetworks), a coordinate conversion operation and a preprocessing operation connected in sequence. The intermediate conversion module first uses a convolutional neural network to extract features from the input, then uses the residual network ResNet to process and output a depth map, and then performs a coordinate conversion operation to convert the pixel coordinates in the depth map into point cloud coordinates in three-dimensional space, thereby obtaining point cloud data. Finally, the point cloud data is subjected to denoising, smoothing, hole filling, point cloud simplification and other operations in sequence to complete the preprocessing operation.

[0015] The convolutional neural network consists of several layers of convolutional layers connected in sequence, which gradually extract low-level to high-level features of the image, such as edges, textures, and shapes. The residual network ResNet is used for pixel-level depth prediction. The mapping relationship between the learned image features and depth is used to assign a depth value to each pixel, thereby constructing the depth map of the image. The coordinate transformation operation combines the camera's intrinsic parameter matrix (focal length, optical center, etc.) and possible extrinsic parameter matrix (camera position, orientation, etc.) for transformation.

[0016] The generative network is specifically a generative adversarial network GAN (Generative Adversarial Networks) or a variational autoencoder VAE (Variational Auto-Encoder).

[0017] The generative network first uses generative algorithms such as generative adversarial networks (GANs) or variational autoencoders (VAEs) to generate a series of realistic two-dimensional contents based on the input conditions. These two-dimensional contents have rich scene or object information.

[0018] The modeling network is specifically a deep learning network SSRNet (soft Stagewise Regression Network).

[0019] In step S2, the virtual reality VR scene content data includes text description, sketch and scene point cloud data of the virtual reality VR scene.

[0020] In the step S3, the image data of the virtual reality VR scene to be generated includes images of each subject in the virtual reality VR scene at different angles, lighting conditions and detail levels.

[0021] Collect images of specific subjects (such as people, objects, etc.) related to VR scenes. Usually, a large number of high-quality images related to VR modeling scenes are required for fine-tuning the model. After fine-tuning, when using the model to generate training scenes and obtain parameters, a series of high-quality pictures or videos shot around the target VR space need to be prepared. These materials cover different angles, lighting conditions and levels of detail to ensure that the model can fully learn and understand the uniqueness of the scene.

[0022] During the generative training process, the model not only analyzes and learns the spatial layout and texture details of the input data, but also captures dynamic elements such as light and shadow changes, thereby building a detailed and controllable virtual environment parameter representation within the model. This representation can then be used to generate new VR scenes that retain the key parts of the original material while being able to be flexibly adjusted and optimized according to the needs of the creator, such as changing the perspective, adjusting the lighting, adding or removing specific elements, etc.

[0023] The method of the present invention learns a large amount of VR content through an intelligent modeling model, and performs three-dimensional modeling to automatically analyze key elements such as scene layout, character design, and animation effects, generating high-quality and diverse VR scenes and characters. At the same time, combined with user behavior data and preference analysis, personalized recommendations and intelligent adjustments of VR content are achieved, thereby improving user experience and market competitiveness.

[0024] The beneficial effects of the present invention are:

[0025] The method of the present invention reduces the steps of conceiving and designing which require a lot of time in traditional modeling methods, saves manpower and material resources, and improves modeling efficiency.

[0026] The present invention proposes an idea of ​​a personalized modeling algorithm, which improves the personalized design capability of modeling and enriches the modeling content by combining customized modeling parameters of the modeling algorithm and adding intelligent color and style shift. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is the overall flow chart of the present invention;

[0028] Figure 2 It is a schematic diagram of the intelligent modeling network of the present invention. DETAILED DESCRIPTION

[0029] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0030] like Figure 1 As shown, the VR content generation method based on the intelligent modeling model of the present invention is specifically as follows:

[0031] S1: Establish a specific intelligent modeling model including an intermediate conversion module. The intelligent modeling model includes a generation network, an intermediate conversion module and a modeling network connected in sequence. The input of the intelligent modeling model is processed by the generation network to generate two-dimensional content, which is then input into the intermediate conversion module to convert the two-dimensional content into pre-processed point cloud content, and finally input into the modeling network for processing to output the environment modeling parameters.

[0032] The current modeling algorithms basically use point cloud data or neural network fields to obtain scene information and complete the modeling process according to the information content. The main purpose is to increase the degree of scene restoration. The present invention proposes a new modeling idea. The main purpose is not to restore the scene, but to create it according to the actual scene and generate a related but personalized modeling algorithm.

[0033] like Figure 2 As shown, the intermediate conversion module includes a convolutional neural network, a residual network ResNet, a coordinate conversion operation and a preprocessing operation connected in sequence. The intermediate conversion module first uses a convolutional neural network to extract features from the input, then uses the residual network ResNet to process and output a depth map, and then performs a coordinate conversion operation to convert the pixel coordinates in the depth map into point cloud coordinates in three-dimensional space, thereby obtaining point cloud data. Finally, the point cloud data is subjected to denoising, smoothing, hole filling and point cloud simplification operations in sequence to complete the preprocessing operation.

[0034] The convolutional neural network consists of several layers of convolutional layers connected in sequence, which gradually extract low-level to high-level features of the image, such as edges, textures, and shapes. The residual network ResNet is used for pixel-level depth prediction. The mapping relationship between the learned image features and depth is used to assign a depth value to each pixel, thereby constructing the depth map of the image. The coordinate transformation operation combines the camera's intrinsic parameter matrix (focal length, optical center, etc.) and possible extrinsic parameter matrix (camera position, orientation, etc.) for transformation.

[0035] The generative network is specifically a generative adversarial network GAN or a variational autoencoder VAE. The generative network first uses a generative algorithm such as a generative adversarial network GAN or a variational autoencoder VAE to generate a series of realistic two-dimensional contents according to the input conditions. These two-dimensional contents have rich scene or object information.

[0036] The modeling network is specifically the deep learning network SSRNet.

[0037] S2: Inputting a number of virtual reality VR scene content data at the current moment and the previous moment into the intelligent modeling network for training to obtain a trained intelligent modeling model.

[0038] Virtual reality VR scene content data includes text descriptions, sketches, and scene point cloud data of the virtual reality VR scene.

[0039] S3: Obtain a number of image data of the virtual reality VR scene to be generated and input them into the trained intelligent modeling model for processing. After processing, the trained intelligent modeling model outputs the environmental modeling parameters of the virtual reality VR scene to be generated, and obtains the three-dimensional model of the virtual reality VR scene after three-dimensional modeling, and finally realizes the generation of virtual reality VR scene content. The image data of the virtual reality VR scene to be generated includes images of each subject in the virtual reality VR scene at different angles, lighting conditions and detail levels.

[0040] Collect images of specific subjects (such as people, objects, etc.) related to VR scenes. Usually, a large number of high-quality images related to VR modeling scenes are required for fine-tuning the model. After fine-tuning, when using the model to generate training scenes and obtain parameters, a series of high-quality pictures or videos shot around the target VR space need to be prepared. These materials cover different angles, lighting conditions and levels of detail to ensure that the model can fully learn and understand the uniqueness of the scene.

[0041] During the generative training process, the model not only analyzes and learns the spatial layout and texture details of the input data, but also captures dynamic elements such as light and shadow changes, thereby building a detailed and controllable virtual environment parameter representation within the model. This representation can then be used to generate new VR scenes that retain the key parts of the original material while being able to be flexibly adjusted and optimized according to the needs of the creator, such as changing the perspective, adjusting the lighting, adding or removing specific elements, etc.

[0042] When performing 3D modeling, first enter the required parameters in the modeling software interface based on the data generated by the model and the virtual environment representation. These parameters define the basic framework and characteristics of the scene. Subsequently, the internal advanced algorithm calculates and optimizes based on these parameters, and gradually builds a 3D model of the scene. During this process, the software can intelligently handle the occlusion relationship between objects, the reflection and refraction effects of light, and the surface texture of the material, ensuring that the final model is both in line with physical laws and has a high degree of visual realism.

[0043] For complex scenes, such as VR environments with a lot of details and dynamic elements, the modeling software also supports modular design and parametric adjustment. Parameters can be adjusted at any time during the modeling process, such as changing the layout of the scene, adjusting the size and position of objects, changing material textures, and adjusting lighting effects, without having to rebuild from scratch, which greatly improves work efficiency and flexibility. It has advanced customization functions such as intelligent color matching, style migration, and dynamic element addition. Through these functions, personal style or specific themes can be integrated into VR scenes to create a unique personalized space. This method not only simplifies the cumbersome process of traditional modeling, but also makes it possible to create high-quality, personalized VR scenes.

[0044] The method of the present invention can use VR shooting equipment to shoot the modeling scene to obtain high-quality panoramic pictures and video materials. The collected materials are screened and preprocessed to ensure that the image is clear, the angle is comprehensive, and every corner of the scene can be covered. Then the collected VR content is preprocessed, and the preprocessed panoramic pictures and videos are used as training data and input into the intelligent modeling algorithm for training. During the training process, focus on capturing the key detail elements of the scene (such as characteristic furniture, decorative paintings, lighting effects, etc.). By adjusting the training parameters and the number of iterations, the model performance is optimized to ensure that the generated modeling parameters can highly restore and surpass the personalized features of the original scene. After the training is completed, personalized modeling parameters are extracted from the intelligent modeling model, including but not limited to scene layout, object size, material properties, lighting effects, etc. These parameters are sorted and optimized to ensure that they can be seamlessly integrated into the VR modeling software. The personalized modeling parameters generated by intelligent modeling are typed into the VR modeling module. Using the modeling function of the software, a three-dimensional model of the modeling scene is constructed according to the parameters. During this process, the details of the model can be further adjusted as needed, such as adding personalized decorations, adjusting placement, style transfer, etc., so that the modeled scene can have the modeling effect of the original scene and can also add new parameters to personalize the scene.

Claims

1. A method for generating VR content based on an intelligent modeling model, characterized in that: include: S1: Establish a specific intelligent modeling model including intermediate conversion modules; S2: Inputting a number of virtual reality VR scene content data at the current moment and the previous moment into the intelligent modeling network for training, and obtaining a trained intelligent modeling model; S3: Acquire a number of image data of the virtual reality VR scene to be generated and input them into the trained intelligent modeling model for processing. After processing, the trained intelligent modeling model outputs the environmental modeling parameters of the virtual reality VR scene to be generated, and obtains the three-dimensional model of the virtual reality VR scene after three-dimensional modeling, thereby finally realizing the generation of virtual reality VR scene content.

2. The method for generating VR content based on the intelligent modeling model according to claim 1, characterized in that: In the step S1, the intelligent modeling model includes a generation network, an intermediate conversion module and a modeling network connected in sequence. The input of the intelligent modeling model is processed by the generation network to generate two-dimensional content, which is then input into the intermediate conversion module to convert the two-dimensional content into pre-processed point cloud content, and finally input into the modeling network for processing and output of environmental modeling parameters.

3. The method for generating VR content based on the intelligent modeling model according to claim 2, characterized in that: The intermediate conversion module includes a convolutional neural network, a residual network ResNet, a coordinate conversion operation and a preprocessing operation connected in sequence. The intermediate conversion module first uses a convolutional neural network to extract features from the input, then uses the residual network ResNet to process and output a depth map, and then performs a coordinate conversion operation to convert the pixel coordinates in the depth map into point cloud coordinates in three-dimensional space, thereby obtaining point cloud data. Finally, the point cloud data is subjected to denoising, smoothing, hole filling and point cloud simplification operations in sequence to complete the preprocessing operation.

4. The method for generating VR content based on the intelligent modeling model according to claim 2, characterized in that: The generative network is specifically a generative adversarial network GAN or a variational autoencoder VAE.

5. The method for generating VR content based on the intelligent modeling model according to claim 2, characterized in that: The modeling network is specifically a deep learning network SSRNet.

6. The method for generating VR content based on an intelligent modeling model according to claim 1, characterized in that: In step S2, the virtual reality VR scene content data includes a text description, a sketch and scene point cloud data of the virtual reality VR scene.

7. The method for generating VR content based on an intelligent modeling model according to claim 1, characterized in that: In the step S3, the image data of the virtual reality VR scene to be generated includes images of each subject in the virtual reality VR scene at different angles, lighting conditions and detail levels.

Images