A dynamic scene automatic generation method based on a generative adversarial network

Abstract

The application discloses a kind of dynamic scene automatic generation methods based on generative adversarial network, S1, input random noise vector, generate preliminary space-time graph structure, noise vector is mapped into high-dimensional space-time graph structure;S2, learn the space-time dependence between object and background, extract the space-time feature of space-time graph structure;S3, dynamically adjust the convolution kernel weight in graph convolution operation;S4, combine the space-time feature of space-time graph structure, optimize the graph convolution layer in generator, and further adjust the weight of convolution kernel;S5, evaluate the generated space-time graph structure, and optimize generation process through feedback signal;S6, further improve the quality of generated dynamic scene through adversarial training;S7, iteration S2 to S6 step, until generated dynamic scene meets requirements.The application can provide efficient, scientific optimization scheme in dynamic scene automatic generation, and bring significant technical value and economic benefits for practical application.

Description

Technical Field

[0001] This invention relates to the field of dynamic scene automatic generation technology, and in particular to a method for dynamic scene automatic generation based on generative adversarial networks. Background Technology

[0002] With the development of technologies such as virtual reality, augmented reality, and autonomous driving, dynamic scene generation is playing an increasingly important role in many fields. Dynamic scene generation technology is mainly used to create realistic virtual environments for simulation training, game development, intelligent transportation, film and television production, and other fields. Existing dynamic scene generation methods are mainly based on traditional deep learning models such as generative adversarial networks, convolutional neural networks, and recurrent neural networks. These methods perform well in generating static images and simple dynamic scenes, but they still have many limitations in generating complex spatiotemporal interactions and long-term sequences.

[0003] While existing dynamic scene generation methods based on generative adversarial networks (GANs) can generate high-quality images, they have shortcomings in handling spatiotemporal dependencies. Traditional GAN ​​architectures typically rely solely on spatial features for scene generation, with weak modeling of spatiotemporal features. This is especially true when capturing complex dynamic interactions between objects and time-varying motion patterns, where performance is limited. Furthermore, GANs are prone to pattern collapse during training, meaning that the generated scenes lack diversity and stability. Although traditional convolutional neural networks (CNNs) and recurrent neural networks (RNNs) are used to process time-series data in some applications, these methods are inadequate for handling complex spatiotemporal structures, especially when scene changes have multi-dimensional dependencies.

[0004] To overcome these shortcomings, researchers have proposed some methods that combine graph neural networks in recent years, attempting to use the advantages of graph structures to capture complex spatiotemporal dependencies. Graph neural networks have a natural advantage in processing the relationships between nodes and edges and can efficiently model spatial information. However, existing graph convolutional networks mainly focus on spatial features and neglect the fusion of temporal information, making it difficult to cope with the complex interactions of objects in the temporal dimension in dynamic scene generation.

[0005] Another major problem in existing technologies is the insufficient ability to fuse spatiotemporal features. When generating dynamic scenes, it is necessary not only to consider the spatial relationships between objects, but also to deeply understand the changes of objects over time, especially the interactions between objects. Traditional spatiotemporal modeling methods often use fixed convolutional kernels for feature extraction, failing to adaptively adjust the learning strategy according to the dynamic changes of the scene. This fixed convolutional kernel limits the model's ability to adapt to complex scene changes, resulting in a lack of flexibility and diversity in the generated results.

[0006] Another drawback of existing technologies is their inability to effectively handle spatiotemporal dependencies in long-term series. Although models such as RNN and LSTM perform well in modeling sequence data, they still suffer from gradient vanishing and gradient exploding problems when dealing with long-term series, making it difficult to capture long-term dependencies. In addition, these methods are also limited when dealing with graph-structured data because they lack an effective mechanism to handle complex spatiotemporal relationships between nodes.

[0007] Therefore, existing technologies still have many problems to be solved in dynamic scene generation, especially when dealing with complex spatiotemporal interactions, long time series, and multi-dimensional scene changes. Traditional deep learning methods are insufficient. To make up for these shortcomings, a new method is needed that can capture spatiotemporal dependencies and has flexible adaptive capabilities. By introducing spatiotemporal adaptive graph convolutional networks, the problems of insufficient spatiotemporal feature extraction and inaccurate spatiotemporal dependency modeling in existing technologies can be effectively solved. The spatiotemporal adaptive mechanism enables the network to dynamically adjust its learning strategy according to different time steps and spatial regions, thereby improving the quality and realism of scene generation.

[0008] Therefore, this invention proposes a dynamic scene generation method based on generative adversarial networks and spatiotemporally adaptive graph convolutional networks, aiming to overcome the shortcomings of existing technologies and improve the spatiotemporal consistency, coherence and diversity of generated scenes. Summary of the Invention

[0009] One objective of this invention is to propose a method for automatically generating dynamic scenes based on generative adversarial networks. This invention can provide an efficient and scientific optimization scheme in the automatic generation of dynamic scenes, bringing significant technical value and economic benefits to practical applications.

[0010] According to an embodiment of the present invention, a method for automatically generating dynamic scenes based on generative adversarial networks includes the following steps:

[0011] S1. Input a random noise vector and use a generative adversarial network to generate a preliminary spatiotemporal graph structure. The generator maps the noise vector into a high-dimensional spatiotemporal graph structure through an encoder.

[0012] S2. Input the generated spatiotemporal graph structure into the spatiotemporal adaptive graph convolutional network. Learn the spatiotemporal dependencies between objects and the background through graph convolution operations, capture the interaction between objects and the motion patterns that change over time, and extract the spatiotemporal features of the spatiotemporal graph structure.

[0013] S3. Based on the spatiotemporal features of the spatiotemporal graph structure, dynamically adjust the kernel weights in graph convolution operations;

[0014] S4. Utilize an adaptive learning mechanism, combine the spatiotemporal features of the spatiotemporal graph structure, optimize the learning strategy of the graph convolutional layer inside the generator, and further adjust the weights of the convolutional kernels based on the spatiotemporal smoothing factor.

[0015] S5. The generated spatiotemporal graph structure is input into the discriminator for evaluation. The discriminator judges the authenticity of the generated dynamic scene based on the spatiotemporal dependency and guides the generator to further optimize the generation process through feedback signals.

[0016] S6. Based on the feedback signal from the discriminator, optimize the parameters of the spatiotemporal graph convolutional layer of the generator, and further improve the quality of the generated dynamic scene through adversarial training;

[0017] S7. Iterate through steps S2 to S6 until the generated dynamic scene meets the requirements in terms of quality, diversity, coherence, and realism.

[0018] Optionally, S1 includes the following steps:

[0019] S11. Input random noise vector z, and through the encoder of the generative adversarial network, map the random noise vector z to a high-dimensional spatiotemporal graph structure G(z), where G is the generator.

[0020] S12. In the output layer of the encoder, a multi-layer convolutional network is used to perform preliminary feature extraction on the spatiotemporal graph structure G(z) to generate a graph structure containing spatiotemporal information. Each node in the graph represents an object or background in the dynamic scene, and the edges between nodes represent the spatiotemporal dependencies between objects and between objects and the background.

[0021] Optionally, S2 includes the following steps:

[0022] S21. Input the generated graph structure containing spatiotemporal information into the spatiotemporal adaptive graph convolutional network. Extract features from nodes and edges in the spatiotemporal graph structure through spatiotemporal convolution operations to capture the spatiotemporal dependencies between objects and between objects and the background, and learn the interaction between objects and the motion patterns that change over time.

[0023] S22. The spatiotemporal adaptive graph convolutional network dynamically calculates the spatiotemporal weights of each node based on a spatiotemporal adaptive mechanism, and performs feature extraction through a multi-layer convolutional network. The graph convolution operation of the l-th layer is represented as:

[0024]

[0025] in, Let v be the feature representation of node v at layer l+1. Let v be the neighborhood of node v. Let w be the feature representation of the neighbor node u at layer l. vu (tv ) represents the spatiotemporal weights between node v and its neighboring node u, and c represents the weights between them. vu b is the normalization factor. (l) Here, σ is the bias term, t is the Sigmoid activation function, and t is the t-term. v This represents the time step at which node v is located.

[0026] Optionally, S22 includes the following steps:

[0027] S221, The formula for calculating the spatiotemporal weight of each node is:

[0028] w vu (t v )=α·exp(-β·|t v -t u |)·(1+γ·distance(x v ,x u ));

[0029] Among them, w vu (t v ) represents the spatiotemporal weights between node v and node u, exp(.) is the natural exponential function, α, β, and γ are adjustment factors, and |t v -t u | represents the time difference between node v and node u, distance(x) v ,x u ) represents the spatial distance between node v and node u.

[0030] Optionally, S3 includes the following steps:

[0031] S31. Based on the features in the spatiotemporal graph structure, dynamically adjust the convolution kernel weights in the graph convolution operation, and calculate the weight update factor ΔW for each spatiotemporal node. v This factor is adaptively optimized based on the spatiotemporal dependencies of nodes:

[0032]

[0033] Wherein, ΔW v The weight update factor for node v. Let u be the feature representation of node u at layer l. Let v be the feature representation of node v at layer l. Let α represent the neighborhood of node v, α be the learning rate, and c be the learning rate. vu λ is the normalization factor, λ is the spatiotemporal dynamic correlation adjustment factor, and Δt is the normalization factor. u This represents the time difference between the time step at which node u is located and the current time step;

[0034] S32. Based on the results of the spatiotemporal graph convolution operation, fine-tune the graph convolution kernel weights at each time step so that the convolution kernel can adaptively adapt to the feature differences of different time steps and spatial regions. Within each time step, the convolution kernel weights W... v The update formula is:

[0035]

[0036] in, These are the weights of the (l+1)th layer convolutional kernel. These are the weights of the l-th layer convolutional kernel. Let v be the feature representation of node v at layer l. Let t be the feature representation of node v at layer l-1. v Let ΔW be the time step at which node v is located. v The weight update factor for node v. This is a non-linear adjustment function used to weight and adjust the features of the current node and the features of the previous layer, and a time step factor t is introduced. v This is to ensure the generator's dynamic response to changes in spatiotemporal characteristics.

[0037] Optionally, S4 includes the following steps:

[0038] S41. Utilize an adaptive learning mechanism to adjust the learning strategy of the spatiotemporal graph convolutional layer, optimize the feature extraction capability of the inner graph convolutional layer of the generator, introduce a smoothing adjustment mechanism for spatiotemporal features, and calculate the spatiotemporal smoothing factor for each node. This factor is adaptively adjusted based on changes in each time step and spatial location, and the update formula is as follows:

[0039] S42. Further adjust the weights of the convolution kernel based on the spatiotemporal smoothing factor:

[0040]

[0041] in, The weights of the convolutional kernel in layer (l+1) are... Here, α represents the weights of the convolutional kernel in layer l, and α is the learning rate. It is the node feature representation of the l-th layer.

[0042] Optionally, S41 includes the following steps:

[0043] S411, Spatiotemporal smoothing factor for each node The calculation method is as follows:

[0044]

[0045] in, For spatiotemporal smoothing factor, The neighborhood of node v is represented by β and γ, which are adjustment factors, and δ is the time difference decay factor. Let u be the feature representation of node u at layer l. Let c be the feature representation of node v at layer l. vu Let exp(.) be the normalization factor, and |t| be the natural exponential function. v -t u | represents the time difference between node v and node u.

[0046] Optionally, S5 includes the following steps:

[0047] S51. Input the generated spatiotemporal graph structure into the discriminator. The discriminator extracts the spatiotemporal features in the graph structure through the spatiotemporal graph convolutional layer.

[0048] S52. Based on the features output by the spatiotemporal graph convolutional layer, the discriminator calculates the spatiotemporal consistency and visual consistency of the generated scene, and generates the discrimination result using the following formula:

[0049]

[0050] Where D(G(z)) is the discriminator's evaluation of the authenticity of the generated spatiotemporal graph structure G(z). The weights of the convolutional kernel in layer l are... Let V be the feature representation of node v at layer l, where V represents all nodes in the graph and σ is the Sigmoid activation function.

[0051] S53. The discriminator evaluates the generated dynamic scene based on the spatiotemporal dependency, generates feedback signals, and guides the generator to further optimize the generation process based on the feedback signals.

[0052] S54. Based on the feedback signal given by the discriminator, the generator adjusts the learning strategy of the spatiotemporal graph convolutional layer to optimize the generation process of spatiotemporal graph features.

[0053] Optionally, S6 includes the following steps:

[0054] S61. Based on the feedback signal from the discriminator, update the spatiotemporal graph convolutional layer parameters of the generator using the backpropagation algorithm, and optimize the convolution kernel weights in the graph convolution operation. The update formula for the convolution kernel is:

[0055]

[0056] in, The weights of the convolutional kernel in layer (l+1) are... Here, η represents the weights of the convolutional kernel in layer l, and η is the learning rate. Generator loss function The gradient relative to the convolution kernel weights;

[0057] S62. Through adversarial training between the generator and the discriminator, based on the loss function... Optimize the spatiotemporal graph convolutional layer feature representation in the generator by adjusting the parameters of the spatiotemporal graph convolutional layer in the generator.

[0058] Optionally, S62 includes the following steps:

[0059] S621, the loss function for:

[0060]

[0061] in, For the desired operation, P data For the true data distribution, p z Let G(x) be the random noise distribution input to the generator, D(x) be the true data judgment value output by the discriminator, and G(z) be the generated spatiotemporal graph structure.

[0062] The beneficial effects of this invention are:

[0063] (1) Based on the combination of generative adversarial networks and spatiotemporally adaptive graph convolutional networks, this invention can effectively solve the problems of insufficient spatiotemporal feature extraction, inaccurate spatiotemporal dependency modeling, and stability of the generation process in existing dynamic scene generation methods. By introducing the spatiotemporal adaptive mechanism of graph convolutional networks into the generator, the learning strategy of the convolutional kernel can be dynamically adjusted according to the different time steps and spatial regions, thereby accurately capturing the complex spatiotemporal interaction relationships between objects and between objects and the background in the scene. This adaptive capability enables the generator to maintain good spatiotemporal consistency and coherence when facing dynamic scenes with varying characteristics.

[0064] (2) The generator of the present invention dynamically optimizes the weights of the graph convolution kernel at each time step, avoiding the limitation of fixed spatiotemporal feature processing in traditional methods. Through this flexible spatiotemporal feature fusion mechanism, the generated scene not only has a reasonable object layout in space, but also effectively captures the dynamic characteristics of objects changing over time, improving the naturalness and realism of scene generation. In addition, through the optimization of the generator, the generated dynamic scene can still maintain the stability of spatiotemporal dependency in long-term time series, avoiding the gradient vanishing and gradient explosion problems common in traditional RNN and LSTM models in long-term series.

[0065] (3) By applying a spatiotemporal adaptive graph convolutional network, this invention effectively improves the quality and diversity of generated dynamic scenes. Scenes generated by traditional methods often perform poorly in spatiotemporal regions with high complexity of change. However, the solution of this invention can adaptively adjust for each moment and each spatial region, so that the generated scenes not only conform to spatiotemporal laws, but also have richer visual effects and higher detail expression. The generated scenes can be applied in fields such as virtual reality, augmented reality and autonomous driving, meeting the high requirements for spatiotemporal consistency, realism and dynamic change. Attached Figure Description

[0066] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0067] Figure 1 This is a flowchart of a dynamic scene automatic generation method based on generative adversarial networks proposed in this invention;

[0068] Figure 2 This is a flowchart illustrating the extraction of spatiotemporal features of spatiotemporal graph structure in a dynamic scene automatic generation method based on generative adversarial networks proposed in this invention.

[0069] Figure 3 This is a flowchart of the generative adversarial network (GAN) training process in the dynamic scene automatic generation method based on GAN proposed in this invention. Detailed Implementation

[0070] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0071] refer to Figures 1-3 A method for automatically generating dynamic scenes based on generative adversarial networks includes the following steps:

[0072] S1. Input a random noise vector and use a generative adversarial network to generate a preliminary spatiotemporal graph structure. The generator maps the noise vector into a high-dimensional spatiotemporal graph structure through an encoder.

[0073] S2. Input the generated spatiotemporal graph structure into the spatiotemporal adaptive graph convolutional network. Learn the spatiotemporal dependencies between objects and the background through graph convolution operations, capture the interaction between objects and the motion patterns that change over time, and extract the spatiotemporal features of the spatiotemporal graph structure.

[0074] S3. Based on the spatiotemporal features of the spatiotemporal graph structure, dynamically adjust the kernel weights in graph convolution operations;

[0075] S4. Utilize an adaptive learning mechanism, combine the spatiotemporal features of the spatiotemporal graph structure, optimize the learning strategy of the graph convolutional layer inside the generator, and further adjust the weights of the convolutional kernels based on the spatiotemporal smoothing factor.

[0076] S5. The generated spatiotemporal graph structure is input into the discriminator for evaluation. The discriminator judges the authenticity of the generated dynamic scene based on the spatiotemporal dependency and guides the generator to further optimize the generation process through feedback signals.

[0077] S6. Based on the feedback signal from the discriminator, optimize the parameters of the spatiotemporal graph convolutional layer of the generator, and further improve the quality of the generated dynamic scene through adversarial training;

[0078] S7. Iterate through steps S2 to S6 until the generated dynamic scene meets the requirements in terms of quality, diversity, coherence, and realism.

[0079] In the initialization step of the generative adversarial network, random noise vectors are mapped to high-dimensional spatiotemporal graph structures, constructing a preliminary representation of the dynamic scene. Through this process, the generator can generate graph structures with spatiotemporal characteristics from random noise, thus providing a foundation for subsequent spatiotemporal dependency modeling. This method solves the problem of insufficient spatiotemporal information processing in traditional scene generation, laying a solid foundation for generating dynamic scenes with spatiotemporal consistency and coherence.

[0080] In this embodiment, S1 includes the following steps:

[0081] S11. Input random noise vector z, and through the encoder of the generative adversarial network, map the random noise vector z to a high-dimensional spatiotemporal graph structure G(z), where G is the generator.

[0082] S12. In the output layer of the encoder, a multi-layer convolutional network is used to perform preliminary feature extraction on the spatiotemporal graph structure G(z) to generate a graph structure containing spatiotemporal information. Each node in the graph represents an object or background in the dynamic scene, and the edges between nodes represent the spatiotemporal dependencies between objects and between objects and the background.

[0083] Spatiotemporal adaptive graph convolutional networks learn the spatiotemporal dependencies between objects and the background through graph convolution operations, capturing the interactions between objects and the motion patterns that change over time. This process can effectively capture the spatiotemporal features in dynamic scenes and improve the model's ability to learn complex relationships between objects. Compared with traditional convolutional neural networks and recurrent neural networks, spatiotemporal adaptive graph convolutional networks can better cope with multi-dimensional changes in spatiotemporal structures and improve the naturalness and diversity of generated scenes.

[0084] In this embodiment, S2 includes the following steps:

[0085] S21. Input the generated graph structure containing spatiotemporal information into the spatiotemporal adaptive graph convolutional network. Extract features from nodes and edges in the spatiotemporal graph structure through spatiotemporal convolution operations to capture the spatiotemporal dependencies between objects and between objects and the background, and learn the interaction between objects and the motion patterns that change over time.

[0086] S22. The spatiotemporal adaptive graph convolutional network dynamically calculates the spatiotemporal weights of each node based on a spatiotemporal adaptive mechanism, and performs feature extraction through a multi-layer convolutional network. The graph convolution operation of the l-th layer is represented as:

[0087]

[0088] in, Let v be the feature representation of node v at layer l+1. Let v be the neighborhood of node v. Let w be the feature representation of the neighbor node u at layer l. vu (t v ) represents the spatiotemporal weights between node v and its neighboring node u, and c represents the weights between them. vu b is the normalization factor. (l) Here, σ is the bias term, t is the Sigmoid activation function, and t is the t-term. v This represents the time step at which node v is located.

[0089] In this embodiment, S22 includes the following steps:

[0090] S221, The formula for calculating the spatiotemporal weight of each node is:

[0091] w vu (t v )=α·exp(-β·|t v -t u |)·(1+γ·distance(x v ,x u ));

[0092] Among them, w vu (t v ) represents the spatiotemporal weights between node v and node u, exp(.) is the natural exponential function, α, β, and γ are adjustment factors, and |t v -t u | represents the time difference between node v and node u, distance(x) v ,x u ) represents the spatial distance between node v and node u.

[0093] In the feature adjustment based on spatiotemporal graph structure, the dynamic adjustment of the weights of the graph convolution kernel ensures that the scene generated at each time step can correctly reflect spatiotemporal changes and interactions between objects. This adaptive mechanism enhances the model's adaptability to different time steps and spatial regions, enabling the scene generation to dynamically adapt to the motion trajectory of objects and changes in the environment. This avoids the limitations of traditional methods in dynamically changing environments and improves the spatiotemporal coherence of the generated scene.

[0094] In this embodiment, S3 includes the following steps:

[0095] S31. Based on the features in the spatiotemporal graph structure, dynamically adjust the convolution kernel weights in the graph convolution operation, and calculate the weight update factor ΔW for each spatiotemporal node. v This factor is adaptively optimized based on the spatiotemporal dependencies of nodes:

[0096]

[0097] Wherein, ΔW v The weight update factor for node v. Let u be the feature representation of node u at layer l. Let v be the feature representation of node v at layer l. Let α represent the neighborhood of node v, α be the learning rate, and c be the learning rate. vu λ is the normalization factor, λ is the spatiotemporal dynamic correlation adjustment factor, and Δt is the normalization factor. u This represents the time difference between the time step at which node u is located and the current time step;

[0098] S32. Based on the results of the spatiotemporal graph convolution operation, fine-tune the graph convolution kernel weights at each time step so that the convolution kernel can adaptively adapt to the feature differences of different time steps and spatial regions. Within each time step, the convolution kernel weights W... v The update formula is:

[0099]

[0100] in, These are the weights of the (l+1)th layer convolutional kernel. These are the weights of the l-th layer convolutional kernel. Let v be the feature representation of node v at layer l. Let t be the feature representation of node v at layer l-1. v Let ΔW be the time step at which node v is located. v The weight update factor for node v. This is a non-linear adjustment function used to weight and adjust the features of the current node and the features of the previous layer, and a time step factor t is introduced. v This is to ensure the generator's dynamic response to changes in spatiotemporal characteristics.

[0101] By introducing an adaptive learning mechanism and combining it with the spatiotemporal features of the spatiotemporal graph structure, the learning strategy of the graph convolutional layer inside the generator is optimized, making the scene generation more accurate for different time periods and spatial regions. This mechanism further enhances the model's ability to capture complex spatiotemporal features, resulting in a significant improvement in the diversity and accuracy of the generated scenes. The adaptive learning capability enables the generator to flexibly adjust in different scenes, ensuring high-quality scene generation.

[0102] In this embodiment, S4 includes the following steps:

[0103] S41. Utilize an adaptive learning mechanism to adjust the learning strategy of the spatiotemporal graph convolutional layer, optimize the feature extraction capability of the inner graph convolutional layer of the generator, introduce a smoothing adjustment mechanism for spatiotemporal features, and calculate the spatiotemporal smoothing factor for each node. This factor is adaptively adjusted based on changes in each time step and spatial location, and the update formula is as follows:

[0104] S42. Further adjust the weights of the convolution kernel based on the spatiotemporal smoothing factor:

[0105]

[0106] in, The weights of the convolutional kernel in layer (l+1) are... Here, α represents the weights of the convolutional kernel in layer l, and α is the learning rate. It is the node feature representation of the l-th layer.

[0107] In this embodiment, S41 includes the following steps:

[0108] S411, Spatiotemporal smoothing factor for each node The calculation method is as follows:

[0109]

[0110] in, For spatiotemporal smoothing factor, The neighborhood of node v is represented by β and γ, which are adjustment factors, and δ is the time difference decay factor. Let u be the feature representation of node u at layer l. Let c be the feature representation of node v at layer l. vu Let exp(.) be the normalization factor, and |t| be the natural exponential function. v -t u | represents the time difference between node v and node u.

[0111] During the discriminator evaluation process, the spatiotemporal graph structure is input into the discriminator for realism assessment. The discriminator judges the generated dynamic scene based on spatiotemporal dependencies and passes the feedback signal to the generator for optimization. This process ensures that the dynamic scene generated by the generator not only has visual consistency, but also conforms to the laws of the real world in terms of time and space, further improving the naturalness and credibility of the generated result.

[0112] In this embodiment, S5 includes the following steps:

[0113] S51. Input the generated spatiotemporal graph structure into the discriminator. The discriminator extracts the spatiotemporal features in the graph structure through the spatiotemporal graph convolutional layer.

[0114] S52. Based on the features output by the spatiotemporal graph convolutional layer, the discriminator calculates the spatiotemporal consistency and visual consistency of the generated scene, and generates the discrimination result using the following formula:

[0115]

[0116] Where D(G(z)) is the discriminator's evaluation of the authenticity of the generated spatiotemporal graph structure G(z). The weights of the convolutional kernel in layer l are... Let V be the feature representation of node v at layer l, where V represents all nodes in the graph and σ is the Sigmoid activation function.

[0117] S53. The discriminator evaluates the generated dynamic scene based on the spatiotemporal dependency, generates feedback signals, and guides the generator to further optimize the generation process based on the feedback signals.

[0118] S54. Based on the feedback signal given by the discriminator, the generator adjusts the learning strategy of the spatiotemporal graph convolutional layer to optimize the generation process of spatiotemporal graph features.

[0119] The generator optimizes the parameters of the spatiotemporal graph convolutional layer based on the feedback signal from the discriminator, thereby improving the quality of the generated dynamic scene. Through adversarial training, the generator can continuously improve the generation strategy and achieve high accuracy and naturalness in multiple dimensions such as spatiotemporal consistency, visual effects and dynamic changes. The optimization process effectively avoids the shortcomings of traditional models in generating high-quality dynamic scenes and further improves the stability and realism of the scene.

[0120] In this embodiment, S6 includes the following steps:

[0121] S61. Based on the feedback signal from the discriminator, update the spatiotemporal graph convolutional layer parameters of the generator using the backpropagation algorithm, and optimize the convolution kernel weights in the graph convolution operation. The update formula for the convolution kernel is:

[0122]

[0123] in, The weights of the convolutional kernel in layer (l+1) are... Here, η represents the weights of the convolutional kernel in layer l, and η is the learning rate. Generator loss function The gradient relative to the convolution kernel weights;

[0124] S62. Through adversarial training between the generator and the discriminator, based on the loss function... Optimize the spatiotemporal graph convolutional layer feature representation in the generator by adjusting the parameters of the spatiotemporal graph convolutional layer in the generator.

[0125] In this embodiment, S62 includes the following steps:

[0126] S621, the loss function for:

[0127]

[0128] in, For the desired operation, P data For the true data distribution, p z Let G(x) be the random noise distribution input to the generator, D(x) be the true data judgment value output by the discriminator, and G(z) be the generated spatiotemporal graph structure.

[0129] Example:

[0130] In virtual reality game development, the generation of dynamic scenes is the key to achieving a high-quality immersive experience. In order to provide a realistic and dynamic virtual environment, this embodiment applies a dynamic scene generation method based on generative adversarial networks and spatiotemporal adaptive graph convolutional networks. This method effectively solves the problems of inaccurate spatiotemporal feature modeling, inflexible capture of spatiotemporal dependencies, and lack of diversity and coherence in the generation process in the prior art.

[0131] In this embodiment, the implementer uses a virtual city simulation game as the scenario. This game needs to simulate a city environment containing multiple dynamic events, such as human activities, traffic flow, and weather changes. Traditional game environment generation often struggles to provide a coherent scene in terms of space and time, resulting in a relatively monotonous player experience. To address this issue, the implementer uses a dynamic scene generation method based on generative adversarial networks and spatiotemporally adaptive graph convolutional networks to generate spatiotemporally dependent city scenes from random noise, ensuring that the interaction relationships between each dynamic event are natural and realistic.

[0132] Specifically, each scene in the game is composed of multiple elements, including characters, vehicles, buildings, roads, weather, etc. The implementer generates a preliminary spatiotemporal graph structure by inputting random noise vectors, and uses a spatiotemporal adaptive graph convolutional network to capture the spatiotemporal dependencies between objects and between objects and the background. For example, when generating a street scene, the spatiotemporal adaptive graph convolutional network dynamically adjusts the generation process according to different time steps and spatial regions to ensure that the spatial relationships between pedestrians, vehicles and buildings on the street are reasonable, and that the changes in dynamic events in the scene (such as vehicle flow, pedestrian movement, etc.) conform to physical laws over time.

[0133] When optimizing the generator, the implementer adjusts the parameters of the spatiotemporal graph convolutional layer by the feedback signal of the discriminator, making the generated scene more realistic. For example, the generated traffic scene can dynamically adjust factors such as traffic flow and pedestrian density according to weather changes (sunny, rainy, snowy, etc.). The flow of vehicles and the behavior of pedestrians in the generated street scene can also present a natural pattern as time changes. In addition, through the spatiotemporal adaptive mechanism, the generator can generate diverse and natural scenes in different time periods (such as morning, noon, and night) and spatial areas (such as city center and suburbs).

[0134] To verify the effectiveness of the method of the present invention, it was compared with traditional dynamic scene generation methods based on convolutional neural networks and recurrent neural networks. Generation and evaluation were carried out for the same scene. Through game testing, the implementer selected three typical scenes: a busy city street, a quiet suburban road, and a street scene affected by weather changes. In each scene, the implementer tested the spatiotemporal coherence of the generation, the naturalness of object interaction, and the visual effects.

[0135] The following are the specific data after implementation:

[0136] Table 1 Test Data Table for Dynamic Scene Generation Method

[0137]

[0138] These experimental data demonstrate that the dynamic scene generation method based on generative adversarial networks and spatiotemporally adaptive graph convolutional networks outperforms traditional methods in terms of spatiotemporal coherence, naturalness of object interaction, and visual effects. Furthermore, it generates scenes in a shorter time and can more efficiently provide high-quality dynamic scenes for applications such as virtual reality and intelligent transportation. This verifies the beneficial effects of the method of this invention in dynamic scene generation.

[0139] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for automatically generating dynamic scenes based on generative adversarial networks, characterized in that, Includes the following steps: S1. Input a random noise vector and use a generative adversarial network to generate a preliminary spatiotemporal graph structure. The generator maps the noise vector into a high-dimensional spatiotemporal graph structure through an encoder. S2. Input the generated spatiotemporal graph structure into the spatiotemporal adaptive graph convolutional network. Learn the spatiotemporal dependencies between objects and the background through graph convolution operations, capture the interaction between objects and the motion patterns that change over time, and extract the spatiotemporal features of the spatiotemporal graph structure. S3. Based on the spatiotemporal features of the spatiotemporal graph structure, dynamically adjust the kernel weights in graph convolution operations; S4. Utilize an adaptive learning mechanism, combine the spatiotemporal features of the spatiotemporal graph structure, optimize the learning strategy of the graph convolutional layer inside the generator, and further adjust the weights of the convolutional kernels based on the spatiotemporal smoothing factor. S5. The generated spatiotemporal graph structure is input into the discriminator for evaluation. The discriminator judges the authenticity of the generated dynamic scene based on the spatiotemporal dependency and guides the generator to further optimize the generation process through feedback signals. S6. Based on the feedback signal from the discriminator, optimize the parameters of the spatiotemporal graph convolutional layer of the generator, and further improve the quality of the generated dynamic scene through adversarial training; S7. Iterate through steps S2 to S6 until the generated dynamic scene meets the requirements in terms of quality, diversity, coherence and realism. S3 includes the following steps: S31. Based on the features in the spatiotemporal graph structure, dynamically adjust the convolution kernel weights in the graph convolution operation, and calculate the weight update factor for each spatiotemporal node. : ； in, For nodes The weight update factor, For nodes In the Layer feature representation, For nodes In the Layer feature representation, Represents a node The neighborhood, For learning rate, As the normalization factor, As a spatiotemporal dynamic correlation modulator, Represents a node The time difference between the current time step and the current time step; S32. Based on the results of the spatiotemporal graph convolution operation, fine-tune the graph convolution kernel weights at each time step so that the convolution kernel can adaptively adapt to the feature differences of different time steps and spatial regions. Within each time step, the convolution kernel weights... Updated to: ； in, It is the first Layer convolution kernel weights, It is the first Layer convolution kernel weights, For nodes In the Layer feature representation, For nodes In the Layer feature representation, For nodes Time step, For nodes The weight update factor, It is a nonlinear adjustment function.

2. The method for automatically generating dynamic scenes based on generative adversarial networks according to claim 1, characterized in that, S1 includes the following steps: S11. Input random noise vector z, and through the encoder of the generative adversarial network, map the random noise vector z to a high-dimensional spatiotemporal graph structure G(z), where G is the generator. S12. In the output layer of the encoder, the spatiotemporal graph structure is processed through a multi-layer convolutional network. Preliminary feature extraction is performed to generate a graph structure containing spatiotemporal information. Each node in the graph represents an object or background in a dynamic scene, and the edges between nodes represent the spatiotemporal dependencies between objects and between objects and the background.

3. The method for automatically generating dynamic scenes based on generative adversarial networks according to claim 1, characterized in that, S2 includes the following steps: S21. Input the generated graph structure containing spatiotemporal information into the spatiotemporal adaptive graph convolutional network. Extract features from nodes and edges in the spatiotemporal graph structure through spatiotemporal convolution operations to capture the spatiotemporal dependencies between objects and between objects and the background, and learn the interaction between objects and the motion patterns that change over time. S22. The spatiotemporal adaptive graph convolutional network dynamically calculates the spatiotemporal weights of each node based on a spatiotemporal adaptive mechanism, and performs feature extraction through a multi-layer convolutional network. The layer graph convolution operation is represented as: ； in, For nodes In the Layer feature representation, For nodes The neighborhood, For neighboring nodes In the Layer feature representation, For nodes and neighboring nodes The spatiotemporal weights between them As the normalization factor, For bias terms, It is the Sigmoid activation function. For nodes The current time step.

4. The method for automatically generating dynamic scenes based on generative adversarial networks according to claim 3, characterized in that, S22 includes the following steps: S221, the formula for calculating the spatiotemporal weight of each node is: ； in, For nodes and nodes The spatiotemporal weights between them It is a natural exponential function. As a regulating factor, For nodes and nodes Time difference, Represents a node and nodes Distance in space.

5. The method for automatically generating dynamic scenes based on generative adversarial networks according to claim 1, characterized in that, S4 includes the following steps: S41. Utilize an adaptive learning mechanism to adjust the learning strategy of the spatiotemporal graph convolutional layer, optimize the feature extraction capability of the inner graph convolutional layer of the generator, introduce a smoothing adjustment mechanism for spatiotemporal features, and calculate the spatiotemporal smoothing factor for each node. ; S42. Adjust the weights of the convolution kernels further based on the spatiotemporal smoothing factor.

6. The method for automatically generating dynamic scenes based on generative adversarial networks according to claim 5, characterized in that, S41 includes the following steps: S411, Spatiotemporal smoothing factor for each node The calculation method is as follows: ； in, For spatiotemporal smoothing factor, Represents a node The neighborhood, and As a regulating factor, The time difference decay factor, For nodes In the Layer feature representation, For nodes In the Layer feature representation, As the normalization factor, It is a natural exponential function. Represents a node and nodes The time difference between them.

7. The method for automatically generating dynamic scenes based on generative adversarial networks according to claim 1, characterized in that, S5 includes the following steps: S51. Input the generated spatiotemporal graph structure into the discriminator. The discriminator extracts the spatiotemporal features in the graph structure through the spatiotemporal graph convolutional layer. S52. Based on the features output by the spatiotemporal graph convolutional layer, the discriminator calculates the spatiotemporal consistency and visual consistency of the generated scene and generates a discrimination result. S53. The discriminator evaluates the generated dynamic scene based on the spatiotemporal dependency, generates feedback signals, and guides the generator to further optimize the generation process based on the feedback signals. S54. Based on the feedback signal given by the discriminator, the generator adjusts the learning strategy of the spatiotemporal graph convolutional layer to optimize the generation process of spatiotemporal graph features.

8. The method for automatically generating dynamic scenes based on generative adversarial networks according to claim 1, characterized in that, S6 includes the following steps: S61. Based on the feedback signal given by the discriminator, update the spatiotemporal graph convolutional layer parameters of the generator using the backpropagation algorithm to optimize the kernel weights in the graph convolution operation. S62. Through adversarial training between the generator and the discriminator, based on the loss function... Optimize the spatiotemporal graph convolutional layer feature representation in the generator by adjusting the parameters of the spatiotemporal graph convolutional layer in the generator.

9. The method for automatically generating dynamic scenes based on generative adversarial networks according to claim 8, characterized in that, S62 includes the following steps: S621, the loss function for: ； in, For the desired operation, For the true data distribution, The random noise distribution is the input to the generator. The true data judgment value output by the discriminator. The generated spatiotemporal graph structure.

Images