Game image picture AI intelligent generation and optimization processing system

Abstract

The application belongs to the technical field of image generation, and discloses an AI intelligent generation and optimization processing system for game image pictures, which runs on a computing device loaded with a GPU and a game engine. The system first acquires basic three-dimensional assets of a game scene and scene generation instructions, constructs scene implicit representation by using a neural radiation field technology, and completes multi-view consistency verification. A lattice Boltzmann simulation algorithm is used to calculate space-time sequence data of fluid and particle dynamic elements. A multi-scale semantic alignment Vision Transformer generation network is used to fuse the two types of data to generate a two-dimensional game picture feature map. After depth buffer and light map adaptation, the generated picture is embedded into a game engine real-time rendering process. Through adaptive tensor decomposition, proxy model acceleration and other optimization strategies, combined with physical constraint closed loop verification, the picture generation precision, efficiency and physical authenticity are improved, high-fidelity, real-time intelligent generation and optimization of game pictures are realized, and the system is suitable for multi-scene game development requirements.

Description

Technical Field

[0001] This invention relates to the field of image generation technology and discloses an AI-powered intelligent generation and optimization system for game images, which runs on a computing device equipped with a GPU and a game engine. Background Technology

[0002] Current game scene image generation technologies mainly rely on manual modeling or image compositing methods based on 2D generative models. Manual modeling requires artists to construct 3D assets one by one, which is time-consuming and labor-intensive. Methods based on 2D generative models utilize generative adversarial networks or diffusion models to generate images in 2D pixel space based on text or sketches. These methods focus on the synthesis of textures and styles on the surface of the image, and their generation process does not establish a relationship between the 3D geometric topology and physical properties of the scene.

[0003] The aforementioned 2D generation model lacks a 3D spatial constraint mechanism, which makes the generated dynamic elements prone to logical errors such as clipping and violating physical laws during movement, and it is difficult to maintain spatial consistency in the generated content from different perspectives. At the same time, the 2D image data generated by the existing method cannot be directly converted into the depth buffer and lightmap data required by the game engine rendering pipeline, so the generated results cannot be directly used by the real-time rendering process and require heavy manual secondary modification and format conversion. Summary of the Invention

[0004] The purpose of this invention is to provide a solution that can effectively address the problems described in the background section.

[0005] To achieve the above objectives, the technical solution adopted by the present invention includes: Obtain basic 3D assets and scene generation instructions for the game scene; The neural radiation field technology is used to construct implicit representations and verify multi-view consistency of basic 3D assets, and output scene implicit representation data. Based on the boundary conditions of the scene's implicit representation data, the lattice Boltzmann simulation algorithm is used to calculate the spatiotemporal sequence data of fluid and particle dynamic elements; The implicit representation data of the scene and the spatiotemporal sequence data are input into the VisionTransformer generative network with multi-scale semantic alignment, which performs pixel-level generation and outputs a two-dimensional game screen feature map. The feature map of the 2D game screen is adapted and mapped by depth buffering, light mapping and rendering queue, and the generated screen is embedded into the game engine's real-time rendering process.

[0006] The implicit representation construction and multi-view consistency verification of the basic 3D assets using neural radiation field technology includes: The basic 3D assets are voxelized to extract geometric feature point clouds; Multi-resolution hash grid coding technology is used to perform feature mapping on geometric feature point clouds to construct sparse scene representations; The spatial density and color values ​​of sparse scene representations are predicted by a fully connected neural network, multi-view images are generated using a volume rendering equation, and multi-view consistency verification is completed by calculating the structural similarity loss function between the synthetic image and the real view image.

[0007] Furthermore, the calculation of the spatiotemporal sequence data of fluid and particle dynamic elements using the lattice Boltzmann simulation algorithm includes: extracting the boundary grid information of fluid and particle motion based on the scene implicit representation data; Construct a D3Q19 discrete velocity model and initialize the distribution function; The macroscopic physical quantities of fluid micro-elements are calculated iteratively using collision and migration operators, with the collision operator employing a multi-relaxation time model to ensure numerical stability. Apply bounce boundary conditions based on the boundary grid information, and output fluid velocity field and particle trajectory data that satisfy mass and momentum conservation as spatiotemporal sequence data.

[0008] Further, according to the system of claim 1, the step of inputting the scene implicit representation data and spatiotemporal sequence data into a multi-scale semantically aligned Vision Transformer generator network to perform pixel-level generation and output a two-dimensional game screen feature map includes: The implicit representation data of the scene is encoded into a first feature vector, and the spatiotemporal sequence data is encoded into a second feature vector; Construct a hierarchical Transformer encoder and use a self-attention mechanism to fuse the first feature vector and the second feature vector; The semantic similarity between fused features and text description instructions is calculated through a cross-modal semantic alignment module to optimize the semantic representation of the feature map; The optimized feature map is upsampled into a two-dimensional game screen using a decoder, and a frequency domain loss function is introduced to constrain the generation of high-frequency texture details.

[0009] Furthermore, the adaptation mapping of depth buffering, light mapping, and rendering queue for the feature map of the 2D game screen includes: Parse the geometry buffer data format requirements of the game engine; Depth and normal information are extracted from the feature map of the 2D game screen to generate depth buffer data and normal map adapted to the game engine. Based on the lighting probe information in the scene's implicit representation data, generate light maps and shadow maps; Establish a rendering priority queue, package depth buffer data, normal maps, light maps, and shadow maps into rendering call instructions, and inject them into the game engine's deferred rendering pipeline.

[0010] Furthermore, an adaptive tensor decomposition strategy is introduced during the construction of the sparse scene representation: The feature tensors in the multi-resolution hash grid are decomposed into normalized polynomials, which decompose the high-dimensional feature tensors into the product of the low-dimensional core tensor and the factor matrix, thus compressing the storage space. A Bayesian optimization algorithm is introduced to evaluate the contribution of the current view sampling to the accuracy of the implicit representation, and the number of sampling rays is dynamically adjusted. When the convergence error of the scene's implicit representation data is lower than a preset threshold, the parameter updates of the fully connected neural network are automatically frozen, and redundant intermediate feature caches are released.

[0011] Furthermore, the process of iteratively calculating macroscopic physical quantities of fluid micro-elements using collision and migration operators incorporates a surrogate model acceleration technique: A fluid dynamics surrogate model based on graph neural networks is constructed, which uses the fluid velocity field at the current moment as input to predict the rough solution of macroscopic physical quantities at the next moment. Using the coarse solution as the initial guess value for the iterative calculation of the lattice Boltzmann simulation algorithm reduces the number of iterations required to reach steady state; By using a reinforcement learning agent to adjust the relaxation parameters in a multi-relaxation time model in real time based on the fluid Reynolds number, the time step of the simulation calculation is optimized while ensuring the physical conservation laws.

[0012] Furthermore, the cross-modal semantic alignment module integrates style transfer and knowledge distillation mechanisms: Load a pre-trained art style knowledge base and transfer the texture style of the 2D game screen to the target art style through an adaptive instance normalization algorithm; Construct a teacher-student network architecture, where the teacher network is a high-parameter visual generation model and the student network is a lightweight deployment model; By minimizing the KL divergence between the teacher and student networks in the intermediate layer feature maps, high-fidelity generation capabilities are transferred to lightweight networks.

[0013] Furthermore, frame prediction and resource preloading are performed before the deferred rendering pipeline injected into the game engine: Based on the game screen feature map of historical frames, a long short-term memory network is used to predict the trend of depth buffer data change in the next frame. Based on the prediction results, the video memory bandwidth is scheduled in advance, and the expected updated light map data is loaded into the cache. By using a double buffering mechanism, the rendering call instructions are assembled in the background. When the main thread rendering logic of the game engine is triggered, the buffer pointers are directly swapped to eliminate the delay of the rendering pipeline waiting for the generation of results.

[0014] Furthermore, a physical constraint loss function is introduced to perform closed-loop verification of the generated results: In the computation process of surrogate model acceleration technology and lattice Boltzmann simulation algorithm, a physical law constraint layer is embedded to calculate the residuals of fluid continuity equation and Navier-Stokes equation; The residuals are backpropagated to the Vision Transformer generative network with multi-scale semantic alignment to correct the pixel distribution of the generated image. If the corrected physical error still exceeds the preset tolerance range, a rollback mechanism is triggered to reinitialize the boundary conditions and iteratively calculate until a game image that meets the physical realism index is generated.

[0015] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention employs neural radiation field technology to construct an implicit representation of the scene, performs multi-view consistency verification on the basic 3D assets, establishes 3D spatial geometric constraints on the scene, and eliminates content conflicts in the generated images from different perspectives. It utilizes the lattice Boltzmann simulation algorithm to calculate the spatiotemporal sequence data of fluids and particles based on boundary conditions, ensuring that dynamic elements follow the laws of conservation of mass and momentum, thus resolving the problems of clipping and physical logic errors in the generated images.

[0016] 2. This invention utilizes a game engine real-time rendering pipeline adaptation module to transform the feature map of the generated image into depth buffers, lightmaps, and rendering queue data, eliminating the need for manual image retouching and asset conversion. Combined with a multi-scale semantically aligned visual transformer generation network, it ensures semantic consistency between the generated image and user commands. Attached Figure Description

[0017] Figure 1 This is an overall flowchart of the present invention; Figure 2 This is a flowchart illustrating the implicit representation construction and multi-perspective consistency verification process for the present invention. Figure 3 This is a flowchart of the simulation calculation process for lattice Boltzmann fluid and particle dynamic elements in this invention; Figure 4 This is a flowchart of the game screen generation process of the Vision Transformer multi-scale semantic alignment of the present invention. Figure 5 This is a flowchart illustrating the game screen rendering adaptation and engine embedding process of the present invention. Figure 6 This is a flowchart of the system optimization strategy and physical constraint closed-loop verification process of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] As a preferred embodiment, please refer to the appendix. Figure 1 To be continued Figure 5 The system first acquires the basic 3D assets of the game scene and scene generation instructions. The basic 3D assets of the game scene include the static 3D model of the game scene, material textures, lighting parameters, and collision data. The format of the 3D model includes any one of FBX, OBJ, and glTF. The scene generation instructions include text-based scene description instructions, viewpoint parameter instructions, and dynamic element configuration instructions. The dynamic element configuration instructions include fluid type, particle emission source parameters, and motion duration parameters.

[0020] The system utilizes neural radiation field technology to construct implicit representations and perform multi-view consistency verification on basic 3D assets, outputting scene implicit representation data. The specific execution process is as follows: The basic 3D asset is voxelized to extract geometric feature point clouds. The 3D model of the basic 3D asset is imported into the voxelization module. The bounding box space of the 3D model is uniformly divided at a preset voxel resolution to generate a 3D voxel mesh. A signed distance function is performed on each voxel mesh to determine whether it lies on the surface or inside the 3D model. The center coordinates of voxel meshes located on the model surface are retained as geometric feature points. All geometric feature points constitute the geometric feature point cloud. ,in N is the total number of geometric feature points.

[0021] Multi-resolution hash grid coding technology is used to perform feature mapping on geometric feature point clouds to construct sparse scene representations. A hash grid with L levels of resolution is set, with each level's resolution starting from 16... 3 Linear growth to 2048 3 For each level of the hash grid, the geometric feature points are... The 3D coordinates are mapped to the grid vertices of that level of mesh, and the indices of the grid vertices are encoded using a hash function, which is:

[0022] in, , , This represents the integer coordinate index of the grid vertex at the corresponding resolution. This is the bitwise XOR operator. The maximum number of entries in the hash table; for each level of the hash grid, assign an F-dimensional feature vector to each hash entry; calculate the geometric feature points using trilinear interpolation. The encoded features in each level of the hash grid are concatenated, and the encoded features of the L-level hash grids are then combined to obtain geometric feature points. The corresponding high-dimensional feature vector The high-dimensional feature vectors of all geometric feature points constitute a sparse scene representation.

[0023] The spatial density and color values ​​of sparse scene representations are predicted using a fully connected neural network. Multi-view images are generated using a volumetric rendering equation, and multi-view consistency is verified by calculating the structural similarity loss function between the synthesized images and the real-view images. The fully connected neural network consists of 8 fully connected layers, and the input is a high-dimensional feature vector of the sparse scene representation. The first 6 fully connected layers output spatial density The last two fully connected layers combine the view direction vector d∈R 3 Output RGB color values .

[0024] For any sampling ray ,in For the camera's optical center coordinates, Let be the unit direction vector of the ray. Given the sampling depth of the ray, the pixel color value corresponding to the ray is calculated using the volume rendering equation. The volume rendering equation is:

[0025] in, Let be the near-plane depth of the ray. Let be the far-plane depth of the ray. For rays from arrive Transmittance at that location .

[0026] In numerical calculations, the sampled ray is uniformly discretely sampled to obtain N_t sampling points. The discrete form of the volume rendering equation is:

[0027] in, For the i-th sampling point and the i-th sampling point +1 depth interval between sampling points .

[0028] The specific process of multi-view consistency verification is as follows: obtain the corresponding data of the basic 3D asset. Camera parameters and real images from a real viewpoint are obtained, wherein the camera parameters include intrinsic and extrinsic parameter matrices; based on the camera parameters, a synthetic image corresponding to the viewpoint is generated using the aforementioned volumetric rendering equation; the structural similarity loss function between the synthetic image and the corresponding real viewpoint image is calculated. and mean squared error loss function Total loss function ,in The weight coefficients are used to iteratively optimize the parameters of the fully connected neural network and the feature vectors of the multi-resolution hash grid through the backpropagation algorithm until the total loss function is lower than the preset convergence threshold. This completes the multi-view consistency verification and outputs the converged multi-resolution hash grid features and fully connected neural network parameters as implicit representation data of the scene.

[0029] The system calculates the spatiotemporal sequence data of fluid and particle dynamic elements based on the boundary conditions of the scene's implicit representation data and using the lattice Boltzmann simulation algorithm. The specific execution process is as follows: Boundary mesh information for fluid and particle motion is extracted from the scene's implicit representation data. The spatial density distribution of the scene is extracted from the implicit representation data, and solid boundary regions of the scene are determined based on spatial density thresholds. These solid boundary regions are then meshed to generate boundary meshes that match the resolution of the fluid simulation mesh. The normal vector and solid properties of each boundary mesh are labeled to obtain the boundary mesh information for fluid and particle motion.

[0030] Construct a D3Q19 discrete velocity model and initialize the distribution function. The D3Q19 discrete velocity model contains 19 discrete velocity directions and a set of discrete velocities. for:

[0031] in, =0,1,...,18, where c is the lattice speed of sound. = , For the grid step size, This is the simulation time step.

[0032] Initialize distribution function ,in For grid node coordinates, For simulation time, the initial time. When = 0, the distribution function is calculated based on the initial macroscopic physical quantities through the equilibrium distribution function. The expression is:

[0033] in, In the D3Q19 model, the weighting coefficients corresponding to the discrete velocity directions are... =0 = , =1-6 hours = , =7-18 hours = ; For the speed of sound in a grid, = ; For the macroscopic density of the fluid, This refers to the macroscopic velocity of the fluid.

[0034] Macroscopic physical quantities of fluid micro-elements are calculated iteratively using collision and migration operators. The collision operator employs a multi-relaxation time model to ensure numerical stability. Within each simulation time step, the collision step is executed first, followed by the migration step.

[0035] In the collision step, the expression for the multi-relaxation time collision operator is:

[0036] in, The moment transformation matrix is ​​the distribution function. Moment vectors transformed into moment space ; This is a diagonal relaxation matrix, where the diagonal elements are the reciprocals of the relaxation times corresponding to each moment; The equilibrium moment vector is determined by the macroscopic density. With macro speed Calculated.

[0037] The calculation method for macroscopic physical quantities is as follows:

[0038]

[0039] In the migration step, the post-collision distribution function is migrated to adjacent grid nodes according to the discrete velocity direction. The expression for the migration process is:

[0040] Based on the boundary mesh information, a bounce boundary condition is applied, and the fluid velocity field and particle trajectory data that satisfy mass and momentum conservation are output as spatiotemporal sequence data. For mesh nodes located on solid boundaries, a half-range bounce boundary condition is used to bounce the post-collision distribution function in the opposite direction of the discrete velocity, ensuring the no-slip condition of the fluid at the solid boundary. The collision step and migration step are executed iteratively until the preset simulation duration is reached, and the fluid velocity field at each simulation moment is output. density field The data, along with particle trajectory data calculated using the Lagrange particle tracking algorithm based on the velocity field, are arranged in a time series to obtain spatiotemporal sequence data.

[0041] The system inputs implicit scene representation data and spatiotemporal sequence data into a multi-scale semantically aligned VisionTransformer generative network, performs pixel-level generation, and outputs a two-dimensional game screen feature map. The specific execution process is as follows: The implicit scene representation data is encoded as a first feature vector, and the spatiotemporal sequence data is encoded as a second feature vector. The multi-resolution hash grid features in the implicit scene representation data are encoded using a 1×1 convolutional layer and a global average pooling layer to obtain a first feature vector of dimension D. Fluid velocity field and particle trajectory data in spatiotemporal sequence data are encoded using a three-dimensional convolutional neural network and a temporal coding layer to obtain a dimension of... The second eigenvector The timing coding layer uses sinusoidal position coding, and the expression for position coding is:

[0042]

[0043] in, For time step index, Indexed by feature dimensions.

[0044] A hierarchical Transformer encoder is constructed, which fuses the first and second feature vectors using a self-attention mechanism. The hierarchical Transformer encoder comprises four encoding stages, each containing two Transformer blocks. Each Transformer block includes a multi-head self-attention layer and a feedforward neural network layer. The first feature vector F1 and the second feature vector F2 are concatenated and input into the Transformer encoder along with a learnable classification token. The attention weights of the multi-head self-attention layer are calculated using the following expression:

[0045] in, For querying the matrix, The key matrix, For value matrices, Dimensions for each attention head.

[0046] Through layer-by-layer feature extraction and fusion using the Transformer encoder, a fused feature map is output. , where H and W are the height and width of the feature map.

[0047] The semantic similarity between fused features and text description instructions is calculated through a cross-modal semantic alignment module to optimize the semantic representation of the feature map. This module includes a pre-trained text encoder that encodes the text description instructions in the scene generation instructions into text feature vectors. ; Calculate the fusion feature map Feature vectors of each spatial location and text feature vectors The cosine similarity is used to obtain the semantic similarity matrix. The expression for calculating cosine similarity is:

[0048] in, For spatial location indexing of feature maps; for semantic similarity matrix With fusion feature map Element-wise multiplication is performed to obtain the semantically optimized feature map. This achieves semantic alignment between fusion features and text description instructions.

[0049] The decoder upsamples the optimized feature map into a 2D game image and introduces a frequency domain loss function to constrain the generation of high-frequency texture details. The decoder includes four upsampling blocks, each containing a transposed convolutional layer, a batch normalization layer, and a GELU activation function layer. Through step-by-step upsampling, the semantically optimized feature map is upsampled... The game screen is restored to the target resolution in 2D; simultaneously, a 2D discrete Fourier transform is performed on the generated 2D game screen and the reference screen to convert them to the frequency domain, and the frequency domain loss function is calculated. The expression for the frequency domain loss function is:

[0050] in, For the generated 2D game screen, For reference, For two-dimensional fast Fourier transform, The frequency domain loss function is combined with the pixel space L1 loss function, and the parameters of the Vision Transformer generator network are optimized through backpropagation to constrain the generation of high-frequency texture details. Finally, a two-dimensional game screen feature map is output, which includes RGB color features, depth features, normal features, and lighting features.

[0051] The system performs depth buffering, lightmap mapping, and rendering queue adaptation mapping on the feature map of the 2D game screen, and embeds the generated screen into the game engine's real-time rendering process. The specific execution process is as follows: The game engine's geometry buffer data format requirements are analyzed. The geometry buffer configuration information for the current rendering pipeline is obtained through the game engine's SDK interface, including the number of buffers, the pixel format of each buffer, data channel definitions, and memory layout. The geometry buffers at least include a depth buffer, a normal buffer, a diffuse color buffer, a specular color buffer, and a lighting information buffer.

[0052] Depth and normal information are extracted from the 2D game screen feature map to generate depth buffer data and normal maps adapted to the game engine. Normalized depth values ​​are extracted from the depth feature channels of the 2D game screen feature map and mapped to the non-linear depth range corresponding to the game engine's depth buffer, converting them into depth buffer data conforming to the game engine's data format. Normal vectors are extracted from the normal feature channels of the 2D game screen feature map, transformed from world space to the tangent space required by the game engine, and normalized and encoded to generate a normal map whose format matches the game engine's normal buffer format.

[0053] Based on the lighting probe information in the scene's implicit representation data, light maps and shadow maps are generated. The lighting probe positions, spherical harmonic function coefficients, light intensity, and color information of the scene are extracted from the scene's implicit representation data. Based on the lighting probe information, lighting baking is performed on the static surfaces of the scene to generate a light map containing indirect lighting information. The UV coordinates of the light map are matched with the UV coordinates of the base 3D assets. Using a shadow mapping algorithm, depth maps corresponding to light sources in the scene are calculated to generate shadow maps adapted to the game engine. These shadow maps include one or more of the following: parallel light shadow maps, point light shadow maps, and spotlight shadow maps.

[0054] A rendering priority queue is established, and depth buffer data, normal maps, lightmaps, and shadow maps are packaged into rendering call instructions and injected into the game engine's deferred rendering pipeline. Rendering priorities are set according to the execution order of the game engine's rendering pipeline, with depth buffer data having the highest priority, followed by normal maps, lightmaps, and shadow maps. A rendering queue is established according to the rendering priority, and the above data, along with the corresponding shader parameters and texture sampling states, are packaged into rendering call instructions. Through the game engine's rendering plugin interface, the rendering call instructions are injected between the geometry and lighting stages of the game engine's deferred rendering pipeline. This allows the game engine to directly call the buffer data and texture data generated by this system to perform lighting calculations and pixel shading after completing the rasterization of the scene geometry data, achieving seamless integration of the generated image with the game engine's real-time rendering process.

[0055] As a preferred embodiment, please refer to the appendix. Figure 6The first part focuses on optimizing the construction of implicit scene representations. An adaptive tensor decomposition strategy is introduced during the construction of the sparse scene representation. The specific execution process is as follows: A normalized multinomial decomposition is performed on the feature tensors in a multi-resolution hash grid, decomposing the high-dimensional feature tensors into the product of a low-dimensional core tensor and a factor matrix, thus compressing storage space. The feature tensors in the multi-resolution hash grid are... ,in , , The three dimensions of the grid are... The feature dimension is used; the feature tensor is decomposed using normalized polynomial decomposition. Decomposed into The sum of rank-1 tensors, decomposed as follows:

[0056] in, It is the outer product operator. , , , For the factor vector of the corresponding dimension, Let be the rank of the tensor; optimize each factor vector using alternating least squares to ensure that the reconstruction error between the decomposed tensor and the original tensor is below a preset threshold; store the decomposed core tensor and factor matrix to replace the original high-dimensional feature tensor.

[0057] A Bayesian optimization algorithm is introduced to evaluate the contribution of the current viewpoint sampling to the accuracy of implicit representation, and the number of sampling rays is dynamically adjusted. A Gaussian process model is constructed, taking viewpoint parameters as input and the reconstruction error between the synthesized image and the real image at that viewpoint as output, to fit the mapping relationship between viewpoint parameters and reconstruction error. The contribution of each candidate viewpoint to reducing the global error of implicit representation is evaluated through a Bayesian-optimized acquisition function. For views with a contribution higher than a preset threshold, the number of sampling rays is increased; for views with a contribution lower than the preset threshold, the number of sampling rays is decreased. This achieves dynamic allocation of sampling resources, reducing the computational load of sampling while ensuring the accuracy of implicit representation.

[0058] When the convergence error of the scene implicit representation data is lower than a preset threshold, the parameter updates of the fully connected neural network are automatically frozen, and redundant intermediate feature caches are released. The total loss function value during the multi-view consistency verification process is monitored in real time. When the total loss function value is lower than the preset convergence threshold for 10 consecutive iterations, the scene implicit representation is determined to have converged. The weight parameters and bias parameters of the fully connected neural network are frozen, and backpropagation updates are stopped. At the same time, redundant GPU resources such as intermediate feature maps, gradient data, and temporary sampled data cached during training are released, and only the GPU space required for the converged scene implicit representation data is retained.

[0059] The second part focuses on the optimization of fluid simulation calculations. In the process of iteratively calculating the macroscopic physical quantities of fluid micro-elements using collision and migration operators, a surrogate model acceleration technique is integrated. The specific execution process is as follows: A fluid dynamics surrogate model based on a graph neural network is constructed. Taking the fluid velocity field at the current moment as input, it predicts a coarse solution for macroscopic physical quantities at the next moment. The graph neural network adopts a graph convolutional network architecture, treating each grid node of the fluid simulation as a vertex of the graph, and the connections between adjacent grid nodes as edges, thus constructing an undirected graph corresponding to the fluid simulation grid. The input to the graph neural network is the current moment... fluid velocity field density field The output is the next time step. fluid velocity field density field As a rough solution for macroscopic physical quantities, the graph neural network is trained using a pre-simulated fluid dynamics dataset. The loss function during training is the MSE loss between the predicted value and the actual simulated value. After training, a converged fluid dynamics surrogate model is obtained.

[0060] Using a coarse solution as the initial guess for the iterative calculation of the lattice Boltzmann simulation algorithm reduces the number of iterations required to reach steady state. Within each simulation time step, a coarse solution for the macroscopic physical quantities at the next moment is first predicted using a fluid dynamics surrogate model. Based on the coarse solution, an initial distribution function is calculated and used as the initial value for the collision and migration iterations of the lattice Boltzmann simulation algorithm, reducing the number of iterations and improving the efficiency of the simulation calculation.

[0061] A reinforcement learning agent is used to adjust the relaxation parameters in a multi-relaxation time model in real time based on the fluid Reynolds number, optimizing the simulation time step while ensuring physical conservation laws. The reinforcement learning agent employs a deep Q-network architecture. The agent's state space includes the current fluid Reynolds number Re, the simulation residual, and the number of iterations. Its action space consists of the adjustment amounts of the diagonal elements of the relaxation matrix S in the multi-relaxation time model and the simulation time step. The adjustment amount; the reward function is set as follows: when the simulation result satisfies the laws of conservation of mass and momentum and the residual is lower than the preset threshold, a positive reward is given, and the reward value is positively correlated with the improvement of simulation efficiency; when the simulation result does not satisfy the laws of physical conservation, a negative reward is given; through reinforcement learning training, the agent can adjust the relaxation parameters and time step in real time according to the fluid Reynolds number, so as to maximize the time step of simulation calculation and reduce the total number of iterations while ensuring numerical stability and physical conservation.

[0062] The third part concerns the optimization of the image generation network. The cross-modal semantic alignment module integrates style transfer and knowledge distillation mechanisms, and the specific execution process is as follows: A pre-trained art style knowledge base is loaded, and an adaptive instance normalization algorithm is used to transfer the texture style of the 2D game screen to the target art style. The pre-trained art style knowledge base contains style feature encoders for various game art styles; the expression for the adaptive instance normalization algorithm is:

[0063] in, This is a content feature map of the generated 2D game screen. A stylistic feature diagram for the target art style. The mean of the feature map. The standard deviation of the feature map; through The algorithm aligns the mean and standard deviation of the target art style to the content feature map, so that the generated 2D game screen conforms to the requirements of the target art style.

[0064] A teacher-student network architecture is constructed, where the teacher network is a high-parameter visual generation model, and the student network is a lightweight deployment model. The teacher network adopts the multi-scale semantic alignment Vision Transformer generation network described in this invention, which has high-fidelity image generation capabilities; the student network is a lightweight Vision Transformer network, which achieves lightweighting by reducing the number of Transformer blocks, reducing feature dimensions, and using depthwise separable convolutions instead of standard convolutions, thus adapting to the computing power limitations of edge devices.

[0065] By minimizing the KL divergence between the teacher and student networks in the intermediate layer feature maps, high-fidelity generation capabilities are transferred to lightweight networks. During knowledge distillation training, the same input data is fed into both the teacher and student networks. Feature maps from the intermediate layers of both networks are extracted, and the KL divergence loss function L_KL is calculated. The expression for KL divergence is:

[0066] in, This represents the probability distribution of the feature maps in the intermediate layer of the teacher network. The probability distribution of the feature maps of the intermediate layers of the student network; combined with pixel space. Loss function, total distillation loss function ,in The weight coefficients are used to iteratively optimize the parameters of the student network through backpropagation, so that the feature distribution and generation effect of the student network are close to those of the teacher network, and the high-fidelity generation capability is transferred to the lightweight student network.

[0067] Part Four: Optimization of the rendering adaptation module. Frame prediction and resource preloading are performed before the deferred rendering pipeline injected into the game engine. The specific execution process is as follows: Based on the game screen feature maps of historical frames, a Long Short-Term Memory (LSTM) network is used to predict the change trend of depth buffer data in the next frame. The LSM network takes K consecutive historical frames of depth buffer data as input and outputs the predicted value of the depth buffer data for the next frame. The LSM network controls the transmission of temporal information through input gates, forget gates, and output gates, learns the temporal change pattern of depth data in historical frames, and predicts the change trend of depth buffer data in the next frame, including the region and magnitude of depth value changes.

[0068] Based on the prediction results, memory bandwidth is pre-allocated to load the expected updated lightmap data into the cache. Based on the changing trend of the depth buffer data in the next frame, the changes in the camera's perspective and the movement range of dynamic elements in the scene are determined to identify the areas of the lightmap that need to be updated in the next frame. Memory bandwidth is pre-allocated, and during the rendering interval of the current frame, the expected updated lightmap data is loaded from non-volatile storage media or system memory into the GPU's cache to avoid memory bandwidth bottlenecks during the rendering of the next frame.

[0069] A double-buffering mechanism is used to assemble rendering call instructions in the background. When the game engine's main thread rendering logic is triggered, the buffer pointers are directly swapped, eliminating the delay of the rendering pipeline waiting for the generation result. A front buffer and a back buffer are set up. The front buffer stores the rendering call instructions being invoked by the game engine in the current frame, and the back buffer is used to assemble the rendering call instructions for the next frame. During the rendering of the current frame, the system generates the depth buffer data, normal map, light map, and shadow map for the next frame in a background thread, assembles them into rendering call instructions, and stores them in the back buffer. When the game engine's main thread completes the rendering of the current frame and triggers the rendering logic for the next frame, the pointers of the front buffer and the back buffer are directly swapped, allowing the game engine to directly read the assembled rendering call instructions without waiting for the generation process, thus eliminating the waiting delay of the rendering pipeline.

[0070] As a preferred embodiment, please refer to the appendix. Figure 6 In the computation process of the surrogate model acceleration technology and the lattice Boltzmann simulation algorithm, a physical constraint layer is embedded to calculate the residuals of the fluid continuity equation and the Navier-Stokes equation. This physical constraint layer is embedded at the output of the graph neural network surrogate model and, during the iteration process of the lattice Boltzmann simulation algorithm, calculates the conservation residuals of fluid physical quantities in real time.

[0071] The expression for the fluid continuity equation is:

[0072] The Navier-Stokes equations are expressed as follows:

[0073] in, This refers to fluid pressure. For fluid dynamic viscosity, This is the acceleration due to gravity.

[0074] For discrete simulation mesh nodes, the residuals of the above equations are calculated using the finite difference method, and the residuals of the continuity equation are also calculated. for:

[0075] Residuals of the Navier-Stokes equations for:

[0076] The L2 norm of the residuals of all grid nodes is used as the total physical residual. .

[0077] The residuals are backpropagated to a multi-scale semantically aligned Vision Transformer generative network to correct the pixel distribution of the generated image. The total physical residuals are then... As a physical constraint loss function It is added to the total loss function of the VisionTransformer generator network, and the total loss function is... ,in The weight coefficients are used; the gradient of the physical residual is passed to the parameters of each layer of the Vision Transformer generation network through the backpropagation algorithm, the network parameters are iteratively optimized, and the pixel distribution of the fluid and particle dynamic elements in the generated image is corrected so that the dynamic elements in the generated image conform to the laws of fluid physics.

[0078] If the corrected physical error still exceeds the preset tolerance range, a rollback mechanism is triggered to reinitialize boundary conditions and iteratively calculate until a game image that meets the physical realism index is generated. The preset physical error tolerance threshold is... When the corrected total physical residual If the generated image fails to meet the physical realism requirements, a rollback mechanism is triggered. During the rollback, boundary mesh information is re-extracted from the scene's implicit representation data, the distribution function and boundary conditions of the lattice Boltzmann simulation algorithm are initialized, the simulation iteration step number is increased, the time step is decreased, and the fluid simulation calculation and image generation process are re-executed. This closed-loop verification process is repeated until the total physical residual is found. ≤ Output game graphics that meet physical realism standards.

[0079] The module division in all the above embodiments of the present invention is only a logical function division. In actual implementation, it can be merged or split according to hardware architecture, computing power configuration and application scenario. All the above modifications are within the protection scope of the present invention.

Claims

1. A game image AI intelligent generation and optimization processing system, characterized in that, include: Obtain basic 3D assets and scene generation instructions for the game scene; The neural radiation field technology is used to construct implicit representations and verify multi-view consistency of basic 3D assets, and output scene implicit representation data. Based on the boundary conditions of the scene's implicit representation data, the lattice Boltzmann simulation algorithm is used to calculate the spatiotemporal sequence data of fluid and particle dynamic elements; The implicit representation data of the scene and the spatiotemporal sequence data are input into the Vision Transformer generative network with multi-scale semantic alignment, which performs pixel-level generation and outputs a two-dimensional game screen feature map. The feature map of the 2D game screen is adapted and mapped by depth buffering, light mapping and rendering queue, and the generated screen is embedded into the game engine's real-time rendering process.

2. The system according to claim 1, characterized in that, The implicit representation construction and multi-view consistency verification of the basic 3D assets using neural radiation field technology includes: The basic 3D assets are voxelized to extract geometric feature point clouds; Multi-resolution hash grid coding technology is used to perform feature mapping on geometric feature point clouds to construct sparse scene representations; The spatial density and color values ​​of sparse scene representations are predicted by a fully connected neural network, multi-view images are generated using a volume rendering equation, and multi-view consistency verification is completed by calculating the structural similarity loss function between the synthetic image and the real view image.

3. The system according to claim 1, characterized in that, The calculation of spatiotemporal sequence data of fluid and particle dynamic elements using the lattice Boltzmann simulation algorithm includes: Extract boundary mesh information of fluid and particle motion from scene implicit representation data; Construct a D3Q19 discrete velocity model and initialize the distribution function; The macroscopic physical quantities of fluid micro-elements are calculated iteratively using collision and migration operators, with the collision operator employing a multi-relaxation time model to ensure numerical stability. Apply bounce boundary conditions based on the boundary grid information, and output fluid velocity field and particle trajectory data that satisfy mass and momentum conservation as spatiotemporal sequence data.

4. The system according to claim 1, characterized in that, The Vision Transformer generative network, which inputs implicit scene representation data and spatiotemporal sequence data into a multi-scale semantically aligned network, performs pixel-level generation and outputs a two-dimensional game screen feature map, including: The implicit representation data of the scene is encoded into a first feature vector, and the spatiotemporal sequence data is encoded into a second feature vector; Construct a hierarchical Transformer encoder and use a self-attention mechanism to fuse the first feature vector and the second feature vector; The semantic similarity between fused features and text description instructions is calculated through a cross-modal semantic alignment module to optimize the semantic representation of the feature map; The optimized feature map is upsampled into a two-dimensional game screen using a decoder, and a frequency domain loss function is introduced to constrain the generation of high-frequency texture details.

5. The system according to claim 1, characterized in that, The process of performing depth buffering, light mapping, and rendering queue adaptation mapping on the feature map of the 2D game screen includes: Parse the geometry buffer data format requirements of the game engine; Depth and normal information are extracted from the feature map of the 2D game screen to generate depth buffer data and normal map adapted to the game engine. Based on the lighting probe information in the scene's implicit representation data, generate light maps and shadow maps; Establish a rendering priority queue, package depth buffer data, normal maps, light maps, and shadow maps into rendering call instructions, and inject them into the game engine's deferred rendering pipeline.

6. The system according to claim 2, characterized in that, An adaptive tensor decomposition strategy is introduced during the construction of the sparse scene representation: The feature tensors in the multi-resolution hash grid are decomposed into normalized polynomials, which decompose the high-dimensional feature tensors into the product of the low-dimensional core tensor and the factor matrix, thus compressing the storage space. A Bayesian optimization algorithm is introduced to evaluate the contribution of the current view sampling to the accuracy of the implicit representation, and the number of sampling rays is dynamically adjusted. When the convergence error of the scene's implicit representation data is lower than a preset threshold, the parameter updates of the fully connected neural network are automatically frozen, and redundant intermediate feature caches are released.

7. The system according to claim 3, characterized in that, The technique of integrating a surrogate model to accelerate the iterative calculation of macroscopic physical quantities of fluid micro-elements using collision and migration operators is described below. A fluid dynamics surrogate model based on graph neural networks is constructed, which uses the fluid velocity field at the current moment as input to predict the rough solution of macroscopic physical quantities at the next moment. Using the coarse solution as the initial guess value for the iterative calculation of the lattice Boltzmann simulation algorithm reduces the number of iterations required to reach steady state; By using a reinforcement learning agent to adjust the relaxation parameters in a multi-relaxation time model in real time based on the fluid Reynolds number, the time step of the simulation calculation is optimized while ensuring the physical conservation laws.

8. The system according to claim 4, characterized in that, The cross-modal semantic alignment module integrates style transfer and knowledge distillation mechanisms: Load a pre-trained art style knowledge base and transfer the texture style of the 2D game screen to the target art style through an adaptive instance normalization algorithm; Construct a teacher-student network architecture, where the teacher network is a high-parameter visual generation model and the student network is a lightweight deployment model; By minimizing the KL divergence between the teacher and student networks in the intermediate layer feature maps, high-fidelity generation capabilities are transferred to lightweight networks.

9. The system according to claim 5, characterized in that, The frame prediction and resource preloading are performed before the deferred rendering pipeline is injected into the game engine. Based on the game screen feature map of historical frames, a long short-term memory network is used to predict the trend of depth buffer data change in the next frame. Based on the prediction results, the video memory bandwidth is scheduled in advance, and the expected updated light map data is loaded into the cache. The rendering call instructions are assembled in the background using a double buffering mechanism. When the main thread rendering logic of the game engine is triggered, the buffer pointers are directly swapped.

10. The system according to claim 7, characterized in that, A physical constraint loss function is introduced to perform closed-loop verification of the generated results: In the computation process of surrogate model acceleration technology and lattice Boltzmann simulation algorithm, a physical law constraint layer is embedded to calculate the residuals of fluid continuity equation and Navier-Stokes equation; The residuals are backpropagated to the Vision Transformer generative network with multi-scale semantic alignment to correct the pixel distribution of the generated image. If the corrected physical error still exceeds the preset tolerance range, a rollback mechanism is triggered to reinitialize the boundary conditions and iteratively calculate until a game image that meets the physical realism index is generated.

Images