730 results about "Real-time rendering" patented technology

The invention provides a game cartoon character display method based on a virtual reality technology. The method comprises the following steps: acquiring three-dimensional model data of a game cartoon character in a target display area through a user interaction terminal; the virtual reality content management platform determines a dynamic rendering precision level according to the model data, and generates a real-time rendering strategy including a model patch reduction coefficient, a texture compression rate and a skeleton animation updating frequency in combination with terminal performance parameters; the strategy is sent to a virtual reality supervision platform and a user interaction terminal, and a virtual reality rendering engine platform is instructed to execute real-time rendering; and the virtual reality supervision platform monitors the frame rate fluctuation data of the head-mounted display device, calculates the scene rendering stability, sends a rendering optimization instruction if the scene rendering stability is lower than a threshold value, and adjusts the model data acquisition frequency and the video memory cleaning period to optimize the performance. The rendering efficiency and the system stability can be improved, and the hardware load and the frame rate fluctuation are reduced.

The invention provides a virtual human design and application platform, method and device based on artificial intelligence, and relates to the technical field of virtual digital humans. The method comprises the steps of performing local anonymization on multi-modal input data on user equipment, encoding generated anonymized multi-modal features to obtain a multi-modal feature vector, and inputting the multi-modal feature vector into an emotion calculation model to obtain a user emotion intensity quantized value; inputting the multi-modal feature vector into a context sensing model, and generating a user intention vector after context correction in combination with a knowledge graph; generating an updated personality parameter matrix according to the user emotion intensity quantized value and the user intention vector; and outputting voice waveform data, facial muscle motion parameters and skeleton joint coordinate data based on the personality parameter matrix, and driving the virtual digital human three-dimensional model to perform real-time rendering. According to the scheme, the naturalness, emotional resonance and long-term user retention rate of virtual digital human interaction can be improved, and user privacy data security is protected.

The invention discloses a three-dimensional scene reconstruction method and device based on large model geometric prior, and a medium, and aims to solve the problems that a conventional 3DGS is liable to have artifacts and detail loss in geometric discontinuity, data redundancy and illumination variation scenes, and predicts a dense depth map and a normal map from a monocular image by using a pre-trained large model. The position and form of the Gaussian kernel are constrained as additional geometric priori; a primitive adjustment strategy based on kernel density estimation is introduced in the training stage, small Gaussian primitives with similar structures and adjacent spaces are combined into a large Gaussian primitive, the rendering quality is kept, redundancy is reduced, and the volume of the model is reduced; an exposure coefficient is adaptively estimated for each input image, an exposure compensation image loss function is constructed, and floating artifacts caused by illumination differences at shooting moments are eliminated. Experiments show that compared with the prior art, the method improves the three-dimensional reconstruction precision and real-time rendering quality of complex illumination and less-texture areas in a public data set and an unmanned aerial vehicle aerial photography scene.

The invention provides a game scene optimization method and system based on dynamic rendering, and belongs to the technical field of game development.The game scene optimization method comprises the steps that firstly, a real-time rendering state data set of a current game scene, rendering resource occupation information containing scene elements and picture frame rendering time consumption parameters are obtained; then, a pre-trained rendering demand analysis model is called for analysis, a rendering demand feature set of the scene elements is obtained, the rendering demand feature set comprises element visual importance features and rendering resource sensitivity features, and a scene element dynamic optimization strategy is generated based on the rendering demand feature set; the scene element dynamic optimization strategy comprises an element detail level adjustment rule and a rendering resource allocation priority parameter, executing a rendering parameter adjustment operation on a target scene element according to the scene element dynamic optimization strategy, generating adjusted scene rendering configuration data, and finally inputting the scene rendering configuration data into a game rendering pipeline for real-time rendering processing. And outputting the optimized game scene picture frame sequence. Therefore, the game picture quality and the operation performance can be improved.

The invention discloses a high-resolution three-dimensional reconstruction method of a fusion diffusion model, which belongs to the technical field of image data processing, and comprises the following steps: constructing an original data set D; constructing an enhanced training set; constructing a three-dimensional reconstruction network which comprises a text encoder, a renderer, a VAE encoder, a conditional diffusion model, a VAE decoder and an MVS module; training and fine-tuning the conditional diffusion model in three stages to obtain a three-dimensional reconstruction model, acquiring an image sequence and a text instruction of a scene to be reconstructed, and performing reconstruction by using the three-dimensional reconstruction model. According to the method, highly consistent geometric and color reduction can be kept under the multi-view condition, and splicing artifacts are remarkably reduced. Through semantic guidance optimization, texture details and structural consistency of the reconstruction model are greatly improved. Conditional diffusion sampling enables the model to accurately restore local details in a complex scene, and the stability of real-time rendering is improved.

The invention provides a dynamic scene reconstruction method and system based on spatial decomposition and Gaussian splashing, and belongs to the technical field of computer vision and graphics. The method comprises the following steps: decomposing a dynamic scene into a static standard space and a dynamic playground; reconstructing the standard space by using three-dimensional Gaussian splashing to obtain Gaussian primitives of the standard space; decomposing the characteristics of the motion field into standard space characteristics and a plurality of motion subspace characteristics based on tensor decomposition; fusing the standard space feature and the multiple motion subspace features through a motion information decoder, and decoding motion information of Gaussian primitives in the standard space; and applying the motion information to Gaussian primitives of a standard space, generating any moment representation of a dynamic scene, and rendering a target view angle image through a snowball throwing method to realize reconstruction of the dynamic scene. Through the method, the reconstruction precision and the rendering quality are effectively improved, and the real-time rendering capability is kept.

The invention provides a digital human generation method based on a multi-modal large model. The method comprises the following steps: constructing a digital human basic model; generating a structured training set; generating a question and answer model supporting multi-channel interaction; semantic answers of the user questions are output, text emotional tendencies of the semantic answers are extracted, and emotional intensity parameters are output; generating facial muscle movement track data, and performing real-time rendering on the digital human basic model according to the facial muscle movement track data to output a digital human three-dimensional image with emotion expression. According to the embodiment of the invention, cross-modal alignment is carried out on text, image and audio data, and a multi-modal large model containing visual, voice and knowledge models is optimized by using a joint training method, so that more natural and smoother multi-channel interaction experience is realized; in addition, by introducing an emotion recognition model and a face interaction model, the emotion tendency contained in the semantic answer can be captured and reflected more accurately, so that a digital human three-dimensional image with real emotion expression is output.

The invention belongs to the technical field of three-dimensional scene interaction, and particularly relates to a three-dimensional immersive content interaction method and system based on 3D Gaussian spattering. The method comprises the following steps: collecting a multi-view image and carrying out motion structure recovery to obtain a camera pose matrix; according to the camera pose matrix, performing three-dimensional reconstruction processing on the image data through a 3D Gaussian spattering algorithm to obtain a reconstruction model; performing behavior analysis on the historical viewport track of the user to obtain a visual angle prediction result; and according to the visual angle prediction result and the reconstruction result of the reconstruction model, video frame generation is carried out at a client through a real-time rendering engine, and a transmission result is obtained. According to the invention, the application of the 3DGS in the three-dimensional expression technology is realized, the rendering quality is improved, and the degree of participation and immersion of the user are improved.

The invention discloses rendering equipment based on three-dimensional Gaussian sputtering. The hardware architecture includes a memory interface, a pre-processing engine, a cardinality ordering engine, a rasterization engine, and the like. The preprocessing engine converts the three-dimensional Gaussian point into a two-dimensional representation and generates a key value pair containing a tile identifier and depth information; the cardinal number sorting engine performs staged efficient sorting through a parallel processing mechanism and a BRAM alternate working mode; and the rasterization engine performs rendering processing by adopting a full-pipeline design and an early-stage stopping mechanism. According to the method, parallel execution of depth sequencing and rasterization is achieved, memory access and calculation redundancy are reduced through optimized data flow design, an efficient and low-power-consumption hardware acceleration solution is provided for three-dimensional Gaussian sputtering rendering, and the method is particularly suitable for application scenes such as virtual reality, games and scientific visualization requiring real-time rendering.

The invention belongs to the technical field of augmented reality (XR) and computer vision crossing, relates to fusion application in immersive digital performance, and provides a virtual actor reconstruction and immersive presentation system based on 4D Gaussian splash modeling and XR space positioning. The system comprises a set of spherical multi-camera-position high-synchronization camera shooting matrix used for capturing dynamic images of actors; performing dynamic modeling on the multi-angle image through a 4D Gaussian splashing technology, and outputting a virtual actor point cloud model which can be used by XR equipment; vPS visual positioning and an SLAM tracking module are combined, and precise mapping positioning of a performance space is achieved in AR / MR equipment. The method supports the immersive watching of the actor image at the audience end at a 360-degree free visual angle, and realizes the natural presentation of the virtual actor without dead angles and wearing in cooperation with shielding judgment and a real-time rendering engine. The system is widely applicable to on-site entertainment scenes such as immersive theaters, text travel performances, brand activities, concerts and television programs.

The invention discloses a dynamic three-dimensional scene reconstruction and real-time rendering method based on free space-time Gaussian sputtering, and the method comprises the steps: firstly obtaining a multi-view-angle dynamic scene video, extracting cross-view-angle feature points for three-dimensional reconstruction, initializing free space-time Gaussian primitives, and carrying out the real-time rendering of the free space-time Gaussian primitives; an optimizable explicit motion function and a time opacity function are set for each Gaussian primitive and are used for representing the geometry and appearance of the dynamic three-dimensional scene; then sputtering and rendering the Gaussian primitive at the current moment based on the observation visual angle, and outputting a high-fidelity dynamic scene image of the corresponding visual angle; finally, Gaussian primitive parameters are optimized in a combined mode through a rendering loss function and a four-dimensional regularization strategy, and meanwhile low-influence Gaussian primitives are relocated periodically. According to the method, the movable Gaussian primitive is introduced at any position in space and time, the explicit motion function and the time opacity function are combined, a complex dynamic scene is effectively modeled, and efficient reconstruction and real-time rendering of the dynamic three-dimensional scene are achieved.

The invention discloses a multi-dimensional holographic image real-time rendering method and system based on dynamic light field modulation, and relates to the technical field of holographic display, the method is applied to terminal equipment and a server, the terminal equipment determines a space coordinate range based on quantum dot super-structure surface characteristics by obtaining related parameters of a target holographic scene, and the real-time rendering of a holographic image is realized. A quantum dot excitation sequence is generated to regulate and control the micro-nano structure of the super-structure surface, a joint coding model is constructed, a target light field modulation instruction set is generated through optimization, corresponding multi-dimensional light field data is requested from a server, and a holographic scene is generated through processing, fusion and rendering; the server receives the request, calls original holographic data, constructs a coding model and a compression model, and sends multi-dimensional light field data subjected to entropy compression to the terminal through a multi-channel transmission protocol; according to the method, efficient real-time rendering of the multi-dimensional holographic image is achieved, the rendering efficiency and quality are improved, and the method is suitable for scenes with high requirements for real-time performance of the holographic image.

The invention relates to the technical field of real-time rendering, cloud edge collaboration and three-dimensional visualization, in particular to a multi-instance cloud edge collaboration rendering method based on an unreal engine. Through cloud centralized scheduling and edge node plug flow collaborative architecture, on-demand distribution, dynamic start and stop and life cycle management of rendering instances are realized, and the resource utilization efficiency is remarkably improved; the system combines a relay agent mechanism of TURN and Nginx, gets through access restrictions among a public network, a local area network and a plurality of subnetworks, and realizes unified access and high-adaptability pixel flow transmission capability. Through edge-side integrated pixel flow and signaling service, distributed instance management and load balancing scheduling, the concurrent processing capability and horizontal expansibility of the system are enhanced, and the requirements of complex three-dimensional application scenes such as multiple users, real-time interaction and high-performance rendering are met. Through a cloud edge separation architecture, a relay forwarding mechanism and instance dynamic scheduling, efficient and stable pushing of unreal engine pixel streams in a multi-network environment is realized.

The invention discloses a Gaussian model construction method and device, a program product and a storage medium, and relates to the technical field of computer vision. The method comprises the following steps: acquiring multi-view images, and obtaining a three-dimensional Gaussian element corresponding to each multi-view image; establishing a hierarchical data structure; selecting a training view angle to obtain a trained three-dimensional Gaussian element; calculating a stability score of each trained three-dimensional Gaussian element, and performing hierarchical construction operation according to each stability score to obtain a plurality of space clusters; calculating a representative parent cluster; performing rasterization processing to obtain a rendering result; calculating rendering loss, and adding the representative parent cluster into the hierarchical data structure to obtain a target hierarchical data structure; and repeatedly executing the training operation and the hierarchical construction operation until the rendering loss is lower than a preset loss threshold value, and outputting a three-dimensional Gaussian scene model containing the target hierarchical data structure. By implementing the technical scheme provided by the invention, the performance pressure faced by real-time rendering can be reduced.

The invention discloses a video stream concurrent access front-end dynamic scheduling control system and method based on a digital twinborn scene, and relates to the technical field of computer software and network communication, the system comprises three core modules: a self-adaptive request queue module manages UE rendering resources based on a session token pool, distributes tokens through preemptive scheduling, and sends the tokens to a server; in combination with a GPU frame rate feedback dynamic adjustment strategy, queuing through a Promise asynchronous queue when the request fails; the front-end fusing and backoff retry module monitors the connection failure rate, when the connection failure rate reaches a threshold value, fusing is triggered, retry is delayed by adopting an exponential backoff strategy, and a semi-open state tends to recover; and the local cache collaboration module intercepts the request through Service Worker, caches the latest video clip, pushes the local content when the request is delayed, and ensures the freshness through ETag verification. The method and the device are used for efficiently managing loading and interaction of multiple users on real-time rendering pictures at a browser end.

The invention relates to the technical field of geological analysis, and discloses a method for fusing a three-dimensional geological model and a WebGIS (Web Geographic Information System), aiming at solving the problems of poor data intercommunity, limited spatial analysis capability, difficulty in real-time updating and low model loading efficiency of the existing method, and the scheme mainly comprises the following steps: extracting geological model data, and carrying out standardized format processing; lightweight processing is carried out, and geological attributes and model nodes are bound one by one; the format is converted into a binary format embedded with a metadata label, and data integrity verification and automatic error correction are carried out in the conversion process; constructing a non-uniform adaptive octree spatial index, a multi-level attribute data architecture and a real-time updating mechanism of attribute data; after the geological model and the terrain are precisely registered and fused, a dynamic loading mechanism, a high-performance rendering mechanism and a geological model real-time updating mechanism are constructed, and a three-dimensional geological analysis tool is integrated. According to the method, efficient loading, real-time rendering, dynamic updating and spatial analysis of the three-dimensional geologic model in the WebGIS are realized.

The invention relates to the technical field of urban earthquake disaster simulation, and provides a visualization system and method for urban building group earthquake disaster simulation, and the system comprises an offline data preprocessing and block organization module and a block dynamic data management and scheduling module. The GPU acceleration rendering module is used for mapping a global floor index to a vertex to endow the vertex with a unique identifier, and finally submitting the combined model to a GPU, and cooperating with a displacement texture and a user-defined shader to realize real-time rendering of a building form; the GPU parallel displacement picture moving module is deeply coupled to the GPU rendering pipeline and is used for resolving the vertex displacement and the color value in real time and injecting the vertex displacement and the color value into the rendering pipeline; and the efficient interaction module dynamically updates label contents and coordinates to realize three-dimensional interaction. According to the method, massive time sequence displacement data are coded into textures, and parallel calculation is performed by using the GPU vertex shader, so that real-time and smooth animation simulation of earthquake responses of all buildings is realized, and the performance bottleneck of a traditional CPU calculation mode is solved.

The invention provides an AI streaming output rich text real-time rendering method and system based on minimum rendering unit recognition. The method comprises the steps that an AI model outputs a text stream to an AI agent; defining a minimum rendering unit (MRU), wherein the minimum rendering unit is self-consistent in format and complete; the AI agent analyzes the text streams in real time, the analysis process continuously judges whether the currently accumulated text streams form a complete MRU or not, and if yes, a rendering trigger mark is added for output; if not, directly outputting; the client receives the output result and the text stream and judges whether current rendering is triggered or not, and if yes, the rendering result is displayed; and if not, continuing to receive the output result until the rendering result is displayed after the rendering is triggered. According to the method, real real-time rendering can be realized, and a user can see a formatted result to appear on a screen step by step and smoothly while AI generates contents, so that response speed perception and interaction experience are greatly improved, and the pain point of blank screen waiting is solved.

The invention proposes a semantic SLAM method based on three-dimensional Gaussian sputtering, and relates to the field of three-dimensional Gaussian sputtering and SLAM, and the method comprises the steps: constructing a three-dimensional Gaussian distribution model fusing semantic features, representing a scene through a group of isotropic three-dimensional Gaussian distribution, obtaining the RGB-D data of the scene in real time through an RGB-D sensor, and carrying out the calculation of the RGB-D data. S2, generating a pixel-level semantic segmentation mask of the original data by using a pre-trained semantic segmentation network, projecting the three-dimensional Gaussian distribution model to a two-dimensional plane to generate a rendering data set, and obtaining a multi-modal data set by minimizing a joint loss function formed by RGB loss, depth loss and semantic loss between the rendering data set and the multi-modal data set in S2; and iteratively optimizing the sensor pose parameters and optimizable parameters of the three-dimensional Gaussian distribution model to finally obtain a complete sensor pose track and three-dimensional semantic Gaussian distribution map, so that the semantic consistency and real-time rendering efficiency of three-dimensional scene reconstruction are remarkably improved.

The invention relates to the technical field of computers, and discloses a real-time rendering and interaction method and system for an immersive virtual reality scene. The method comprises the following steps: fusing tuner inertial data and eyeball tracking data, and constructing a prospective state prediction model; generating a predictive focus field in combination with scene visual saliency; synthesizing an anisotropic temporal-spatial resolution graph according to the predicted head angular velocity; gPU variable-rate coloring is driven to realize non-uniform rendering; and re-projection or dynamic fuzzy correction is executed in a self-adaptive manner according to the attitude prediction error before display. According to the technical scheme, the perception delay and the rendering load are remarkably reduced, and the frame rate stability and the visual immersion in a high-dynamic scene are improved.

The invention discloses an augmented reality intelligent navigation and operation quantitative evaluation method and system in an operating room, and the method comprises the steps: collecting multispectral image data and depth point cloud data of an operation region in the operating room in real time, and generating a multidimensional perception data set; performing feature extraction and spatial registration on the multi-dimensional perception data set, and decomposing the dynamic three-dimensional augmented reality model into an anatomical structure feature layer and an instrument interaction feature layer; performing dynamic matching and real-time rendering on the anatomical structure feature layer and the instrument interaction feature layer based on a digital twinning technology to generate augmented reality navigation information and an operation guidance scheme; and performing multi-dimensional quantitative analysis on the surgical operation process according to the augmented reality navigation information and the operation guidance scheme, and outputting a quantitative evaluation result. By utilizing the embodiment of the invention, augmented reality navigation guidance which is synchronous with a real operation scene in real time can be provided, meanwhile, accurate quantitative evaluation is carried out on the operation operation, and the accuracy, the safety and the trainability of the operation are improved.

A method of generating physically-based rendering (PBR) textures is disclosed. A 3D mesh input is received. One or more text prompts associated with the 3D mesh input are received. The 3D mesh input and the one or more text prompts are in put into a trained machine-learning model. Output data is received from the trained machine-learning model. The output data comprises material and texture information. The material and texture information enables rendered views of the 3D mesh with the output data to respond to changes in lighting in a real-time 3D rendering environment.

The invention provides a mine safety training method and system based on digital twinning and virtual reality, and the method comprises the steps: obtaining laser scanning point cloud and geological exploration data of an underground roadway of a mine, and constructing a high-precision virtual underground environment model based on the data; on the basis of the virtual underground environment model, combined with explosion condition judgment, disaster simulation is triggered, and a virtual training environment is generated by applying real-time rendering and dynamic labeling technologies; obtaining operation behavior data of trainees in the virtual training environment, and evaluating the operation behavior data to obtain a behavior evaluation result; and based on the behavior evaluation result and the environmental parameter change data, constructing and training an evaluation model by utilizing machine learning, and generating customized training content in combination with geological conditions and mining process characteristics of different mining areas. According to the method, a high-reduction-degree virtual scene can be constructed, the problem that a traditional training scene is single is solved, quantitative evaluation of the training effect is achieved, and training pertinence and effectiveness are improved.

The invention discloses a volume cloud rendering method and system based on three-dimensional Gaussian splash, and the method comprises the steps: carrying out the feature matching and structure reconstruction of a volume cloud image through an SfM algorithm, obtaining a sparse point cloud in a scene, and representing the sparse point cloud as an anisotropic Gaussian ellipsoid; decomposing the illumination transmission process of the volume cloud into two parts of forward single scattering and internal multiple scattering based on a radiation transmission equation to obtain an illumination modeling formula embedded volume cloud rendering process; determining the contribution of each three-dimensional Gaussian ellipsoid to a rendering result according to the opacity index of the three-dimensional Gaussian ellipsoid, and dynamically cutting the low-contribution three-dimensional Gaussian ellipsoid by adopting a delay deletion strategy to reduce redundancy; a comprehensive loss function is constructed, so that the three-dimensional Gaussian model is focused on the volume cloud region and the structure detail reduction capability is enhanced; and outputting the optimized volume cloud rendering result and the three-dimensional Gaussian model. According to the method, the volume cloud rendering efficiency and quality are effectively improved on the premise of ensuring real-time rendering.

The invention discloses a real-time rendering optimization method based on dynamic rendering and multi-hardware acceleration cooperation, and relates to the technical field of AR and VR real-time rendering, and the method comprises the following specific steps: dividing a user field angle into three-level resolution regions through an eye movement tracking module, and dynamically adjusting the resolution and anti-aliasing strategy of each region; stage processing is realized through an asynchronous time warping hardware acceleration module; the asynchronous time warping hardware acceleration module is combined with the optical flow compensation module to cooperatively optimize anti-aliasing compensation, and dynamic key points are extracted through a sparse optical flow algorithm; a multi-hardware acceleration collaborative architecture is adopted, and GPU rendering pipelines are divided according to a multi-viewport parallel rendering module; and the motion prediction module synchronizes IMU data in real time through a special AXI bus, so that optimization model updating is realized, and a final rendering frame is obtained. Through cooperation of dynamic rendering and multi-hardware acceleration, the bottleneck of a traditional scheme in adaptability, energy efficiency and real-time performance is solved, and the rendering quality and cruising ability of AR / VR equipment are remarkably improved.

The invention discloses a dynamic scene incremental reconstruction and rendering method based on 3DGS, and belongs to the field of specific computer models, and the method comprises the steps: constructing a 3DGS model at an initial moment based on an original image set; obtaining any visual angle image at the moment t in the dynamic scene, and determining a first updating area through semantic segmentation and target recognition; generating an increment updating region based on the luminosity error, the local similarity and the global semantic feature; generating a second update region, modeling in the second update region, and minimizing region reconstruction errors to generate an optimized Gaussian point set; and fusing and optimizing the real-time model at the previous moment to obtain a real-time model Gt at the moment t for real-time rendering. According to the method, geometric and texture information of a scene can be effectively coded, accurate detection and local increment updating of a dynamic region are supported, Gaussian point parameters are optimized to improve the continuity and visual quality of a model, low-delay real-time rendering is realized, and the efficiency and quality of three-dimensional scene processing in a dynamic environment are effectively improved.

The invention relates to the technical field of image processing, and provides an indoor three-dimensional image intelligent rendering method based on image processing. Comprising the following steps of indoor multi-view image acquisition, image preprocessing, intelligent analysis of indoor scene elements, construction of an indoor three-dimensional initial model with attributes, intelligent generation of rendering parameters, adaptive LOD real-time rendering and intelligent interaction optimization. According to the intelligent interaction optimization system, through deep fusion of natural language processing, user preference learning and real-time rendering technologies, a set of efficient, visual and personalized virtual scene rendering interaction process is constructed. According to the method, the problems that traditional graphic software is complex in operation, high in learning cost and low in debugging efficiency are solved, and the user satisfaction and creation efficiency are remarkably improved through an intelligent recommendation and rapid iteration mechanism.

The invention relates to the technical field of augmented reality, in particular to an AR real-time scene reconstruction and illumination matching system and method based on neural rendering, and the system comprises a real-time scene understanding module, a real-time neural radiation field module and a real-time reflection inference and enhancement module. Comprising a forward network, a feature transformation layer and a backward network, the forward network maps a high-dimensional rendering space to a feature manifold, the feature transformation layer executes mapping from the feature manifold to a rendering manifold, and the backward network performs rendering calculation based on a Riemannian metric driven adaptive ray tracing mechanism; the real-time reflection inference and enhancement module performs material prediction and illumination information calculation based on an illumination dynamic adaptation system on a differential manifold, real-time application of a neural radiation field technology in an AR scene is realized through an innovative differential geometry framework, and a real-time rendering frame rate of 30-60 fps is achieved while a high-quality rendering effect is kept.

The invention provides an adaptive multi-view track roaming optimization method based on Cesium, and the method comprises the steps: obtaining an original path point, and carrying out the processing of the original path point, and generating a smooth path; dynamically adjusting the time rate of the smooth path movement; dynamically adjusting the number of interpolation points of the smooth path; loading the dynamically adjusted path, and binding with the path through a dual-mode view angle; and rendering the path in real time, and interacting with a user. According to the method, a Catmull-Rom spline interpolation and inflection point optimization method is adopted, so that turning is smoother; the terrain height is dynamically sampled, terrain adaptation is accurate, and the model is prevented from being suspended or falling into the ground; according to the invention, first person / third person switching is supported, and flexible visual angle control is realized; the speed calculation based on the path length ensures that the motion rhythm is natural.

Methods, systems, and computer readable media for rendering context-aware and interactive artificial intelligence-generated videos in real time. A talking face (“TF”) model may traverse from a first node to a second node of a state graph via an edge based on a TF instruction generated by an interaction model. The TF model may retrieve a transition video associated with the edge and a pre-computed video template associated with the second node from one or more TF databases. The pre-computed video template may include a plurality of masked video frames and a plurality of pre-computed mouth positions for each masked video frame. The TF model may inpaint a pre-computed mouth position into a masked region of each masked video frame to form a video frame stream. The interaction model may generate a video from the transition video and the video frame stream and present the video on a user device.