11 results about "Video texture" patented technology

The invention relates to the technical field of image processing, and further relates to a video decoding and unreal engine rendering method based on GPU full-link zero copy, which comprises the following steps of: 1, performing hardware decoding on an input video code stream on a GPU, and constructing a motion vector description buffer region for storing compressed domain motion vector information according to a macro block sequence; step 2, reading the motion vector description buffer area on the GPU through a calculation shader, and generating dynamic special effect control buffer areas in one-to-one correspondence with the macro blocks; and step 3, in a post-processing material of the unreal engine rendering module, taking the shared video texture resource as an input texture, and outputting a video picture. On the premise of not depending on a host processor and not excessively occupying video memory bandwidth, unified management of pixel data and motion data is achieved, delay is remarkably reduced, and special effect stability and direction consistency are improved.

The invention relates to the technical field of XR operation, and discloses a WebXR-based panoramic immersive robot teleoperation system and method. The method comprises the following steps: carrying out distortion correction and splicing fusion on original images collected by a double-fisheye lens to obtain a panoramic image frame; encoding the panoramic image frame into a video code stream, transmitting the video code stream to a browser end, and decoding the video code stream at the browser end to obtain a panoramic video texture; mapping the panoramic video texture to the inner surface of a sphere, setting a virtual camera at the center of the sphere, and constructing a panoramic sphere rendering model; and converting the attitude quaternion of the VR head-mounted display into a sight decoupling rotation matrix, performing local GPU rendering on the panoramic sphere rendering model to obtain a real-time visual angle picture, and mapping a rocker shaft value of the gamepad into a control instruction to be sent to a robot dog end. In the whole view angle switching process, vertex transformation and texture sampling are completed at the browser end, no pan-tilt rotation instruction is generated or sent to the robot dog end through the network, and delay is reduced.

The present application relates to the technical field of image processing, and more particularly to a video decoding and Unreal Engine rendering method based on GPU full-link zero-copy, which comprises the following steps: step one, hardware decoding of an input video code stream on a GPU, and construction of a motion vector description buffer for storing compressed domain motion vector information in macroblock order; step two, reading of the motion vector description buffer by a calculation shader on the GPU to generate a dynamic special effect control buffer corresponding to each macroblock; and step three, in the post-processing material of the Unreal Engine rendering module, taking a shared video texture resource as an input texture and outputting a video picture. The present application realizes unified management of pixel data and motion data without relying on a host processor and without excessively occupying video memory bandwidth, significantly reduces delay, and improves special effect stability and direction consistency.

The application provides a video texture mapping and real-time rendering method and system based on a digital twin scene, relates to the technical field of digital twin rendering, and comprises the following steps: acquiring a real scene monitoring video stream and performing real-time image content analysis; dividing the texture details of different regions of the video stream according to the features and numerical values of the image analysis and generating region detail identifiers associated with the video content; identifying the dynamic features of the video picture based on the identifiers and generating scene state codes representing the comprehensive picture conditions; dynamically selecting a target engine and corresponding rendering quality parameters from preset heterogeneous rendering engines in combination with the current load state of a graphics processor; and mapping the video stream image data to the surface of a corresponding model of the digital twin scene as dynamic texture to realize efficient rendering of the picture, so that dynamic texture mapping and adaptive real-time rendering based on video content analysis and hardware load in the digital twin scene can be realized.

Embodiments of the present disclosure provide a video texture migration method and device, electronic equipment and storage medium, which obtain an original video, perform feature extraction on a target frame in the original video to generate first feature information of the target frame, wherein the target frame is a video frame after an Nth video frame in the original video, and the first feature information represents an image structure contour of the target frame; perform feature fusion on the first feature information of the target frame and reference feature information to obtain second feature information of the target frame, wherein the reference feature information is used to represent an image structure contour of a reference frame, the reference frame is a video frame before the target frame, and the second feature information is the first feature information without random noise; and generate a texture migration video according to the second feature information of the target frame and corresponding texture feature information, wherein the texture feature information represents image texture details of a reference image, thereby avoiding frame flickering and improving the display effect of the texture migration video.

The invention provides a video texture mapping and real-time rendering method and system based on a digital twin scene, and relates to the technical field of digital twin rendering. The method comprises the following steps: dynamically dividing texture detail levels of different regions of a video stream according to features and numerical values of image analysis, generating region detail identifiers associated with video contents, identifying dynamic features of a video picture based on the identifiers, and generating a scene state code representing a comprehensive picture condition; a target engine and corresponding rendering quality parameters are dynamically selected from preset heterogeneous rendering engines in combination with the current load state of a graphics processor, and video stream image data are mapped to the surface of a model corresponding to a digital twin scene as dynamic textures, so that efficient rendering of pictures is realized. Dynamic texture mapping and self-adaptive real-time rendering based on video content analysis and hardware load in a digital twinning scene can be realized.

The application discloses a mobile platform digital twin and AR fusion method and system based on dynamic video projection, and the method comprises the following steps: acquiring real-time video stream collected by a mobile platform and real-time pose data of the mobile platform, and dynamically solving a projection matrix of the mobile platform based on the real-time pose data; constructing a three-dimensional real scene model of a target area, projecting video images in the real-time video stream frame by frame to corresponding surface areas of the three-dimensional real scene model through the projection matrix, and obtaining a dynamic three-dimensional real scene scene with real-time video texture; performing depth pre-rendering based on the dynamic three-dimensional real scene scene, acquiring depth buffer information corresponding to a real scene, performing depth testing and occlusion judgment on virtual augmented information by using the depth buffer information, and obtaining an augmented reality fusion picture with correct virtual-actual occlusion relationship. Through the above steps, the application solves the problems of real-time accurate fusion and physically correct occlusion of dynamic video of a mobile platform and a three-dimensional real scene model.

The application discloses a multi-view video saliency prediction method fusing spatial audio, a medium and equipment, and adopts an audio-video multi-modal fusion and cross-view alignment scheme.The application acquires time-synchronous multi-view video and audio signals and constructs a view order relationship, extracts video texture and motion feature fusion to obtain visual representation; the audio signal is subjected to time-frequency transformation and multi-scale coding to obtain audio content spectrum features.Through a sound source energy distribution estimation module, audio-video correlation features are modeled, multi-modal features are fused to obtain single-view saliency features; cross-view saliency alignment is completed relying on a bidirectional feature propagation mechanism, each view saliency distribution map is generated, and network training is performed through pixel-level loss supervision.The application fuses spatial audio semantic information, makes up for the defects of a single visual modal, improves the precision, robustness and view consistency of multi-view video saliency prediction, and is suitable for multimedia perception, intelligent visual detection and other scenes.

A method of operating a virtual reality (VR) memory bank includes creating a 360° virtual location from a 360° photograph to be displayed on a virtual reality headset; automatically detecting one or more planar regions, within the 360° virtual location, associated with a visual media device within the physical location; instantiating and anchoring a corresponding video-texture surface to each planar region; automatically detecting one or more objects, within the 360° virtual location, associated with an audio media device within the physical location; instantiating and anchoring a corresponding positional-audio source to each object; including media assets within the 360° virtual location, wherein the media assets comprise at least one visual media asset and / or at least one audio media asset; displaying the at least one visual media asset on at least one video-texture surface; and playing the at least one audio media asset on at least one positional-audio source.

The invention discloses a video GIS (Geographic Information System) real-time projection positioning method and system based on a space viewfinder. The method comprises the following steps: acquiring a real-time video frame and observation parameters associated with the time of the real-time video frame; determining the sight line direction of the video pixel points based on a camera imaging model, and converting to a world coordinate system in combination with the position and attitude of the mobile platform; solving an intersection point of the sight line and the ground model based on ground elevation constraint to obtain a geographic drop point and converting the geographic drop point into geographic coordinates; geographic projection is carried out on the video angular points or grid points to construct a projection geometric object, video textures are attached to the projection geometric object to achieve ground-attached display, and the geometric object is driven to deform along with updating of observation parameters so as to follow in real time; outputting a corresponding geographic coordinate when a user clicks or outputs a target pixel through an algorithm; and attitude and angular point geographic information is synchronously recorded for historical playback reconstruction, so that the video has positioning and automatic drop point capabilities consistent with real time. The system comprises a data acquisition unit, a geographic projection calculation unit, a spatial viewfinder rendering unit, an automatic drop point unit and a data storage and playback unit.