21 results about "Video texture" patented technology

The invention discloses a digital twin three-dimensional scene modeling method based on a webGPU, and relates to the technical field of three-dimensional scene modeling and graphic computing, multi-source sensing data are asynchronously sampled from sparse point cloud, a video texture sequence, a scene semantic tag graph and a structure boundary tuple, and a six-dimensional structure unit group is generated through a normalization operator; constructing a structural unit atlas with nodes representing component entities and edges representing constraint relations, and introducing a tension balance mechanism and multi-scale constraints to generate a modeling path prior model; constructing a graph calculation and graph rendering dual-channel assembly line in the WebGPU, and executing parallel texture mapping and boundary fitting operation; a dynamic sensing module is used for capturing scene disturbance and driving atlas response, and incremental reconstruction of the model is achieved; and finally, mapping the model to a Web terminal, and supporting microscopic semantic query and multi-layer data linkage. According to the method, the response speed, semantic consistency and structural adaptability of three-dimensional modeling in a complex environment are improved.

The invention relates to the technical field of image processing, and further relates to a video decoding and unreal engine rendering method based on GPU full-link zero copy, which comprises the following steps of: 1, performing hardware decoding on an input video code stream on a GPU, and constructing a motion vector description buffer region for storing compressed domain motion vector information according to a macro block sequence; step 2, reading the motion vector description buffer area on the GPU through a calculation shader, and generating dynamic special effect control buffer areas in one-to-one correspondence with the macro blocks; and step 3, in a post-processing material of the unreal engine rendering module, taking the shared video texture resource as an input texture, and outputting a video picture. On the premise of not depending on a host processor and not excessively occupying video memory bandwidth, unified management of pixel data and motion data is achieved, delay is remarkably reduced, and special effect stability and direction consistency are improved.

The invention discloses a time-space consistency-based video super-resolution method and device for a complex operation scene, and a medium, belongs to the field of image and video processing, and aims at solving the problem that a stable, clear and continuous-structure video sequence is difficult to generate in an existing method, and the time-space consistency-based video super-resolution method for the complex operation scene based on video prior is provided. Comprising the following steps: utilizing submerged space modeling, mapping an input low-resolution video to a submerged space, and obtaining a corresponding submerged variable representation; initializing a random noise tensor with the same size as the latent variable expression as an initial noise state; in each step of sampling, dividing a noise state and latent variable representation into a plurality of blocks according to space and time dimensions; denoising is carried out on each tile block; the initial noise is converted into high-quality latent variable representation; and a high-resolution video is reconstructed. According to the method, the power inspection video with high resolution and high consistency can be generated, the generation stability is kept, and the video texture detail expression capability is improved.

The invention relates to the technical field of XR operation, and discloses a WebXR-based panoramic immersive robot teleoperation system and method. The method comprises the following steps: carrying out distortion correction and splicing fusion on original images collected by a double-fisheye lens to obtain a panoramic image frame; encoding the panoramic image frame into a video code stream, transmitting the video code stream to a browser end, and decoding the video code stream at the browser end to obtain a panoramic video texture; mapping the panoramic video texture to the inner surface of a sphere, setting a virtual camera at the center of the sphere, and constructing a panoramic sphere rendering model; and converting the attitude quaternion of the VR head-mounted display into a sight decoupling rotation matrix, performing local GPU rendering on the panoramic sphere rendering model to obtain a real-time visual angle picture, and mapping a rocker shaft value of the gamepad into a control instruction to be sent to a robot dog end. In the whole view angle switching process, vertex transformation and texture sampling are completed at the browser end, no pan-tilt rotation instruction is generated or sent to the robot dog end through the network, and delay is reduced.

The present application relates to the technical field of image processing, and more particularly to a video decoding and Unreal Engine rendering method based on GPU full-link zero-copy, which comprises the following steps: step one, hardware decoding of an input video code stream on a GPU, and construction of a motion vector description buffer for storing compressed domain motion vector information in macroblock order; step two, reading of the motion vector description buffer by a calculation shader on the GPU to generate a dynamic special effect control buffer corresponding to each macroblock; and step three, in the post-processing material of the Unreal Engine rendering module, taking a shared video texture resource as an input texture and outputting a video picture. The present application realizes unified management of pixel data and motion data without relying on a host processor and without excessively occupying video memory bandwidth, significantly reduces delay, and improves special effect stability and direction consistency.

The invention relates to the technical field of 3D rendering, and discloses a rendering method based on video texture rendering, and the method comprises the steps: carrying out the rendering of a to-be-rendered object, and carrying out the time domain anti-aliasing processing, and obtaining a color image and a depth image; and performing video texture rendering on the to-be-rendered object based on the color image and the depth image to obtain a video texture template edge image, and performing pixel jitter elimination on pixels at the junction of the to-be-rendered object based on the video texture template edge image. According to the invention, the problem that the junction of the image shakes in the prior art is solved.

The application provides a video texture mapping and real-time rendering method and system based on a digital twin scene, relates to the technical field of digital twin rendering, and comprises the following steps: acquiring a real scene monitoring video stream and performing real-time image content analysis; dividing the texture details of different regions of the video stream according to the features and numerical values of the image analysis and generating region detail identifiers associated with the video content; identifying the dynamic features of the video picture based on the identifiers and generating scene state codes representing the comprehensive picture conditions; dynamically selecting a target engine and corresponding rendering quality parameters from preset heterogeneous rendering engines in combination with the current load state of a graphics processor; and mapping the video stream image data to the surface of a corresponding model of the digital twin scene as dynamic texture to realize efficient rendering of the picture, so that dynamic texture mapping and adaptive real-time rendering based on video content analysis and hardware load in the digital twin scene can be realized.

Embodiments of the present disclosure provide a video texture migration method and device, electronic equipment and storage medium, which obtain an original video, perform feature extraction on a target frame in the original video to generate first feature information of the target frame, wherein the target frame is a video frame after an Nth video frame in the original video, and the first feature information represents an image structure contour of the target frame; perform feature fusion on the first feature information of the target frame and reference feature information to obtain second feature information of the target frame, wherein the reference feature information is used to represent an image structure contour of a reference frame, the reference frame is a video frame before the target frame, and the second feature information is the first feature information without random noise; and generate a texture migration video according to the second feature information of the target frame and corresponding texture feature information, wherein the texture feature information represents image texture details of a reference image, thereby avoiding frame flickering and improving the display effect of the texture migration video.

The invention relates to the technical field of video compression, in particular to a real-time transmission system of an unmanned aerial vehicle power line inspection video. The system comprises a video acquisition module used for acquiring an original inspection video; the texture analysis module is used for analyzing the pixel change direction and the pixel change amplitude of a video frame in the original inspection video to obtain a texture feature value; the parameter determination module is used for determining the compression parameter information matched with the texture feature value as target compression parameter information in a plurality of pieces of preset compression parameter information; and the video compression module is used for performing video compression on the original inspection video according to the target compression parameter information to obtain a target inspection video. According to the invention, the video contents with different texture details are dynamically adapted through the differential video compression strategy, so that the overall resource overhead of video transmission is reduced while more details of the transmitted video can be kept.

The application relates to a webpage three-dimensional scene rendering method and device, computer equipment and a readable storage medium, and relates to the technical field of image rendering. The method comprises the following steps: obtaining a scene video stream of a modeling scene based on a game engine modeling; determining a video texture effect based on the scene video stream; rendering a first picture of the modeling scene based on the video texture effect on a webpage side by using a preset business component; and displaying the first picture in a webpage three-dimensional scene on the basis of including a second picture which has been completely rendered. The first picture is obtained by rendering the modeling scene of the game engine modeling on the webpage side by using the preset business component, and the first picture does not need to be rendered in the game engine. The first picture and the second picture which is rendered based on a preset rendering mode are displayed in the webpage three-dimensional scene, and the second picture corresponding UI and business do not need to be redeveloped and rendered in the game engine. The method is beneficial to reducing the development cost and workflow of the game engine.

The application discloses a multi-view video saliency prediction method fusing spatial audio, a medium and equipment, and adopts an audio-visual multi-modal fusion and cross-view alignment scheme.The application acquires time-synchronous multi-view video and audio signals and constructs a view order relationship, extracts video texture and motion feature fusion to obtain visual representation; the audio signal is subjected to time-frequency transformation and multi-scale coding to obtain audio content spectrum features.Through a sound source energy distribution estimation module, audio-visual correlation features are modeled, multi-modal features are fused to obtain single-view saliency features; cross-view saliency alignment is completed relying on a bidirectional feature propagation mechanism, each view saliency distribution map is generated, and network training is performed through pixel-level loss supervision.The application fuses spatial audio semantic information, makes up for the defects of a single visual modal, improves the precision, robustness and view consistency of multi-view video saliency prediction, and is suitable for multimedia perception, intelligent visual detection and other scenes.

The invention provides a two-axis turntable image rotation eliminating method based on inertial navigation data, which comprises the following steps: acquiring an original image in real time by an image acquisition device mounted on a two-axis turntable, and determining parameter information of a video texture map according to the original image; calculating a rotation transformation matrix A1 by using azimuth data and pitch angle data generated by rotation of the rotary table, and calculating a rotation transformation matrix A2 by using inertial navigation data of an inertial navigation system; converting a normal vector of a horizontal plane from a horizontal coordinate system xlylzl to a sensor coordinate system xsyszs, and calculating an image spinor alpha; and carrying out image rotation elimination display on the video texture map by taking the image rotation alpha as a rotation parameter and utilizing a rotation and cutting mechanism of OpenGL (Open Graphics Library). According to the attitude angle of the base of the two-axis rotary table and the axis angle data of the two-axis rotary table, the angle amount of the horizon line projected in the image space is calculated, the angle amount serves as the rotation compensation amount during display, namely, the image spinor alpha, and display is achieved through the rotation and cutting mechanism of OpenGL.

The invention provides a video texture mapping and real-time rendering method and system based on a digital twin scene, and relates to the technical field of digital twin rendering. The method comprises the following steps: dynamically dividing texture detail levels of different regions of a video stream according to features and numerical values of image analysis, generating region detail identifiers associated with video contents, identifying dynamic features of a video picture based on the identifiers, and generating a scene state code representing a comprehensive picture condition; a target engine and corresponding rendering quality parameters are dynamically selected from preset heterogeneous rendering engines in combination with the current load state of a graphics processor, and video stream image data are mapped to the surface of a model corresponding to a digital twin scene as dynamic textures, so that efficient rendering of pictures is realized. Dynamic texture mapping and self-adaptive real-time rendering based on video content analysis and hardware load in a digital twinning scene can be realized.

The application discloses a mobile platform digital twin and AR fusion method and system based on dynamic video projection, and the method comprises the following steps: acquiring real-time video stream collected by a mobile platform and real-time pose data of the mobile platform, and dynamically solving a projection matrix of the mobile platform based on the real-time pose data; constructing a three-dimensional real scene model of a target area, projecting video images in the real-time video stream frame by frame to corresponding surface areas of the three-dimensional real scene model through the projection matrix, and obtaining a dynamic three-dimensional real scene scene with real-time video texture; performing depth pre-rendering based on the dynamic three-dimensional real scene scene, acquiring depth buffer information corresponding to a real scene, performing depth testing and occlusion judgment on virtual augmented information by using the depth buffer information, and obtaining an augmented reality fusion picture with correct virtual-actual occlusion relationship. Through the above steps, the application solves the problems of real-time accurate fusion and physically correct occlusion of dynamic video of a mobile platform and a three-dimensional real scene model.

The present disclosure belongs to the field of video processing technology, and relates to a video rendering method and device, a storage medium, and an electronic device. The method includes: obtaining a first texture of a first video, and creating a shared environment for the first video, and creating a shared object for the first video, wherein the shared environment is a rendering environment on which the indirectly shared first texture depends, and the shared object is an object required to expose the first texture to the outside for indirect sharing; based on the shared environment, drawing the first texture in the shared object; calling the first texture to the second video corresponding to the first video through the shared object, so as to perform mixed rendering of the first video and the second video to obtain a mixed video. The present disclosure provides a safe and efficient way to share video textures, which avoids the situation where direct sharing causes the texture to be modified, prevents the mixed video from generating a black screen or flickering in advance, ensures the normal display of the color saturation and clarity of the original video, and optimizes the visual effect of the mixed video.

Methods and devices are provided to encode and decode time-varying textured 3D meshes in a data stream. By exploiting the 3D motion information from the base mesh encoder, a 3D motion map is generated that can be derived into different types of maps such as motion amplitude. Those maps have the property of being a bijection mapping between the object texture map and the motion texture map. As the 3D movement cannot be directly used as a motion estimator for the video coding, the amplitude of the 3D motion is interpreted in order to build an importance map that is transformed in a QP map provided to the video texture encoder and decoder.

The application discloses a multi-view video saliency prediction method fusing spatial audio, a medium and equipment, and adopts an audio-video multi-modal fusion and cross-view alignment scheme.The application acquires time-synchronous multi-view video and audio signals and constructs a view order relationship, extracts video texture and motion feature fusion to obtain visual representation; the audio signal is subjected to time-frequency transformation and multi-scale coding to obtain audio content spectrum features.Through a sound source energy distribution estimation module, audio-video correlation features are modeled, multi-modal features are fused to obtain single-view saliency features; cross-view saliency alignment is completed relying on a bidirectional feature propagation mechanism, each view saliency distribution map is generated, and network training is performed through pixel-level loss supervision.The application fuses spatial audio semantic information, makes up for the defects of a single visual modal, improves the precision, robustness and view consistency of multi-view video saliency prediction, and is suitable for multimedia perception, intelligent visual detection and other scenes.

The application discloses a digital twin three-dimensional scene modeling method based on webGPU, relates to the technical field of three-dimensional scene modeling and graph computing, assembles multi-source sensing data from sparse point clouds, video texture sequences, scene semantic label graphs and structural boundary element groups asynchronously, and generates six-dimensional structural element groups through normalization operators; a structural element graph is constructed, in which node represents component entity and edge represents constraint relationship, and a tension balance mechanism and a multi-scale constraint are introduced to generate a modeling path prior model; a graph computing and graph rendering double-channel pipeline is constructed in WebGPU to execute parallel texture mapping and boundary fitting operations; a dynamic sensing module is used to capture scene disturbance and drive graph response to realize incremental reconstruction of the model; finally, the model is mapped to a Web terminal to support micro semantic query and multi-layer data linkage; the method improves the response speed, semantic consistency and structural adaptability of three-dimensional modeling in complex environments.

A method of operating a virtual reality (VR) memory bank includes creating a 360° virtual location from a 360° photograph to be displayed on a virtual reality headset; automatically detecting one or more planar regions, within the 360° virtual location, associated with a visual media device within the physical location; instantiating and anchoring a corresponding video-texture surface to each planar region; automatically detecting one or more objects, within the 360° virtual location, associated with an audio media device within the physical location; instantiating and anchoring a corresponding positional-audio source to each object; including media assets within the 360° virtual location, wherein the media assets comprise at least one visual media asset and / or at least one audio media asset; displaying the at least one visual media asset on at least one video-texture surface; and playing the at least one audio media asset on at least one positional-audio source.

The invention discloses a video GIS (Geographic Information System) real-time projection positioning method and system based on a space viewfinder. The method comprises the following steps: acquiring a real-time video frame and observation parameters associated with the time of the real-time video frame; determining the sight line direction of the video pixel points based on a camera imaging model, and converting to a world coordinate system in combination with the position and attitude of the mobile platform; solving an intersection point of the sight line and the ground model based on ground elevation constraint to obtain a geographic drop point and converting the geographic drop point into geographic coordinates; geographic projection is carried out on the video angular points or grid points to construct a projection geometric object, video textures are attached to the projection geometric object to achieve ground-attached display, and the geometric object is driven to deform along with updating of observation parameters so as to follow in real time; outputting a corresponding geographic coordinate when a user clicks or outputs a target pixel through an algorithm; and attitude and angular point geographic information is synchronously recorded for historical playback reconstruction, so that the video has positioning and automatic drop point capabilities consistent with real time. The system comprises a data acquisition unit, a geographic projection calculation unit, a spatial viewfinder rendering unit, an automatic drop point unit and a data storage and playback unit.