566 results about "Morphing" patented technology

The invention discloses a multi-mode driven virtual digital human face animation generation method and system, and relates to the field of computer graphics, and the method comprises the steps: obtaining voice input and text input, and extracting voice features and text features; dynamically fusing the two features through an attention fusion model to generate facial expression and head posture control parameters, and dynamically adjusting contribution weights of a voice mode and a text mode to the control parameters by adopting a driving strategy of differentiation of facial upper expression and facial lower expression; and performing local deformation on the virtual digital human face image based on the control parameter to generate an initial animation frame, and performing refining processing on the initial animation by using a generative adversarial network to obtain a refined face animation. By means of the technical scheme, the natural and vivid virtual human face animation matched with the voice content and the text semantics can be generated.

In one embodiment, a method includes rendering for displays of an extended-reality (XR) display device a first output image of an XR assistant avatar within an environment in a field of view of a first user of the XR display device, wherein the XR assistant avatar has a first form and a first pose with respect to the first user and is interactable by the first user to access an assistant system, detecting a change in a context of the first user with respect to the environment, determining whether to morph the XR assistant avatar based on the detected change in the context of the first user, and rendering a second output image of the XR assistant avatar for the displays of the XR display device, wherein the XR assistant avatar is morphed to have a second form and a second pose with respect to the first user.

The invention relates to the technical field of computers, and relates to a rendering model training method and device, an illumination rendering method and device, a computer program product and electronic equipment. The training method comprises the following steps: extracting inherent attribute information and global light and shadow information of a to-be-rendered object from an image frame sample based on a teacher model; performing feature decoding on the Gaussian primitives by using an initial student model to obtain a basic physical attribute, and determining a first loss according to the basic physical attribute and the inherent attribute information, the initial student model comprising a plurality of Gaussian primitives attached to a deformable grid; interpolating light and shadow data of the probe model through the initial student model to obtain current light and shadow information, constructing second loss according to the current light and shadow information and global light and shadow information, and arranging the probe model on the surface of the deformable grid; and adjusting parameters in the initial student model according to the first loss and the second loss, and adjusting shadow data of the probe model to obtain a target student model and a target probe model.

The invention provides a texture mapping method, a texture mapping device, a computer storage medium and electronic equipment, and relates to the technical field of computer graphic processing. The method comprises the following steps: acquiring a world space coordinate and a world space normal of a vertex of a target model; performing texture projection on the world space coordinates on a plurality of world coordinate axes to obtain target texture coordinates of the vertexes of the target model on the world coordinate axes; determining a texture weight value based on the included angles between the world space normal and the plurality of world coordinate axes; and carrying out texture sampling according to the target texture coordinate to obtain an initial map sampling result, and carrying out mixed sampling on the initial map sampling result and the texture weight value on the corresponding world coordinate axis to carry out texture mapping on the to-be-processed model based on the target map sampling result. The method can get rid of dependence on a model topological structure and UV expansion of a three-dimensional model, and realizes a more real, natural, accurate and flexible texture fitting effect without deformation stretching.

The invention discloses a video deblurring method based on a deformable space-time sparse converter. The method comprises the following steps: carrying out feature extraction on continuous n frames of fuzzy videos, and calculating forward and backward light streams and corresponding fuzzy images; iteratively updating features of each frame through a bidirectional feature propagation module guided by a multi-scale fuzzy image, and aggregating feature information of different propagation branches; a deformable space-time sparse Transform module is adopted to carry out refinement processing on the aggregation features; and inputting the refined features into a decoder module, and generating a final deblurring result in combination with original input residual connection. The method is suitable for scenes of security monitoring, mobile equipment shooting and the like, the effect of removing the dynamic fuzzy area in the video can be improved, and the overall visual quality of the video is effectively improved.

The invention discloses a 6D pose estimation method and device fusing an attention mechanism, equipment and a medium, and relates to the technical field of object space poses, and the method comprises the steps: processing an RGB image and a depth image of a target object through a preset multi-modal feature fusion model, the model comprises a semantic segmentation module, a feature extraction module, a feature fusion module, an attitude estimation module and an attitude iterative optimization module. A target mask point cloud is obtained through semantic segmentation, feature fusion is performed by using a cross attention mechanism and deformable convolution, and a final pose is obtained through pose estimation and iterative optimization, so that the problems of insufficient feature extraction and weak multi-modal feature association in complex scenes such as weak texture and shielding are solved, and the accuracy and robustness of pose estimation are improved.

The invention discloses a long-tail data generation method based on feed-forward stylized reconstruction. The method comprises the following steps: constructing multi-modal geometric perception input and feature codes; reconstructing a semantic guidance scene based on a feature-level linear modulation mechanism; the invention relates to 3D consistency controllable style migration based on a feedforward reconstruction model. According to the invention, a feedforward reconstruction network based on DINOv2 improvement is constructed. Multi-modal adaptation is performed on a backbone network input layer, and decoupled high-level semantic features are explicitly injected by using a feature-level linear modulation mechanism, so that the geometric accuracy of automatic driving scene reconstruction is improved. On the basis of a three-dimensional scene of feedforward reconstruction, a distillation strategy is adopted, and the generation capability of a 2D diffusion model is migrated to a 3D Gaussian field. According to the method, the phenomena of object deformation, disappearance and the like in the generated data are effectively avoided, and long-tail weather data with geometric consistency of rain, snow and the like can be generated through the text instruction.

Methods and systems are disclosed for performing real-time deforming operations. The system receives an image that includes a depiction of a real-world object. The system applies a machine learning model to the image to generate a warping field and segmentation mask, the machine learning model trained to establish a relationship between a plurality of training images depicting real-world objects and corresponding ground-truth warping fields and segmentation masks associated with a target shape. The system applies the generated warping field and segmentation mask to the image to warp the real-world object depicted in the image to the target shape.

The invention relates to a monocular video three-dimensional human body high-quality reconstruction method based on Gaussian splashing and normal perception, and belongs to the field of computer aided design, graphics and computer vision. The method comprises the following steps: firstly, based on a monocular video frame in an input three-dimensional human body data set, guiding posture deformation through a time sequence, and carrying out whole body correlation deformation on a standard space posture to obtain an observation space human body posture; secondly, through normal perception Gaussian optimization, adaptive Gaussian density control and human body normal mapping supervision are carried out on the human body posture in the observation space, and a human body model of Gaussian representation is obtained; then, carrying out human body shadow feature learning on the input human body normal map by adopting an illumination model to obtain human body shadow features; and finally, light and shadow enhancement human body rendering is performed on the Gaussian representation human body model in combination with the human body light and shadow features and human body color information input into the monocular video, a human body rendering effect is output, and the quality and reality sense of three-dimensional human body reconstruction of the monocular video are improved.

Methods and systems are disclosed for performing real-time deforming operations. The system receives an image that includes a depiction of a real-world object. The system applies a machine learning model to the image to generate a warping field and segmentation mask, the machine learning model trained to establish a relationship between a plurality of training images depicting real-world objects and corresponding ground-truth warping fields and segmentation masks associated with a target shape. The system applies the generated warping field and segmentation mask to the image to warp the real-world object depicted in the image to the target shape.

The invention relates to a method for reducing motion blur of an erf model by using events and frames, and belongs to the technical field of computer vision. The method comprises the following steps: converting an event stream into triple representation; generating a bidirectional optical flow field based on the event flow, and aligning the fuzzy frame and the event flow through event-guided deformation convolution; constructing a double-branch optimization framework; the double-branch features are dynamically fused through frequency domain attention gating, and the weight of the frequency domain attention gating is generated by event frequency spectrums through MLP; generating a pseudo-true value based on an event double integral model, aligning an event spectrum and rendering a high-frequency component, and carrying out chain constraint on the continuity of a cross-frame radiation field through event brightness change; event-frame non-linear response differences are modeled by a learnable camera response function. Clear reconstruction and rendering of the moving target in a complex dynamic scene are realized.

The invention discloses a semantic segmentation method based on phase contour constraint, and belongs to the technical field of computer vision. The method comprises the following steps of: acquiring a two-dimensional texture image and a deformed stripe image of a scene to be detected in the same view field by using a calibrated camera-projector system; resolving the deformed fringe image through a multi-step phase shift and Gray code technology to obtain absolute phase distribution, and calculating a gradient field; according to a preset sudden change condition, extracting a physical edge contour reflecting the depth change of the object from the gradient field; and mapping the physical edge contour into the two-dimensional texture image based on a pixel coordinate corresponding relation to accurately define a semantic object region, endowing a corresponding category label, and finally generating a semantic label mask image. According to the method, the uncertainty of the visual texture edge is corrected by utilizing the physical phase information, the problem of edge extraction in a complex illumination scene is effectively solved, automatic generation of semantic segmentation data is realized, and the training precision and robustness of a semantic segmentation model are improved.

The invention discloses a method for improving the quality of a space-time domain compressed video by using a dense network, and the method mainly comprises the following steps: firstly, inputting a low-quality video obtained by HEVC compression into a network, aligning a certain frame to be enhanced with three adjacent frames before and after the frame in a universal deformable convolution mode, and carrying out the reconstruction of the frame to be enhanced; extracting an initial space-time fusion feature; extracting spatial feature information containing more levels from a to-be-enhanced frame by using the dense structure; the initial space-time fusion features and the spatial features are input into a space-time information adaptive fusion module together, and optimized space-time information is obtained through a series of introduced attention mechanisms and convolution layers; and finally, sending to a reconstruction module, obtaining an optimal residual error through a network, and adding the optimal residual error with a compressed frame to obtain a reconstructed video frame. Experimental results show that the method can effectively suppress the compression effect of the video, improve the video quality and obtain a better visual effect.

The invention discloses a real-time software deformation interaction method based on a physical engine in an environment. The real-time software deformation interaction method comprises the following steps: S1, establishing a virtual environment and a software model; s2, constructing an interactive perception and collision detection network; s3, receiving and analyzing a multi-modal interaction instruction; s4, real-time deformation calculation based on a physical engine; s5, performing multi-user collaborative state synchronization and rendering feedback; and S6, carrying out fault state simulation on the software object. According to the method, real-time and high-precision simulation of software deformation is realized through deep fusion of software dynamics calculation of a physical engine and an interaction instruction stream, and through hierarchical physical attribute binding and client prediction, the network synchronization data volume is greatly reduced while the millimeter-level deformation precision is ensured, so that the network synchronization efficiency is improved. And low-delay and high-consistency experience under multi-person cooperative operation is ensured.

A method for generating photorealistic 4D scenes from text inputs is disclosed. The method utilizes a text-to-video diffusion model to generate a reference video and a freeze-time video. A canonical 3D representation is reconstructed using deformable 3D Gaussian Splats (D-3DGS) based on the freeze-time video. Temporal deformations are learned to capture dynamic interactions in the reference video. The method employs a novel Score Distillation Sampling strategy combining multi-view and temporal aspects to enhance consistency and robustness. The resulting 4D scenes feature multiple objects interacting with detailed background environments, viewable from different angles and times. The method enables flexible camera control and integration with augmented and virtual reality applications. Some examples include features such as image-to-4D generation.

The disclosed computer-implemented method may include receiving, by a computing device, multi-view flat-lit performance data of a subject. Additionally, the method may include rendering, by the computing device, a dynamic sequence of novel-view flat-lit images of the subject based on a deformable three-dimensional Gaussian splatting (3DGS) model. The method may also include providing the rendered dynamic sequence of flat-lit images as input to a diffusion-based relighting model trained on the multi-view flat-lit performance data of the subject. Furthermore, the method may include generating, by the computing device using the diffusion-based relighting model, a relit sequence of the subject under a specified lighting condition. Various other methods, systems, and computer-readable media are also disclosed.

The invention relates to the technical field, in particular to a skeletal animation display method which comprises the steps of creating a static model based on a modeling tool, adjusting topological structures of high and low density areas of joints, verifying the symmetry of the model and correcting and optimizing the model. Establishing a skeleton hierarchy and a father-child skeleton chain based on the model; associating model vertexes to a skeleton system to establish an influence relationship, and optimizing weight parameters; the posture of the skeleton at different time points is defined by setting a key frame, and the animation effect is achieved; and ensuring that the animation basic logic is correct, and testing the motion state characteristics of the skeleton model. According to the skeleton animation display method and device, the equipment and the storage medium, the vertex density of the joint area is adjusted, symmetry processing is carried out, and weight gradual change processing and weight normalization verification are provided, so that the problem that a skeleton model is easy to tear, deform and distort in the animation can be solved; and meanwhile, a skeleton chain hierarchical structure is constructed, and the adaptability of a skeleton system and the model is further enhanced based on an IK controller.

The invention relates to the technical field of computer vision, in particular to a text-guided creativity and time consistency-oriented video coloring method and system. By introducing a time deformable attention block, according to the position and shape change dynamic state of an object in a video frame, dynamic features of an instance are captured in the time dimension, so that feature representation consistency is kept, and color flickering and shifting in the coloring process are effectively prevented; semantic representation of a text noun concept is adjusted through a cross-modal pre-fusion module, mask cross attention in the module is used for enhancing understanding of the model on the noun concept and instance perception, and color distribution accuracy is improved; the global structure of the colored video is maintained through gray level video information introduced in the video decoding process; a cross-fragment fusion mechanism is introduced during reasoning, so that the long-term consistency of video coloring is maintained; through text guidance, the user can color the video according to own demands and creativity.

The invention provides a method for constructing a physical field of a horizontal axis water turbine blade of an agent model, and relates to the technical field of digital twinning. According to the method, rotating speed data of a water turbine under different flow velocity conditions are determined, and rotating speed values corresponding to all flow velocities are obtained; then, fluid-solid coupling simulation is carried out in combination with experimental data, and stress and deformation data of the water turbine blade under the actual operation working condition are further obtained; according to the method, the KNN algorithm is utilized to carry out order reduction processing on the obtained blade structure data, a plurality of grid points are combined into representative points, the calculation complexity is low, the processing requirement of graph rendering software on the data size in the real-time rendering process is met, a combined Kriging method is adopted to carry out digital twinning agent model training on the data after order reduction, and the real-time rendering efficiency is improved. The reconstruction of a real-time physical field of the blade is realized; iteratively optimizing the digital twin physical field proxy model according to a genetic algorithm, and solving the problem of deviation between the proxy physical field model and a real physical model; and finally, deploying the digital twin physical field agent model into a digital twin system and solving physical field distribution in real time.

Provided is a method, system, and computer program product for using a generative adversarial network to visually generate object deformation predictions. A processor may identify, based on an analysis of image data, a plurality of objects that are in a first stack formation. The processor may determine a load distribution of each object of the plurality of objects in relation to a subset of objects of the plurality of objects in the first stack formation. The processor may generate, using a generative adversarial network (GAN) algorithm and based on the load distribution for each object, a visualization depicting deformation of each object of the plurality of object in relation to the subset of objects of the plurality of objects in the first stack formation. The processor may display the visualization depicting the deformation of each object of the plurality of objects to a user.

An example computing device may include memory and one or more processors. The one or more processors may be configured to parallel entropy decode encoded video data from a received bitstream to generate entropy decoded data. The one or more processors may be configured to predict a motion vector based on the entropy decoded data. The one or more processors may be configured to decode a motion vector residual from the entropy decoded data. The one or more processors may be configured to add the motion vector residual and motion vector. The one or more processors may be configured to warp previous reconstructed video data with an overlapped block-based warp function using the motion vector to generate predicted current video data. The one or more processors may be configured to sum the predicted current video data with a residual block to generate current reconstructed video data.

The present application discloses a watermark embedding method and related devices, equipment, and storage medium, wherein the watermark embedding method includes: performing feature encoding based on the original image to obtain image features of the original image, and performing feature projection based on the target string to be embedded to obtain sequence features of the target string; performing deformation convolution on the image features based on the offset obtained by offset prediction of the image features to obtain deformation features of the image features, and performing feature interaction based on the deformation features and the sequence features to obtain a first feature, and performing modulated convolution based on the image features and the sequence features to obtain a second feature; performing fusion decoding based on the first feature and the second feature to obtain a target image after the target string is embedded as an image watermark in the original image. The above scheme can ensure the visual quality after the watermark is embedded as much as possible and improve the anti-attack capability.

Systems, methods, devices, and media for anamorphosis systems to generate and cause display of anamorphic media are disclosed. In one embodiment, an anamorphosis system is configured to identify a set of features of a space, determine relative positions of the set of features, determine a perspective of the mobile device within the space based on the relative positions of the set of features, retrieve anamorphic media based on the location of the mobile device, and apply the anamorphic media to a presentation of the space at the mobile device. The anamorphic media may include media items such as images and videos, configured such that the media items are only visible from one or more specified perspectives. The anamorphic media may include a stylized text string projected onto surfaces of a space such that the stylized text string is correctly displayed when viewed through a user device from a specified perspective.

The embodiment of the invention provides a texture coordinate generation method and device of a curved surface three-dimensional model, electronic equipment and a storage medium. The method comprises the following steps: acquiring three-dimensional coordinate information of a model vertex; determining curved surface geometric feature parameters corresponding to the vertexes based on the three-dimensional coordinate information; determining a plane mapping conversion relation based on the curved surface geometric feature parameters; and generating texture coordinates of the model by applying the plane mapping conversion relation. According to the technical scheme, the accurate plane mapping conversion relation is established based on the geometric feature parameters of the curved surface, the problem of pole deformation in traditional spherical UV mapping is effectively solved, seamless mapping of the square continuous mapping on the curved surface model is achieved, meanwhile, the calculation load of a graphics processor is reduced, and the processing efficiency of texture mapping is improved.

The embodiment of the invention relates to a video generation method and device, computer equipment and a storage medium, and the method comprises the steps: carrying out the filling processing of each frame of image of an initial video, and obtaining a first image; obtaining a second image from the initial video, and obtaining a target audio; extracting audio features from the target audio, extracting first image features from the first image, and extracting second image features from the second image; performing alignment operation on the first image feature and the second image feature; performing spatial deformation on the second image feature according to the audio feature and the aligned second image feature; generating a mouth shape image according to the deformed second image feature and the first image feature; and generating a target video according to each mouth shape image and the target audio. Therefore, the target video with highly synchronous mouth shape and voice content can be generated while the character identity features are kept, and the naturalness and visual reality of mouth shape and voice matching generation are improved.

The invention relates to the technical field of graphic image processing, and discloses a personalized replacement cutting deformation rendering method and system for a design file, and the method comprises the steps: 1, analyzing the design file, binding an external material with a target layer according to a replacement mapping table, and obtaining an adaptive material texture; 2, performing normalization deformation to form a single inverse mapping rule; step 3, sampling according to a single inverse mapping rule and applying a mask to obtain a clipped layer; 4, setting a synthesis area for the cutting group, and pre-multiplying transparency accumulation to obtain local synthesis textures of the cutting group; step 5, pasting the texture back to the main canvas and combining the texture with the non-group layer according to a mixed mode; step 6, determining a dirty rectangular region according to propagation change of the layer dependency graph, and generating a multiplexing identifier and a texture fingerprint; and step 7, outputting the main canvas and exporting the texture and metadata. According to the invention, stable, efficient and traceable reuse of integrated online rendering of personalized replacement, cutting and deformation of design files is realized.

Certain aspects of the present disclosure provide techniques for coarse-to-fine attention-based sensor fusion. The method includes obtaining a 3D voxel image space of an environment, the 3D voxel image space comprising first features of the environment extracted from a plurality of images; obtaining a point cloud corresponding to the environment, the point cloud comprising points, wherein the points are labeled with second features; generating a coarse representation of the environment, the coarse representation comprising a projection of the first features onto the points of the point cloud, wherein the projection is based on combining a respective set of the first features within a first radius from a point with a respective set of the second features corresponding to the point; applying a deformable attention module to predict fine sampling locations and extract fine features from the first features; and generating, with an attention module, a fine representation of the environment.

The invention discloses a homework correction method based on image recognition, relates to the technical field of homework correction, and comprises the following steps: breaking through the limitation of the existing homework correction technology on surface processing of consecutive characters, and bringing consecutive deformation into an expression intention analysis framework through linkage of stroke topological characteristics and semantic compensation; metaphor words intensified by students deliberately, such as a s with lengthened strokes, can be recognized, and the rhetorical structure is prevented from being misjudged as word use errors; the method comprises the following steps of: forcibly merging and eliminating the segmentation of physical paging on a long rhetorical sentence through a cross-row syntax tree, and ensuring that a not only... but also type emphasis sentence is completely analyzed; a dynamic visualization mechanism is adopted to convert an abstract grammar rule into a perceptible scene: a time axis animation of a tense error visually presents dislocation of an action and a time node, and a depth rhetorical analysis triggered by a hand-drawn wavy line accurately marks a metaphor ontology-metaphor body mapping relationship, so that students can understand the essence of the error at a glance.

The invention discloses a three-dimensional model dynamic rendering method and a storage medium, and the method comprises the steps: obtaining three-dimensional data of a target scene, obtaining a plurality of voxels, and enabling each voxel to correspond to a plurality of voxel attributes and an initial comprehensive displacement vector; in any voxel in any time step, obtaining a comprehensive displacement vector of the voxel in the current time step according to a preset physical deformation rule and / or animation curve deformation rule, and performing coordinate transformation on the voxel; after the coordinate transformation, updating the voxel attribute of the voxel corresponding to the current time step length; and performing real-time three-dimensional rendering on the target scene according to the vertex normal of each voxel at the current time step length and the voxel attribute. The physical consistency is ensured in the dynamic deformation process by synthesizing the displacement vector; by updating the voxel attributes, the voxel attributes are kept in smooth transition after deformation; and high-quality real-time rendering of the target scene is realized by recalculating the deformed vertex normal and combining the illumination model.

The invention discloses a path-based deformed character typesetting rendering method, device and equipment. The method comprises the following steps: receiving character content input by a user and a corresponding deformation strength parameter; the text content is transversely typeset by adopting non-automatic line folding; according to the deformation strength parameters, the curvature radius of the target path is dynamically calculated through a Newton iteration method; the origin coordinates of the character content characters are mapped to the target path, and the rotation angle of each character is calculated; when the total width of the characters exceeds the perimeter of the target path, automatically generating a multi-circle path and distributing the characters; a final graph is output through a cross-platform rendering engine, and the display consistency of all platforms is ensured. According to the method and the device, the problems of inconvenience in path adjustment, cross-platform inconsistency and incapability of processing super-long characters in path-based character typesetting in the prior art can be solved. By means of the method, a designer only needs to adjust the strength value, the bending effect of the characters along the dynamic path can be previewed in real time, all devices and browsers are compatible, and the design efficiency is greatly improved.