664 results about "Morphing" patented technology

The invention relates to the technical field of 3D printing models, in particular to a 3D printing model slice feature compensation method and system and a medium. The method comprises the following steps that a 3D printing model is collected, a model feature skeleton is split, transverse slice features are generated, then a 3D printing environment is collected, virtual simulation is carried out, environment stress analysis is carried out on the skeleton slice features to obtain environment stress data, then deformation scale deduction and thermal deformation prediction are carried out based on the environment stress data, and a thermal deformation model is obtained. On the basis, reverse deformation simulation is carried out, deformation repair requirements are evaluated, a deformation compensation path is planned, finally, compensation simulation of a model feature skeleton is achieved, non-target deformation is analyzed, a compensation path is adjusted, and therefore the feature compensation method is optimized. According to the 3D printing model slice feature compensation method, the accuracy and quality of the 3D printing model are integrally improved.

The invention relates to a deformable object interactive operation control method based on a visual touch-language-action multi-mode model, and the method comprises the steps: carrying out the image, touch and language data coding of a deformable object, and obtaining visual, touch and language features; performing cross-modal feature alignment processing on the visual features, the tactile features and the language features to obtain multi-modal fusion features; inputting the multi-modal fusion features into a large model for environment understanding; iteratively performing action planning and execution by adopting a'thinking-decision 'planning mode; and repeatedly executing the steps until the current deformable object interaction operation task is completed. Compared with the prior art, the multi-modal feature alignment capability, the action planning precision and the task adaptability are improved, efficient recognition and interaction of the robot on the deformable object can be achieved, deformation and state changes of the object can be effectively dealt with especially in a complex environment, the operation strategy can be dynamically adjusted, and the operation efficiency is improved. And more intelligent and accurate deformable object operation is realized.

In particular embodiments, a computing system may receive a video comprising a plurality of image frames. The system may generate, for each image frame in the video, an initial depth map using a machine-learning model. The system may compute a misalignment error indicating depth misalignments in initial depth maps using a reprojection technique. The system may generate, for each image frame in the video, an optimized camera pose and a flexible deformation spline associated with the image frame to minimize the misalignment error. The system may generate, for each image frame in the video, a refined depth map by adjusting the initial depth map associated with the frame using the flexible deformation spline associated with the image frame.

The invention discloses a multi-mode driven virtual digital human face animation generation method and system, and relates to the field of computer graphics, and the method comprises the steps: obtaining voice input and text input, and extracting voice features and text features; dynamically fusing the two features through an attention fusion model to generate facial expression and head posture control parameters, and dynamically adjusting contribution weights of a voice mode and a text mode to the control parameters by adopting a driving strategy of differentiation of facial upper expression and facial lower expression; and performing local deformation on the virtual digital human face image based on the control parameter to generate an initial animation frame, and performing refining processing on the initial animation by using a generative adversarial network to obtain a refined face animation. By means of the technical scheme, the natural and vivid virtual human face animation matched with the voice content and the text semantics can be generated.

The invention provides a method for generating a harmonious object in a three-dimensional scene based on physical rendering and uncertainty estimation, and the method comprises the steps: giving a three-dimensional scene, and generating the text description of a target object and a rectangular frame of the position of the target object in a 2D rendering view of the three-dimensional scene; according to the text description of the generated target object, a multi-view image generation diffusion model is utilized to construct a deformable tetrahedral mesh of the geometric appearance of the target object; constructing a feature network comprising a material item prediction module and an uncertainty item prediction module; obtaining a rendered picture of the target object by using a physical rendering technology, and performing training optimization on the feature network; and obtaining a multi-view scene rendering dense view containing the target object and a 3D model file according to the three-dimensional scene, the deformable tetrahedral mesh and a material item output by the trained feature network. According to the method, the problem that the target object and the scene are not harmonious in fusion is effectively solved, and the reality sense and the visual effect of the target object in the scene are enhanced.

The invention discloses an immersive video enhancement method and device based on frequency domain boundary collaborative optimization, and relates to the field of video processing, and the method comprises the steps: obtaining a to-be-reconstructed compressed multi-view texture and depth video sequence, and inputting the sequence into a trained immersive video enhancement model; a current to-be-enhanced video frame passes through a feature extraction module, and high-frequency features and low-frequency features are extracted respectively; the high-frequency features and the low-frequency features pass through a frequency domain enhancement module to obtain a frequency domain enhanced image; inputting the frequency domain enhancement image and the current video frame to be enhanced into a boundary enhancement module to obtain a fused image; and inputting the fused image and an adjacent video frame of the current to-be-enhanced video frame into a space-time deformable convolution module to obtain an aligned fused image, and performing prediction on the aligned fused image through a quality enhancement module to obtain an enhanced residual error and generate a corresponding reconstructed video. According to the method, the problems of compression artifacts, boundary artifacts, low quality of immersive videos and the like are solved.

Methods and computer systems for methods, computer systems, and computer-readable memory media for determining a warp function. An as-designed watertight spline model of an object is received. A set of sweep points along a sweep trajectory for manufacturing the object are used to determine a sweep surface. Model points are sampled from the as-designed watertight spline model and projected onto nearest as-executed points on the sweep surface. The projected as-executed points are used to construct an as-executed model of the object. A warp function is determined based on a difference between the as-designed watertight spline model and the as-executed model. The warp function is a continuous function approximating differences between the as-designed model and the as-manufactured object. The warp function is stored in a non-transitory computer-readable memory medium.

In one embodiment, a method includes rendering for displays of an extended-reality (XR) display device a first output image of an XR assistant avatar within an environment in a field of view of a first user of the XR display device, wherein the XR assistant avatar has a first form and a first pose with respect to the first user and is interactable by the first user to access an assistant system, detecting a change in a context of the first user with respect to the environment, determining whether to morph the XR assistant avatar based on the detected change in the context of the first user, and rendering a second output image of the XR assistant avatar for the displays of the XR display device, wherein the XR assistant avatar is morphed to have a second form and a second pose with respect to the first user.

The invention provides a digital human body modeling system and method based on space-time consistency Gaussian sputtering, and the system comprises a Gaussian set construction module which is used for constructing an initial Gaussian set based on a collected monocular RGB video sequence of human body motions; the non-rigid deformation module is used for updating a pre-constructed standard Gaussian set by adopting a Gaussian decoder based on the initial Gaussian set to obtain a dynamic Gaussian set containing non-rigid deformation; the space-time constraint module is used for applying space-time consistency constraint to the dynamic Gaussian set to obtain the dynamic Gaussian set of the space-time consistency constraint; the rigid deformation module is used for obtaining an attitude-driven Gaussian set based on the rigid deformation of the linear binding skeleton; and the projection rendering module is used for rendering the attitude-driven Gaussian set into a Gaussian image under the current frame attitude. According to the method, a space-time consistency mechanism of motion perception is combined, and the consistency problem of non-rigid deformation in a monocular video is effectively solved.

The invention relates to a dynamic scene reconstruction method and device, equipment and a medium, and the method comprises the steps: obtaining a picture sequence; the image sequences are input into a dynamic scene reconstruction model, a dynamic scene between the image sequences is generated, and the dynamic scene reconstruction model comprises a 3D Gaussian sputtering module used for conducting preliminary scene reconstruction on the image sequences to obtain static scene Gaussian; the dynamic component module is used for generating the variable quantity of the position of the static scene Gaussian according to the position attribute of the static scene Gaussian in combination with the time characteristic; and the color component module is used for generating the variable quantity of the Gaussian color attribute of the static scene by combining the Gaussian motion deformation characteristics according to the Gaussian color attribute of the static scene. According to the invention, a real-time rendering function can be realized on the basis of ensuring high-fidelity scene reconstruction.

The invention relates to the technical field of computers, and relates to a rendering model training method and device, an illumination rendering method and device, a computer program product and electronic equipment. The training method comprises the following steps: extracting inherent attribute information and global light and shadow information of a to-be-rendered object from an image frame sample based on a teacher model; performing feature decoding on the Gaussian primitives by using an initial student model to obtain a basic physical attribute, and determining a first loss according to the basic physical attribute and the inherent attribute information, the initial student model comprising a plurality of Gaussian primitives attached to a deformable grid; interpolating light and shadow data of the probe model through the initial student model to obtain current light and shadow information, constructing second loss according to the current light and shadow information and global light and shadow information, and arranging the probe model on the surface of the deformable grid; and adjusting parameters in the initial student model according to the first loss and the second loss, and adjusting shadow data of the probe model to obtain a target student model and a target probe model.

The invention discloses a grid sequence driven two-dimensional face animation generation method and device, equipment and a medium. The method comprises the steps of obtaining a 2D face image needing to be driven; the initial frame of the grid sequence motion and the 2D face image are bound by using Gaussian splash expression, the topology of the 3D grid and the identity information of the 2D face are bound to a Gaussian model, and an initial Gaussian head portrait is generated; representing motion of the grid sequence on Gaussian expression by combining a deformation field, optimizing a Gaussian head portrait, and generating an animation video sequence through rendering; wherein the 2D replay network processes an input reference face image and a grid current rendering frame as conditions, a gradient is generated, and the posture and the features of the head portrait are optimized, so that the feature height of the generated animation is consistent with the feature height of the reference face image. According to the method, the rendering cost and complexity can be effectively reduced by introducing the medium representation method based on Gaussian splashing and combining the Gaussian deformation field optimization method guided by two-dimensional prior and gradient.

The invention discloses a video virtual fitting framework in a real scene based on a controllable image diffusion model. The video virtual fitting framework comprises a data preprocessing module used for constructing feature expressions of figures and clothes; the single-stage image fitting network is used for generating a virtual fitting image based on the preprocessed data and performing implicit deformation and fusion of clothes by using a cross attention mechanism; the diffusion guiding module is used for enhancing the time sequence consistency of the generated video, a self-supervision model Video MAE and a self-supervision image model DINO-V2 are introduced, the video is masked by the self-supervision model Video MAE, and a complete video is reconstructed through inter-frame information; and the long video generation module is used for generating a long video and keeping the consistency of the video, splitting the complete video into a plurality of short video clips, and reducing the jitter at the clip joints by using a time sequence common noise reduction strategy. The diffusion guiding module is designed, efficient time sequence consistent video virtual fitting is achieved through self-supervised learning, and the method has the advantages of being excellent in shielding processing, stable in long video generation and the like.

The invention relates to a polarization dislocation code fused phase shift coding method and system, and the method comprises the steps: obtaining calibration parameters of a camera and a projector, fusing a phase shift stripe pattern with a dislocation Gray code stripe pattern through the polarization characteristics of the projector, and obtaining a polarization dislocation fused phase shift stripe pattern; obtaining a deformed polarization dislocation fusion phase-shift stripe pattern after the projector projects the polarization dislocation fusion phase-shift stripe pattern to the surface of an object to be measured, extracting a phase-shift stripe pattern and a dislocation Gray code stripe pattern from the deformed polarization dislocation fusion phase-shift stripe pattern, and calculating a wrapped phase by using the extracted phase-shift stripe pattern, carrying out threshold processing on the staggered Gray code stripe pattern to obtain a binary Gray code pattern; and performing phase unwrapping on the wrapped phase through the binarization Gray code pattern, and performing three-dimensional reconstruction on the to-be-measured object through calibration parameters of the camera and the projector according to the unwrapped phase. According to the method disclosed by the invention, measurement errors caused by an underexposure or overexposure phenomenon are inhibited, and the measurement precision and the three-dimensional reconstruction efficiency are improved.

The invention provides a texture mapping method, a texture mapping device, a computer storage medium and electronic equipment, and relates to the technical field of computer graphic processing. The method comprises the following steps: acquiring a world space coordinate and a world space normal of a vertex of a target model; performing texture projection on the world space coordinates on a plurality of world coordinate axes to obtain target texture coordinates of the vertexes of the target model on the world coordinate axes; determining a texture weight value based on the included angles between the world space normal and the plurality of world coordinate axes; and carrying out texture sampling according to the target texture coordinate to obtain an initial map sampling result, and carrying out mixed sampling on the initial map sampling result and the texture weight value on the corresponding world coordinate axis to carry out texture mapping on the to-be-processed model based on the target map sampling result. The method can get rid of dependence on a model topological structure and UV expansion of a three-dimensional model, and realizes a more real, natural, accurate and flexible texture fitting effect without deformation stretching.

An example system include one or more memories and one or more processors coupled to the one or more memories. The one or more processors are configured to perform a first segmentation operation on a previous frame of image data to generate first segmentation data. The one or more processors are configured to perform a deformable convolution operation based on the first segmentation data to generate a deformable convolution output. The one or more processors are configured to perform a second segmentation operation on a current frame of image data to generate second segmentation data. The one or more processors are configured to combine the deformable convolution output with the second segmentation data to generate third segmentation data and control operation of a device based on the third segmentation data.

The invention discloses a video deblurring method based on a deformable space-time sparse converter. The method comprises the following steps: carrying out feature extraction on continuous n frames of fuzzy videos, and calculating forward and backward light streams and corresponding fuzzy images; iteratively updating features of each frame through a bidirectional feature propagation module guided by a multi-scale fuzzy image, and aggregating feature information of different propagation branches; a deformable space-time sparse Transform module is adopted to carry out refinement processing on the aggregation features; and inputting the refined features into a decoder module, and generating a final deblurring result in combination with original input residual connection. The method is suitable for scenes of security monitoring, mobile equipment shooting and the like, the effect of removing the dynamic fuzzy area in the video can be improved, and the overall visual quality of the video is effectively improved.

The invention discloses a 6D pose estimation method and device fusing an attention mechanism, equipment and a medium, and relates to the technical field of object space poses, and the method comprises the steps: processing an RGB image and a depth image of a target object through a preset multi-modal feature fusion model, the model comprises a semantic segmentation module, a feature extraction module, a feature fusion module, an attitude estimation module and an attitude iterative optimization module. A target mask point cloud is obtained through semantic segmentation, feature fusion is performed by using a cross attention mechanism and deformable convolution, and a final pose is obtained through pose estimation and iterative optimization, so that the problems of insufficient feature extraction and weak multi-modal feature association in complex scenes such as weak texture and shielding are solved, and the accuracy and robustness of pose estimation are improved.

The invention discloses a long-tail data generation method based on feed-forward stylized reconstruction. The method comprises the following steps: constructing multi-modal geometric perception input and feature codes; reconstructing a semantic guidance scene based on a feature-level linear modulation mechanism; the invention relates to 3D consistency controllable style migration based on a feedforward reconstruction model. According to the invention, a feedforward reconstruction network based on DINOv2 improvement is constructed. Multi-modal adaptation is performed on a backbone network input layer, and decoupled high-level semantic features are explicitly injected by using a feature-level linear modulation mechanism, so that the geometric accuracy of automatic driving scene reconstruction is improved. On the basis of a three-dimensional scene of feedforward reconstruction, a distillation strategy is adopted, and the generation capability of a 2D diffusion model is migrated to a 3D Gaussian field. According to the method, the phenomena of object deformation, disappearance and the like in the generated data are effectively avoided, and long-tail weather data with geometric consistency of rain, snow and the like can be generated through the text instruction.

Methods and systems are disclosed for performing real-time deforming operations. The system receives an image that includes a depiction of a real-world object. The system applies a machine learning model to the image to generate a warping field and segmentation mask, the machine learning model trained to establish a relationship between a plurality of training images depicting real-world objects and corresponding ground-truth warping fields and segmentation masks associated with a target shape. The system applies the generated warping field and segmentation mask to the image to warp the real-world object depicted in the image to the target shape.

The invention relates to a monocular video three-dimensional human body high-quality reconstruction method based on Gaussian splashing and normal perception, and belongs to the field of computer aided design, graphics and computer vision. The method comprises the following steps: firstly, based on a monocular video frame in an input three-dimensional human body data set, guiding posture deformation through a time sequence, and carrying out whole body correlation deformation on a standard space posture to obtain an observation space human body posture; secondly, through normal perception Gaussian optimization, adaptive Gaussian density control and human body normal mapping supervision are carried out on the human body posture in the observation space, and a human body model of Gaussian representation is obtained; then, carrying out human body shadow feature learning on the input human body normal map by adopting an illumination model to obtain human body shadow features; and finally, light and shadow enhancement human body rendering is performed on the Gaussian representation human body model in combination with the human body light and shadow features and human body color information input into the monocular video, a human body rendering effect is output, and the quality and reality sense of three-dimensional human body reconstruction of the monocular video are improved.

Methods and systems are disclosed for performing real-time deforming operations. The system receives an image that includes a depiction of a real-world object. The system applies a machine learning model to the image to generate a warping field and segmentation mask, the machine learning model trained to establish a relationship between a plurality of training images depicting real-world objects and corresponding ground-truth warping fields and segmentation masks associated with a target shape. The system applies the generated warping field and segmentation mask to the image to warp the real-world object depicted in the image to the target shape.

The invention provides a semantic segmentation method based on an HDR dynamic neural radiation field, and relates to the technical field of image segmentation. According to the semantic segmentation method based on the HDR dynamic neural radiation field, the segmentation capability of a two-dimensional visual large model on pictures and videos is utilized, the segmentation capability of the two-dimensional model is expanded to four dimensions, a hyper-parameter controlled tone mapping module is constructed, mapping from an HDR view to LDR is controlled, and the segmentation capability in an HDR scene is greatly improved; a mask deformation field network and a dynamic mask inverse rendering method are provided, expensive labeling does not need to be performed on a four-dimensional scene, and any object in the four-dimensional dynamic scene can be segmented by using a two-dimensional picture mask.

The invention relates to a method for reducing motion blur of an erf model by using events and frames, and belongs to the technical field of computer vision. The method comprises the following steps: converting an event stream into triple representation; generating a bidirectional optical flow field based on the event flow, and aligning the fuzzy frame and the event flow through event-guided deformation convolution; constructing a double-branch optimization framework; the double-branch features are dynamically fused through frequency domain attention gating, and the weight of the frequency domain attention gating is generated by event frequency spectrums through MLP; generating a pseudo-true value based on an event double integral model, aligning an event spectrum and rendering a high-frequency component, and carrying out chain constraint on the continuity of a cross-frame radiation field through event brightness change; event-frame non-linear response differences are modeled by a learnable camera response function. Clear reconstruction and rendering of the moving target in a complex dynamic scene are realized.

The invention discloses a semantic segmentation method based on phase contour constraint, and belongs to the technical field of computer vision. The method comprises the following steps of: acquiring a two-dimensional texture image and a deformed stripe image of a scene to be detected in the same view field by using a calibrated camera-projector system; resolving the deformed fringe image through a multi-step phase shift and Gray code technology to obtain absolute phase distribution, and calculating a gradient field; according to a preset sudden change condition, extracting a physical edge contour reflecting the depth change of the object from the gradient field; and mapping the physical edge contour into the two-dimensional texture image based on a pixel coordinate corresponding relation to accurately define a semantic object region, endowing a corresponding category label, and finally generating a semantic label mask image. According to the method, the uncertainty of the visual texture edge is corrected by utilizing the physical phase information, the problem of edge extraction in a complex illumination scene is effectively solved, automatic generation of semantic segmentation data is realized, and the training precision and robustness of a semantic segmentation model are improved.

The invention discloses a method for improving the quality of a space-time domain compressed video by using a dense network, and the method mainly comprises the following steps: firstly, inputting a low-quality video obtained by HEVC compression into a network, aligning a certain frame to be enhanced with three adjacent frames before and after the frame in a universal deformable convolution mode, and carrying out the reconstruction of the frame to be enhanced; extracting an initial space-time fusion feature; extracting spatial feature information containing more levels from a to-be-enhanced frame by using the dense structure; the initial space-time fusion features and the spatial features are input into a space-time information adaptive fusion module together, and optimized space-time information is obtained through a series of introduced attention mechanisms and convolution layers; and finally, sending to a reconstruction module, obtaining an optimal residual error through a network, and adding the optimal residual error with a compressed frame to obtain a reconstructed video frame. Experimental results show that the method can effectively suppress the compression effect of the video, improve the video quality and obtain a better visual effect.

The invention discloses a real-time software deformation interaction method based on a physical engine in an environment. The real-time software deformation interaction method comprises the following steps: S1, establishing a virtual environment and a software model; s2, constructing an interactive perception and collision detection network; s3, receiving and analyzing a multi-modal interaction instruction; s4, real-time deformation calculation based on a physical engine; s5, performing multi-user collaborative state synchronization and rendering feedback; and S6, carrying out fault state simulation on the software object. According to the method, real-time and high-precision simulation of software deformation is realized through deep fusion of software dynamics calculation of a physical engine and an interaction instruction stream, and through hierarchical physical attribute binding and client prediction, the network synchronization data volume is greatly reduced while the millimeter-level deformation precision is ensured, so that the network synchronization efficiency is improved. And low-delay and high-consistency experience under multi-person cooperative operation is ensured.

A method for generating photorealistic 4D scenes from text inputs is disclosed. The method utilizes a text-to-video diffusion model to generate a reference video and a freeze-time video. A canonical 3D representation is reconstructed using deformable 3D Gaussian Splats (D-3DGS) based on the freeze-time video. Temporal deformations are learned to capture dynamic interactions in the reference video. The method employs a novel Score Distillation Sampling strategy combining multi-view and temporal aspects to enhance consistency and robustness. The resulting 4D scenes feature multiple objects interacting with detailed background environments, viewable from different angles and times. The method enables flexible camera control and integration with augmented and virtual reality applications. Some examples include features such as image-to-4D generation.

The invention discloses a registration fusion algorithm of a multi-modal image, which relates to the technical field of imaging and comprises the following steps: S1, data preprocessing-motion compensation; according to the method, displacement and deformation caused by physiological movement are inhibited through data preprocessing; the problem of feature extraction in a complex scene is solved through multi-level feature extraction, and uncertainty is reduced; the matching efficiency and precision are improved based on local sensitive hashing and mutual information feature matching; accurately describing non-rigid deformation based on a free deformation model of a B-spline; in the registration fusion link, images are accurately aligned based on a deformation model, and a weighted average fusion strategy is adopted; the registration fusion quality is ensured by a multi-dimensional evaluation system; the steps cooperate with each other to solve the problems of uncertainty of feature extraction and matching in local deformation and non-rigid registration, difficulty in feature extraction caused by local deformation, matching error caused by non-rigid deformation, limitation of non-rigid registration effect by inaccurate feature extraction, and interference of deformation estimation by matching uncertainty in the existing algorithm.

The disclosed computer-implemented method may include receiving, by a computing device, multi-view flat-lit performance data of a subject. Additionally, the method may include rendering, by the computing device, a dynamic sequence of novel-view flat-lit images of the subject based on a deformable three-dimensional Gaussian splatting (3DGS) model. The method may also include providing the rendered dynamic sequence of flat-lit images as input to a diffusion-based relighting model trained on the multi-view flat-lit performance data of the subject. Furthermore, the method may include generating, by the computing device using the diffusion-based relighting model, a relit sequence of the subject under a specified lighting condition. Various other methods, systems, and computer-readable media are also disclosed.