894 results about "Morphing" patented technology

The invention relates to the technical field of 3D printing models, in particular to a 3D printing model slice feature compensation method and system and a medium. The method comprises the following steps that a 3D printing model is collected, a model feature skeleton is split, transverse slice features are generated, then a 3D printing environment is collected, virtual simulation is carried out, environment stress analysis is carried out on the skeleton slice features to obtain environment stress data, then deformation scale deduction and thermal deformation prediction are carried out based on the environment stress data, and a thermal deformation model is obtained. On the basis, reverse deformation simulation is carried out, deformation repair requirements are evaluated, a deformation compensation path is planned, finally, compensation simulation of a model feature skeleton is achieved, non-target deformation is analyzed, a compensation path is adjusted, and therefore the feature compensation method is optimized. According to the 3D printing model slice feature compensation method, the accuracy and quality of the 3D printing model are integrally improved.

The invention relates to a deformable object interactive operation control method based on a visual touch-language-action multi-mode model, and the method comprises the steps: carrying out the image, touch and language data coding of a deformable object, and obtaining visual, touch and language features; performing cross-modal feature alignment processing on the visual features, the tactile features and the language features to obtain multi-modal fusion features; inputting the multi-modal fusion features into a large model for environment understanding; iteratively performing action planning and execution by adopting a'thinking-decision 'planning mode; and repeatedly executing the steps until the current deformable object interaction operation task is completed. Compared with the prior art, the multi-modal feature alignment capability, the action planning precision and the task adaptability are improved, efficient recognition and interaction of the robot on the deformable object can be achieved, deformation and state changes of the object can be effectively dealt with especially in a complex environment, the operation strategy can be dynamically adjusted, and the operation efficiency is improved. And more intelligent and accurate deformable object operation is realized.

The present disclosure is directed toward systems, methods, and non-transitory computer readable media for generating and providing an intelligent assistant interface that integrates with a large language model and a knowledge graph to adaptively change its appearance for presenting and interacting with different content items from various sources. In some embodiments, the disclosed systems provide an intelligent assistant interface that includes a set of interface elements selectable to interact with a large language model. Based on an input intent, the disclosed systems can utilize the large language model to analyze a knowledge graph for accessing and / or generating content items for display within the intelligent assistant interface.

In particular embodiments, a computing system may receive a video comprising a plurality of image frames. The system may generate, for each image frame in the video, an initial depth map using a machine-learning model. The system may compute a misalignment error indicating depth misalignments in initial depth maps using a reprojection technique. The system may generate, for each image frame in the video, an optimized camera pose and a flexible deformation spline associated with the image frame to minimize the misalignment error. The system may generate, for each image frame in the video, a refined depth map by adjusting the initial depth map associated with the frame using the flexible deformation spline associated with the image frame.

The invention discloses a multi-mode driven virtual digital human face animation generation method and system, and relates to the field of computer graphics, and the method comprises the steps: obtaining voice input and text input, and extracting voice features and text features; dynamically fusing the two features through an attention fusion model to generate facial expression and head posture control parameters, and dynamically adjusting contribution weights of a voice mode and a text mode to the control parameters by adopting a driving strategy of differentiation of facial upper expression and facial lower expression; and performing local deformation on the virtual digital human face image based on the control parameter to generate an initial animation frame, and performing refining processing on the initial animation by using a generative adversarial network to obtain a refined face animation. By means of the technical scheme, the natural and vivid virtual human face animation matched with the voice content and the text semantics can be generated.

The invention provides a method for generating a harmonious object in a three-dimensional scene based on physical rendering and uncertainty estimation, and the method comprises the steps: giving a three-dimensional scene, and generating the text description of a target object and a rectangular frame of the position of the target object in a 2D rendering view of the three-dimensional scene; according to the text description of the generated target object, a multi-view image generation diffusion model is utilized to construct a deformable tetrahedral mesh of the geometric appearance of the target object; constructing a feature network comprising a material item prediction module and an uncertainty item prediction module; obtaining a rendered picture of the target object by using a physical rendering technology, and performing training optimization on the feature network; and obtaining a multi-view scene rendering dense view containing the target object and a 3D model file according to the three-dimensional scene, the deformable tetrahedral mesh and a material item output by the trained feature network. According to the method, the problem that the target object and the scene are not harmonious in fusion is effectively solved, and the reality sense and the visual effect of the target object in the scene are enhanced.

The embodiment of the invention provides a character animation generation method and device based on voice driving, equipment and a medium, and belongs to the technical field of financial science and technology. The method comprises the following steps: acquiring an audio signal input by a user, a target human portrait picture and a target scene reference video; encoding the audio signal to obtain an audio code; generating a three-dimensional face model motion coefficient according to the portrait picture of the target person and the audio code; extracting blink information and head posture information from the target scene reference video; correcting the motion coefficient of the three-dimensional face model according to the blinking information and the head posture information to obtain a corrected motion coefficient of the three-dimensional face model; and taking a three-dimensional deformable head model as a potential vector, mapping the three-dimensional deformable head model to the corrected three-dimensional face model motion coefficient frame by frame, converting the three-dimensional face model motion coefficient into a video frame, and generating a head speech video of the target person. According to the embodiment of the invention, the quality and reality of character animation generation driven by voice can be improved.

Disclosed herein are methods, systems, and aircraft for performing image analysis for aiding refueling operations. A method includes a method includes receiving a two-dimensional (2D) image from a camera of a first device, determining 2D keypoints of a second device located within the 2D image based on a predefined point model of a generalized version of the second device, determining a 6 degree-of-freedom (6DOF) pose of the second device using the 2D keypoints, and outputting an estimated position of at least a component of the second device based on the 6DOF pose.

The invention relates to a multi-modal image registration method based on modal alignment distillation, and belongs to the technical field of image processing. The method comprises the steps of adopting a distillation learning guide type training process, performing guide type supervision on a multi-modal registration student model by using intermediate features of a same-modal registration teacher model with excellent performance and an output deformation field, migrating modals of the intermediate features of the student model, and effectively reducing modal differences, a high-quality deformation field close to teacher model estimation is output, and the multi-modal registration performance is improved; a deformable mask training strategy is adopted to deal with the problem that a registration result is inconsistent with a label image boundary caused by a black edge effect of spatial transformation in a registration process (namely, a black edge of the registration result and an original boundary of the label image), so that the negative influence on registration is effectively reduced, and the registration performance of the model is improved. According to the invention, a multi-modal image registration task can be realized, registration challenges caused by modal differences are effectively solved, and the effect of multi-modal image space alignment is achieved.

The invention discloses an immersive video enhancement method and device based on frequency domain boundary collaborative optimization, and relates to the field of video processing, and the method comprises the steps: obtaining a to-be-reconstructed compressed multi-view texture and depth video sequence, and inputting the sequence into a trained immersive video enhancement model; a current to-be-enhanced video frame passes through a feature extraction module, and high-frequency features and low-frequency features are extracted respectively; the high-frequency features and the low-frequency features pass through a frequency domain enhancement module to obtain a frequency domain enhanced image; inputting the frequency domain enhancement image and the current video frame to be enhanced into a boundary enhancement module to obtain a fused image; and inputting the fused image and an adjacent video frame of the current to-be-enhanced video frame into a space-time deformable convolution module to obtain an aligned fused image, and performing prediction on the aligned fused image through a quality enhancement module to obtain an enhanced residual error and generate a corresponding reconstructed video. According to the method, the problems of compression artifacts, boundary artifacts, low quality of immersive videos and the like are solved.

Methods and computer systems for methods, computer systems, and computer-readable memory media for determining a warp function. An as-designed watertight spline model of an object is received. A set of sweep points along a sweep trajectory for manufacturing the object are used to determine a sweep surface. Model points are sampled from the as-designed watertight spline model and projected onto nearest as-executed points on the sweep surface. The projected as-executed points are used to construct an as-executed model of the object. A warp function is determined based on a difference between the as-designed watertight spline model and the as-executed model. The warp function is a continuous function approximating differences between the as-designed model and the as-manufactured object. The warp function is stored in a non-transitory computer-readable memory medium.

In one embodiment, a method includes rendering for displays of an extended-reality (XR) display device a first output image of an XR assistant avatar within an environment in a field of view of a first user of the XR display device, wherein the XR assistant avatar has a first form and a first pose with respect to the first user and is interactable by the first user to access an assistant system, detecting a change in a context of the first user with respect to the environment, determining whether to morph the XR assistant avatar based on the detected change in the context of the first user, and rendering a second output image of the XR assistant avatar for the displays of the XR display device, wherein the XR assistant avatar is morphed to have a second form and a second pose with respect to the first user.

One or more machine learning models (MLMs) may learn implicit 3D representations of geometry of an object and of dynamics of the object from performing an action on the object. Implicit neural representations may be used to reconstruct high-fidelity full geometry of the object and predict a flow-based dynamics field from one or more images, which may provide a partial view of the object. Correspondences between locations of an object may be learned based at least on distances between the locations on a surface corresponding to the object, such as geodesic distances. The distances may be incorporated into a contrastive learning loss function to train one or more MLMs to learn correspondences between locations of the object, such as a correspondence embedding field. The correspondences may be used to evaluate state changes when evaluating one or more actions that may be performed on the object.

The invention provides a digital human body modeling system and method based on space-time consistency Gaussian sputtering, and the system comprises a Gaussian set construction module which is used for constructing an initial Gaussian set based on a collected monocular RGB video sequence of human body motions; the non-rigid deformation module is used for updating a pre-constructed standard Gaussian set by adopting a Gaussian decoder based on the initial Gaussian set to obtain a dynamic Gaussian set containing non-rigid deformation; the space-time constraint module is used for applying space-time consistency constraint to the dynamic Gaussian set to obtain the dynamic Gaussian set of the space-time consistency constraint; the rigid deformation module is used for obtaining an attitude-driven Gaussian set based on the rigid deformation of the linear binding skeleton; and the projection rendering module is used for rendering the attitude-driven Gaussian set into a Gaussian image under the current frame attitude. According to the method, a space-time consistency mechanism of motion perception is combined, and the consistency problem of non-rigid deformation in a monocular video is effectively solved.

The invention discloses a plug-flow volume video rendering method and device based on an image grid, a chip and a medium. The method comprises the following steps: S1, acquiring a picture sequence of a dynamic human body from a plurality of visual angles; s2, performing Gaussian splash reconstruction on the first frame of each group by using the point cloud; then modeling is carried out on deformation generated by Gaussian splashing along with time change, and a Gaussian splashing model is obtained; s3, applying a residual entropy loss constraint and a time sequence loss constraint, and performing appearance optimization; and S4, splashing and baking the 3D gauss into a 2D format, and then compressing to form a video. According to the method, dynamic 3D Gaussian splash is regarded as a 2D Gaussian video, motion and appearance attributes are simulated respectively through grouping training from multi-view RGB video input, and finally, a volume video capable of being transmitted in a streaming manner is constructed, so that high-quality dynamic character rendering can be realized on mobile equipment, a high-quality rendering result is realized, and the dynamic character rendering efficiency is improved. Meanwhile, the method is superior to the prior art in quality, rendering speed and storage, and has a wide application prospect.

The invention relates to a dynamic scene reconstruction method and device, equipment and a medium, and the method comprises the steps: obtaining a picture sequence; the image sequences are input into a dynamic scene reconstruction model, a dynamic scene between the image sequences is generated, and the dynamic scene reconstruction model comprises a 3D Gaussian sputtering module used for conducting preliminary scene reconstruction on the image sequences to obtain static scene Gaussian; the dynamic component module is used for generating the variable quantity of the position of the static scene Gaussian according to the position attribute of the static scene Gaussian in combination with the time characteristic; and the color component module is used for generating the variable quantity of the Gaussian color attribute of the static scene by combining the Gaussian motion deformation characteristics according to the Gaussian color attribute of the static scene. According to the invention, a real-time rendering function can be realized on the basis of ensuring high-fidelity scene reconstruction.

The invention relates to the technical field of computers, and relates to a rendering model training method and device, an illumination rendering method and device, a computer program product and electronic equipment. The training method comprises the following steps: extracting inherent attribute information and global light and shadow information of a to-be-rendered object from an image frame sample based on a teacher model; performing feature decoding on the Gaussian primitives by using an initial student model to obtain a basic physical attribute, and determining a first loss according to the basic physical attribute and the inherent attribute information, the initial student model comprising a plurality of Gaussian primitives attached to a deformable grid; interpolating light and shadow data of the probe model through the initial student model to obtain current light and shadow information, constructing second loss according to the current light and shadow information and global light and shadow information, and arranging the probe model on the surface of the deformable grid; and adjusting parameters in the initial student model according to the first loss and the second loss, and adjusting shadow data of the probe model to obtain a target student model and a target probe model.

The invention discloses a grid sequence driven two-dimensional face animation generation method and device, equipment and a medium. The method comprises the steps of obtaining a 2D face image needing to be driven; the initial frame of the grid sequence motion and the 2D face image are bound by using Gaussian splash expression, the topology of the 3D grid and the identity information of the 2D face are bound to a Gaussian model, and an initial Gaussian head portrait is generated; representing motion of the grid sequence on Gaussian expression by combining a deformation field, optimizing a Gaussian head portrait, and generating an animation video sequence through rendering; wherein the 2D replay network processes an input reference face image and a grid current rendering frame as conditions, a gradient is generated, and the posture and the features of the head portrait are optimized, so that the feature height of the generated animation is consistent with the feature height of the reference face image. According to the method, the rendering cost and complexity can be effectively reduced by introducing the medium representation method based on Gaussian splashing and combining the Gaussian deformation field optimization method guided by two-dimensional prior and gradient.

Described herein are methods and non-transitory computer-readable media of a computing system configured to obtain a plurality of images of an object from a plurality of orientations at a plurality of times. A machine learning model is encoded to represent a continuous density field of the object that maps a spatial coordinate to a density value. The machine learning model comprises a deformation module configured to deform the spatial coordinate in accordance with a timestamp and a trained deformation weight. The machine learning model further comprises a neural radiance module configured to derive the density value in accordance with the deformed spatial coordinate, the timestamp, a direction, and a trained radiance weight. The machine learning model is trained using the plurality of images. A three-dimensional structure of the object is constructed based on the trained machine learning model.

To compute an image of a dynamic 3D scene comprising a 3D object, a description of a deformation of the 3D object is received, the description comprising a cage of primitive 3D elements and associated animation data from a physics engine or an articulated object model. For a pixel of the image the method computes a ray from a virtual camera through the pixel into the cage animated according to the animation data and computes a plurality of samples on the ray. Each sample is a 3D position and view direction in one of the 3D elements. The method computes a transformation of the samples into a canonical cage. For each transformed sample, the method queries a learnt radiance field parameterization of the 3D scene to obtain a color value and an opacity value. A volume rendering method is applied to the color and opacity values producing a pixel value of the image.

The invention relates to a scene text detection method and device based on a VMama framework and a channel-space attention mechanism and computer equipment. The method comprises the following steps: acquiring a scene image reference sample data set and an enhanced sample data set containing texts; constructing a deformable VMama feature encoder by adopting deformable convolution based on a VMama framework, and obtaining a multi-scale text instance feature map corresponding to each scene image; adopting interpolation and maximum pooling methods to obtain a balance feature map; constructing a GASM module based on a channel-space attention mechanism to obtain text instance features after pixel-by-pixel enhancement; constructing a Fourier feature decoder to obtain a reconstructed text instance contour; and training a scene text detection model which takes the enhanced sample data set as input and takes the text instance contour as output by adopting a loss function. By adopting the method, scene text detection with a global receptive field, rapid convergence and accurate positioning can be realized.

The invention discloses a video virtual fitting framework in a real scene based on a controllable image diffusion model. The video virtual fitting framework comprises a data preprocessing module used for constructing feature expressions of figures and clothes; the single-stage image fitting network is used for generating a virtual fitting image based on the preprocessed data and performing implicit deformation and fusion of clothes by using a cross attention mechanism; the diffusion guiding module is used for enhancing the time sequence consistency of the generated video, a self-supervision model Video MAE and a self-supervision image model DINO-V2 are introduced, the video is masked by the self-supervision model Video MAE, and a complete video is reconstructed through inter-frame information; and the long video generation module is used for generating a long video and keeping the consistency of the video, splitting the complete video into a plurality of short video clips, and reducing the jitter at the clip joints by using a time sequence common noise reduction strategy. The diffusion guiding module is designed, efficient time sequence consistent video virtual fitting is achieved through self-supervised learning, and the method has the advantages of being excellent in shielding processing, stable in long video generation and the like.

The invention provides a digital twinborn model construction method and device for a hinge object, a terminal and a medium, and the method comprises the steps: constructing a three-dimensional Gaussian splashing model in an initial state through employing a first multi-view image set of the hinge object in the initial state, and identifying an initial movable Gaussian model based on a deformed three-dimensional Gaussian splashing model predicted by the deformation network, and realizing complete self-supervised optimization of the deformation network based on a second multi-view image set of the hinge object in an end state and the deformed model. And jointly optimizing the three-dimensional Gaussian splashing model, re-segmenting the initial movable Gaussian model to obtain a target movable Gaussian model, and acting the estimated initial hinge motion parameters on the initial movable Gaussian model to optimize the motion parameters to obtain target hinge motion parameters, so as to construct digital twinning of the hinge object. According to the method, digital twinning of the hinge object can be accurately reconstructed through the sparse state of the hinge object in a complete self-supervision mode, and the hinge logic is estimated with high accuracy.

The invention relates to a polarization dislocation code fused phase shift coding method and system, and the method comprises the steps: obtaining calibration parameters of a camera and a projector, fusing a phase shift stripe pattern with a dislocation Gray code stripe pattern through the polarization characteristics of the projector, and obtaining a polarization dislocation fused phase shift stripe pattern; obtaining a deformed polarization dislocation fusion phase-shift stripe pattern after the projector projects the polarization dislocation fusion phase-shift stripe pattern to the surface of an object to be measured, extracting a phase-shift stripe pattern and a dislocation Gray code stripe pattern from the deformed polarization dislocation fusion phase-shift stripe pattern, and calculating a wrapped phase by using the extracted phase-shift stripe pattern, carrying out threshold processing on the staggered Gray code stripe pattern to obtain a binary Gray code pattern; and performing phase unwrapping on the wrapped phase through the binarization Gray code pattern, and performing three-dimensional reconstruction on the to-be-measured object through calibration parameters of the camera and the projector according to the unwrapped phase. According to the method disclosed by the invention, measurement errors caused by an underexposure or overexposure phenomenon are inhibited, and the measurement precision and the three-dimensional reconstruction efficiency are improved.

The invention discloses a video super-resolution method and system, electronic equipment and a storage medium, and the method comprises the steps: building a video super-resolution model in a complex motion scene, and training and testing the model, the model comprises a feature extraction sub-module, a cross-frame alignment sub-module, a feature enhancement sub-module and an up-sampling reconstruction sub-module, the feature extraction sub-module is used for extracting shallow features of input continuous video frames; the cross-frame alignment sub-module performs feature level alignment on different frames by using deformable convolution to realize cross-frame spatio-temporal information fusion and motion compensation; the feature enhancement sub-module is used for enhancing the features; and the up-sampling reconstruction sub-module amplifies the resolution of the enhanced features output by the feature enhancement sub-module, and reconstructs the enhanced features after resolution amplification into video frames. According to the invention, the super-resolution effect of the video in the complex motion scene is improved, and the quality and visual effect of the video are enhanced.

The invention provides a texture mapping method, a texture mapping device, a computer storage medium and electronic equipment, and relates to the technical field of computer graphic processing. The method comprises the following steps: acquiring a world space coordinate and a world space normal of a vertex of a target model; performing texture projection on the world space coordinates on a plurality of world coordinate axes to obtain target texture coordinates of the vertexes of the target model on the world coordinate axes; determining a texture weight value based on the included angles between the world space normal and the plurality of world coordinate axes; and carrying out texture sampling according to the target texture coordinate to obtain an initial map sampling result, and carrying out mixed sampling on the initial map sampling result and the texture weight value on the corresponding world coordinate axis to carry out texture mapping on the to-be-processed model based on the target map sampling result. The method can get rid of dependence on a model topological structure and UV expansion of a three-dimensional model, and realizes a more real, natural, accurate and flexible texture fitting effect without deformation stretching.

The invention provides a dynamic nerve radiation field anti-aliasing reconstruction method based on three planes and multistage textures, which comprises the following steps: inputting a training video frame to be reconstructed, projecting a light cone to each pixel of the training video frame in an observation space, and sampling an inscribed ball on the axis of the light cone to obtain a sampling ball; mapping a sampling ball of the observation space to a standard space by using a dynamic deformation network; the standard space is represented by using multi-level three-plane textures, sampling balls in the standard space are respectively projected to three planes to obtain three corresponding discs, features on corresponding multi-level texture planes are respectively extracted from each disc and are spliced, and multi-level three-plane texture features are obtained; and inputting a to-be-reconstructed video frame into the dynamic deformation network and the multi-level texture three-plane, inputting the obtained multi-level three-plane texture features and the specified view angle into the static neural radiation field network for neural rendering, and reconstructing a rendered image of the to-be-reconstructed video frame under the specified view angle.

An example system include one or more memories and one or more processors coupled to the one or more memories. The one or more processors are configured to perform a first segmentation operation on a previous frame of image data to generate first segmentation data. The one or more processors are configured to perform a deformable convolution operation based on the first segmentation data to generate a deformable convolution output. The one or more processors are configured to perform a second segmentation operation on a current frame of image data to generate second segmentation data. The one or more processors are configured to combine the deformable convolution output with the second segmentation data to generate third segmentation data and control operation of a device based on the third segmentation data.

The invention discloses a deep watermarking method for immune geometric distortion, and the method comprises the steps: firstly constructing a watermark model in combination with a Swinin-Transform and a deformable convolutional network (DCN), and enabling the watermark model to comprise a watermark encoder and a watermark decoder; simulating styles of different types of non-geometric distortion by using a style migration network to form a distortion style combined noise layer; and applying the distortion style combined noise layer to the training of the constructed watermark model, enhancing the multiple robustness of the watermark model to various non-geometric distortions, and improving the generalization ability of the robustness. According to the method, the image copyright protection and image leakage prevention functions can be realized while the watermark embedding is ensured not to influence the image quality, so that the intellectual property and security protection of the image can be enhanced.

The invention discloses a video deblurring method based on a deformable space-time sparse converter. The method comprises the following steps: carrying out feature extraction on continuous n frames of fuzzy videos, and calculating forward and backward light streams and corresponding fuzzy images; iteratively updating features of each frame through a bidirectional feature propagation module guided by a multi-scale fuzzy image, and aggregating feature information of different propagation branches; a deformable space-time sparse Transform module is adopted to carry out refinement processing on the aggregation features; and inputting the refined features into a decoder module, and generating a final deblurring result in combination with original input residual connection. The method is suitable for scenes of security monitoring, mobile equipment shooting and the like, the effect of removing the dynamic fuzzy area in the video can be improved, and the overall visual quality of the video is effectively improved.