Information Synthesis Model for Seamless Video Integration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video synthesis methods often result in defects due to splicing, which affects the reality feel of the synthesized video.
Innovation Solution
A model training method that generates a content mask set by extending the region identified by brief-prompt information, allowing the information synthesis model to expand the boundary appropriately and avoid splicing defects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If brief prompt information is directly used to generate the region and splice with background, then the synthesis process is simple and fast, but splicing defects occur and reality feel deteriorates
Solution Approach 1:
The patent applies preliminary action by pre-generating content masks that extend beyond the target object boundaries before the actual synthesis process. These pre-prepared masks with extended regions are stored and ready for use, allowing the model to seamlessly integrate target objects with backgrounds without real-time splicing operations, thus maintaining both speed and quality
Solution Approach 2:
The patent introduces content masks as an intermediary element between the target object and background. These masks with extended regions act as a mediator that facilitates smooth transitions and seamless integration, eliminating direct splicing defects while preserving synthesis efficiency through the trained information synthesis model
2Manufacturing precision
If the region is tightly fitted to the target object, then the integration is precise, but splicing defects appear at the boundaries
Solution Approach 1:
The patent applies local quality by creating content masks with different regional characteristics - the extended regions beyond the target object boundaries have different properties than the core target region. This allows the model to handle boundary areas differently, ensuring seamless integration while maintaining precise integration within the target object boundaries
Solution Approach 2:
The content masks are pre-generated with extended regions that anticipate potential boundary issues before synthesis occurs. This preliminary preparation ensures that when the target object is integrated with the background, the extended mask regions provide a buffer zone that prevents splicing defects at boundaries
Data Source
AI summary
A model training method includes obtaining an image sample set and brief-prompt information; generating a content mask set according to the image sample set and the brief-prompt information; generating a to-be-trained image set according to the content mask set; obtaining, based on the image sample set and the to-be-trained image set, a predicted image set through a to-be-trained information synthesis model, the predicted image set comprising at least one predicted image, the predicted image being in correspondence to the image sample; and training, based on the predicted image set and the image sample set, the to-be-trained information synthesis model by using a target loss function, to obtain an information synthesis model.