AI-driven local missing element frame compensation method

Abstract

The invention discloses an AI-driven local missing element frame supplementing method. The method comprises the steps of collecting and preprocessing multi-modal data including text data, image data and video data; constructing a multi-modal knowledge system, performing directional feature extraction on multi-modal data, converting the multi-modal data into associable structured data, realizing feature archiving and semantic association through an agent workflow, and establishing a retrieval engine in combination with an RAG system; constructing an AI frame compensation model, generating missing region frame compensation content through feature constraints of a multi-modal knowledge system, and performing local frame compensation on the frame compensation content generated by a large model and an original image; performing cloud rendering on data obtained after frame filling of the AI frame filling model, and outputting results in multiple forms according to application scene requirements; according to the method, a structured knowledge system is constructed, so that the complement frame content is not only visually coherent, but also highly conforms to a real scene in the aspects of historical background, age features, semantic logic and the like, and the historical accuracy and credibility of a repair result are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and more specifically, to an AI-driven method for filling in missing elements in frames. Background Technology

[0002] As a vital visual medium carrying historical imprints, scene information, and emotional memories, the integrity and sequential coherence of images are crucial for the accuracy and completeness of information transmission. However, in the actual preservation and application process, two prominent problems severely restrict the full realization of the value of images. On the one hand, due to the limitations of early photographic technology, the physical characteristics of paper media, and the effects of long-term preservation environments, many old photographs suffer from severe damage, specifically manifested as worn edges, stains covering local areas, and missing key details of people or scenes. These defects not only destroy the visual aesthetics of the photographs but also cause a break in the historical and emotional information they carry. On the other hand, photographs often exist in sequence, with multiple related photographs jointly constructing a complete visual narrative or information chain, such as showcasing the entire streetscape, the flow of activities, or the structure of buildings.

[0003] Faced with the prominent problems of incomplete old photographs and discontinuous related photographs, efficient technical solutions require powerful computing support. Chinese patent ZL201310051128.6 discloses a method and cloud service system for 3D graphics rendering based on a cloud computing architecture. This technology uses block rendering to dynamically allocate rendering resources, effectively balancing rendering quality and real-time performance. Chinese patent ZL201310051044.2 discloses a method and cloud service system for real-time rendering of 3D digital models based on a cloud computing architecture, employing edge computing technology to enable real-time preview of restoration effects. Both patented technologies are based on cloud computing architecture, achieving centralized processing of computing tasks through high-performance graphics server clusters in the cloud, with the terminal only handling lightweight interactive tasks, successfully overcoming the limitations of local hardware computing power.

[0004] While existing cloud computing rendering technologies demonstrate significant advantages in computing power and real-time performance for 3D model processing, they still have obvious shortcomings in specific scenarios such as partially filling in missing parts of old photos and interpolating frames between related photo sequences. Existing technologies focus more on balancing rendering efficiency and quality of 3D graphics, lacking a precise understanding of local semantic information in images and the ability to generate content, making it difficult to reasonably fill in missing details in old photos and generate natural frames at gaps between related photos.

[0005] Based on this, in order to solve the problems of incomplete old photos and discontinuity in related photos more accurately and efficiently, an AI-driven method for locally missing elements to be framed is proposed. Summary of the Invention

[0006] The purpose of this invention is to provide an AI-driven method for filling in missing elements in frames, overcoming the aforementioned deficiencies in the prior art.

[0007] The technical solution to achieve the objective of this invention is: an AI-driven method for interpolating locally missing elements, comprising:

[0008] Collect and preprocess multimodal data, including text, image, and video data;

[0009] A multimodal knowledge system is constructed, and the multimodal data is subjected to targeted feature extraction, which is transformed into associative structured data. Feature archiving and semantic association are realized through intelligent agent workflow, and a retrieval engine is established in conjunction with the RAG system.

[0010] An AI frame interpolation model is constructed, which integrates the large model generation capabilities of text-to-image, image-to-image, and image-to-video. The model generates missing frame content through feature constraints of the multimodal knowledge system, and performs local frame interpolation between the frame content generated by the large model and the original image.

[0011] The data obtained after frame interpolation by the AI ​​frame interpolation model is rendered in the cloud, and the results are output in multiple forms according to the application scenario requirements.

[0012] Preferably or optionally, the feature archiving and semantic association achieved through the intelligent agent workflow specifically includes:

[0013] The classification model is invoked to automatically archive the extracted multimodal features, and the classification results are synchronized to the distributed storage system via the model context protocol.

[0014] Establish cross-modal semantic associations between text and visual data, map text feature vectors to image feature vectors, associate dynamic features in videos with static features in images, and form a closed-loop verification relationship between text definitions and visual samples.

[0015] Preferably or optionally, the establishment of a search engine in conjunction with the RAG system specifically includes:

[0016] By combining keyword retrieval and vector similarity matching, core keywords are extracted according to the AI ​​frame interpolation requirements to locate relevant text features, and based on the text features, semantically similar image visual features and video dynamic features are further retrieved;

[0017] When there are knowledge conflicts in the search results, the candidate knowledge is quantitatively scored based on multiple dimensions such as multi-source cross-validation, feature consistency analysis and historical data correlation, and the knowledge with the highest credibility is selected as the basis for frame supplementation.

[0018] Preferably or optionally, the multimodal knowledge system supports dynamic updates, specifically including:

[0019] The user-reported repair and correction data is transformed into new multimodal feature vectors, and the new feature vectors are archived into the corresponding text, image, video or classification nodes through a data classification process driven by the model context protocol.

[0020] After new features are added to the database, the intelligent agent is automatically triggered to reconstruct the semantic relationships between related knowledge, so as to realize the synchronous evolution of the knowledge system and the actual application needs, and continuously improve the accuracy and scene adaptability of subsequent AI frame interpolation.

[0021] Preferably or optionally, the large model generation capability includes:

[0022] The text-based image model is used to receive historical text descriptions retrieved by the RAG system and combine them with the style features of the target image to generate local scene detail materials that conform to historical authenticity.

[0023] The image-based image model is used to extend or repair missing areas based on the preprocessed target image and referencing multi-view image features in the knowledge base, ensuring that the supplemented frame content is consistent with the original image in terms of lighting, texture and viewpoint.

[0024] The image-based video model is used to generate multi-view dynamic frames with temporal continuity based on static images and video dynamic features in a knowledge base, providing dynamic frame sequence support for 3D scenes.

[0025] Preferably or optionally, the local frame interpolation process includes the following steps:

[0026] Image segmentation algorithms are used to identify damaged, occluded, or missing information regions in the target image, and a patching region mask is generated to determine the patching range and boundary features.

[0027] The intelligent agent workflow retrieves the corresponding text description, reference image, and video dynamic features from the multimodal knowledge base based on the semantic type of the frame-filling region, and schedules the corresponding generative large model to construct a differentiated frame-filling strategy accordingly.

[0028] The interpolated frames generated by each model are fused with the original image at the pixel level to ensure high visual consistency with the overall scene.

[0029] Preferably or optionally, the differentiated frame interpolation strategy is dynamically constructed based on the integrity status of the input image, performs intelligent decision-making, and is collaboratively generated by the text-generated image large model, the image-generated image large model, and the image-generated video large model.

[0030] Preferably or optionally, the differentiated frame interpolation strategy specifically includes:

[0031] When processing a single partially damaged image, the agent generates a requirement label based on the semantic type of the identified missing region, calls the multimodal knowledge base to retrieve historical knowledge from the same period, and sequentially drives the text-generated image model to generate semantic contours and the image-generated image model to perform pixel-level completion with consistent texture. After outputting multiple candidate repair schemes, the optimal result is selected based on style consistency and feature similarity.

[0032] When processing multiple related images, at least one of which is complete, the agent uses the complete image as a reference benchmark, extracts its key visual features and fuses them with historical background information, spatially registers the complete image with the damaged image through image alignment technology, and coordinates the scheduling of the graph-generated image model to ensure structural consistency and the text-generated image model to supplement historical semantics, thereby achieving highly reliable targeted repair.

[0033] When processing multiple incomplete related images, the agent initiates a multi-source knowledge fusion mode, which integrates historical documents, contemporaneous images, and local features of each image to construct a scene semantic framework. This drives the text-based image model to deduce overall content assumptions, and the image-based image model to fuse local features from multiple perspectives for pixel-based deduction. Based on historical authenticity and visual logic consistency, multiple generation schemes are quantitatively evaluated and optimized, thereby reducing the blindness of repair assumptions and improving the historical accuracy and visual rationality of the frame-padding results.

[0034] Preferably or optionally, the cloud rendering method includes:

[0035] The image data generated by AI frame interpolation is uploaded to a high-performance graphics server cluster in the cloud. A block rendering strategy is used to dynamically allocate computing resources to perform fine rendering of architectural details, lighting effects and dynamic elements. Combined with edge computing technology, the rendering results can be previewed in real time and interacted with by the user, and dynamic adjustment of scene perspective and detail presentation is supported.

[0036] Preferably or optionally, the multi-format output includes at least one or more of the following: 3D model files, immersive VR / AR content, and dynamic videos.

[0037] By adopting the above technical solution, the present invention has the following beneficial effects:

[0038] (1) This invention integrates multimodal data such as text, images, and videos to construct a structured knowledge system, so that the frame-filling content is not only visually coherent, but also highly consistent with the real scene in terms of historical background, era characteristics, and semantic logic, which significantly improves the historical accuracy and credibility of the restoration results.

[0039] (2) Based on the intelligent agent workflow, the system can dynamically select the optimal frame-filling strategy according to the type of missing region and the integrity status of the input image, such as a single image being damaged, partially complete, or completely incomplete. This enables intelligent processing of the entire process from semantic understanding to model scheduling, effectively solving the problems of "one-size-fits-all" or reliance on manual intervention in traditional methods.

[0040] (3) This invention has the ability to enhance the generation of multiple models collaboratively. The large models of text-generated image, image-generated image and image-generated video work collaboratively under the unified knowledge constraints, respectively undertaking the tasks of semantic generation, detail repair and temporal extension, taking into account both macro-level rationality and micro-level consistency, and achieving high-quality and high-fidelity local frame interpolation.

[0041] (4) The present invention combines a keyword and vector matching RAG retrieval engine with multi-source cross-validation and quantitative scoring mechanism to ensure the authority and consistency of the knowledge called and avoid frame distortion caused by knowledge conflict.

[0042] (5) This invention transforms corrected data into new features and automatically updates the knowledge base through a closed loop of user feedback, triggering the intelligent agent to reconstruct associations, enabling the system to continuously learn and adapt to new scenarios, and continuously improving the repair accuracy in long-term applications.

[0043] (6) The present invention adopts a cloud architecture that combines block rendering and edge computing to achieve real-time preview and interactive adjustment of high-precision three-dimensional scenes, and supports multiple output formats such as three-dimensional models, VR / AR content, and dynamic videos, meeting the needs of diverse application scenarios such as cultural heritage digitization, film and television restoration, and virtual reality. Attached Figure Description

[0044] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings, wherein...

[0045] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0046] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0047] Example 1, please refer to the appendix. Figure 1 An AI-driven method for interpolating missing elements in frames, with the following specific steps:

[0048] S1. Multimodal Input: Collect and preprocess multimodal data; provide comprehensive information support for subsequent knowledge construction and model frame completion; multimodal data includes text data, image data and video data.

[0049] S2. Constructing a multimodal knowledge system: Targeted feature extraction is performed on multimodal data, transforming it into associative structured data. Feature archiving and semantic association are achieved through an intelligent agent workflow. A retrieval engine is then established in conjunction with the RAG system; the RAG system is a retrieval enhancement generation mechanism. Details are as follows:

[0050] S2.1 The transformation of the data into associative structured data through targeted feature extraction is specifically described as follows: text feature extraction includes entities such as era, scene, and object, and attributes such as clothing, architecture, and lighting; image feature extraction includes entities such as era, scene, and damage type; video feature extraction includes spatial features such as era, scene, and damage type, as well as temporal features such as frame rate, scene dynamic intensity, and action mode.

[0051] S2.2, The feature archiving and semantic association achieved through the intelligent agent workflow includes:

[0052] The classification model is invoked to automatically archive the extracted multimodal features, and the classification results are synchronized to the distributed storage system via MCP (Model Context Protocol).

[0053] Establish cross-modal semantic associations between text and visual data, map text feature vectors to image feature vectors, associate dynamic features in videos with static features in images, and form a closed-loop verification relationship between text definitions and visual samples.

[0054] For example, the textual features of "Nanjing women's hairstyles in the 1930s" include height, 15-20cm, and coiled style, and are bound to the visual features of the corresponding photos and video frames to form a closed-loop verification of textual definitions and visual samples.

[0055] S2.3, The establishment of a search engine in conjunction with the RAG system specifically includes:

[0056] By combining keyword retrieval and vector similarity matching, core keywords are extracted according to the AI ​​frame interpolation requirements to locate relevant text features, and based on the text features, semantically similar image visual features and video dynamic features are further retrieved;

[0057] When there are knowledge conflicts in the search results, the candidate knowledge is quantitatively scored based on multiple dimensions such as multi-source cross-validation, feature consistency analysis and historical data correlation, and the knowledge with the highest credibility is selected as the basis for frame filling.

[0058] The multi-source cross-validation includes comparing authoritative literature and providing evidence from multiple perspectives; the feature consistency analysis includes determining the degree of matching between text semantics and visual features.

[0059] S2.4 The multimodal knowledge system supports dynamic updates, specifically including:

[0060] The user-reported repair and correction data is transformed into new multimodal feature vectors, and the new feature vectors are archived into the corresponding text, image, video or classification nodes through a data classification process driven by the model context protocol.

[0061] After new features are added to the database, the intelligent agent is automatically triggered to reconstruct the semantic relationships between related knowledge, so as to realize the synchronous evolution of the knowledge system and the actual application needs, and continuously improve the accuracy and scene adaptability of subsequent AI frame interpolation.

[0062] S3. Constructing an AI frame interpolation model: The AI ​​frame interpolation model integrates the large-scale model generation capabilities of text-to-image, image-to-image, and image-to-video. It generates missing region frame interpolation content through feature constraints of the multimodal knowledge system, and then performs local frame interpolation between the frame interpolation content generated by the large-scale model and the original image. Specifically:

[0063] S3.1 The large model generation capability includes:

[0064] The text-based image model is used to receive historical text descriptions retrieved by the RAG system and combine them with the style features of the target image to generate local scene detail materials that conform to historical authenticity.

[0065] The image-based image model is used to extend or repair missing areas based on the preprocessed target image and referencing multi-view image features in the knowledge base, ensuring that the supplemented frame content is consistent with the original image in terms of lighting, texture and viewpoint.

[0066] The image-based video model is used to generate multi-view dynamic frames with temporal continuity based on static images and video dynamic features in a knowledge base, providing dynamic frame sequence support for 3D scenes.

[0067] S3.2 The local frame interpolation process includes the following steps:

[0068] Image segmentation algorithms are used to identify damaged, occluded, or missing information regions in the target image, and a patching region mask is generated to determine the patching range and boundary features.

[0069] The intelligent agent workflow retrieves corresponding text descriptions, reference images, and video dynamic features from the multimodal knowledge base based on the semantic type of the frame-filling region, and schedules the corresponding large-scale generative model to construct a differentiated frame-filling strategy accordingly. The differentiated frame-filling strategy is dynamically constructed based on the integrity status of the input image, performs intelligent decision-making, and is collaboratively generated by the text-generated image large-scale model, the image-generated image large-scale model, and the image-generated video large-scale model.

[0070] The differentiated frame interpolation strategy specifically includes:

[0071] When processing a single partially damaged image, the agent generates a requirement label based on the semantic type of the identified missing region, calls the multimodal knowledge base to retrieve historical knowledge from the same period, and sequentially drives the text-generated image model to generate semantic contours and the image-generated image model to perform pixel-level completion with consistent texture. After outputting multiple candidate repair schemes, the optimal result is selected based on style consistency and feature similarity.

[0072] When processing multiple related images, at least one of which is complete, the agent uses the complete image as a reference benchmark, extracts its key visual features and fuses them with historical background information, spatially registers the complete image with the damaged image through image alignment technology, and coordinates the scheduling of the graph-generated image model to ensure structural consistency and the text-generated image model to supplement historical semantics, thereby achieving highly reliable targeted repair.

[0073] When processing multiple incomplete related images, the agent initiates a multi-source knowledge fusion mode, which integrates historical documents, contemporaneous images, and local features of each image to construct a scene semantic framework. This drives the text-based image model to deduce overall content assumptions, and the image-based image model to fuse local features from multiple perspectives for pixel-based deduction. Based on historical authenticity and visual logic consistency, multiple generation schemes are quantitatively evaluated and optimized, thereby reducing the blindness of repair assumptions and improving the historical accuracy and visual rationality of the frame-padding results.

[0074] The interpolated frames generated by each model are fused with the original image at the pixel level to ensure high visual consistency with the overall scene.

[0075] S4. Rendering Output: The data after frame interpolation by the AI ​​frame interpolation model is rendered in the cloud, and the results are output in multiple forms according to the application scenario requirements.

[0076] S4.1 The cloud-based rendering method includes: uploading image data generated by AI frame interpolation to a high-performance graphics server cluster in the cloud; dynamically allocating computing resources using a block rendering strategy; and performing refined rendering of architectural details, lighting effects, and dynamic elements. It also incorporates edge computing technology to achieve real-time preview and user interaction of the rendering results, supporting dynamic adjustments to scene perspective and detail presentation. This rendering method is existing technology and will not be elaborated upon here.

[0077] S4.2 The multi-format output includes one or more of the following: 3D model files, immersive VR / AR content, and dynamic videos.

[0078] The specific workflow of this embodiment is as follows:

[0079] 1. Restoration status of partial damage to a single old photograph:

[0080] (1) When a user uploads a damaged old photo, the missing or damaged areas are first identified and located using an image segmentation algorithm.

[0081] (2) The intelligent agent generates required labels based on the content of old photos, such as the year, photo type, missing content, etc., and then converts the labels into feature vectors and calls the retrieval engine to match relevant reference knowledge from the same period.

[0082] (3) The agent drives the text-generated image and the corresponding large model of the image-generated image to generate the image collaboratively based on the reference knowledge. First, the text-generated image model generates the semantic outline of the missing area based on historical background knowledge. Then, the image-generated image model combines the texture features of the original image to perform pixel-level completion. Finally, three versions of repair candidate schemes are generated.

[0083] (4) Compare the style consistency and related feature similarity between the three versions of the scheme and the original image, select the optimal scheme and output it.

[0084] 2. Repairing missing related photos: Depending on the completeness of the photos, there are two scenarios:

[0085] 2.1 When photo A is complete but photo B is incomplete:

[0086] When photo A is used as the core reference, the agent prioritizes extracting key information feature vectors from photo A, including core visual elements such as the person's posture, the layout of objects, and color and lighting. Simultaneously, the agent retrieves relevant historical records, such as the shooting time, location, and event background, and fuses these with the feature vectors of photo A to clarify the correlation between photo A and photo B within the same scene, such as different angles within the same scene or different actions of the same person.

[0087] Because a complete photo A serves as a direct reference, the number of variables involved in the restoration process is relatively small, and the amount of related knowledge invoked is relatively controllable, resulting in fewer restoration schemes. In practice, the agent uses image pairing technology to precisely align photo A with photo B, driving the collaborative generation of corresponding large-scale models for the image-generated image and the text-generated image. The feature vector of photo A is used as input to the image-generated image model to ensure visual consistency, while the text-generated image model supplements historical scene semantic information. Based on the corresponding regional features of photo A, the missing parts of photo B are expanded and restored, resulting in high accuracy and reliability of the restoration results.

[0088] The entire workflow can be completed in just three steps: feature extraction, correlation analysis, and alignment repair, which is much more efficient than traditional methods.

[0089] 2.2 Neither photos A nor B are complete:

[0090] The agent initiates a "multi-source knowledge fusion" mode. First, it comprehensively extracts historical data related to the photos from the knowledge base, including the shooting background, event descriptions, information about people, and other images from the same period. This information is then transformed into semantic and visual fusion feature vectors to construct a general framework of the photos' complete content. Due to the lack of complete reference photos, the agent simultaneously retrieves and cross-compares existing local feature vectors from photos A and B. Based on the fused knowledge, it drives the collaborative generation of text-based and image-based large-scale models. The text-based model constructs a scene semantic framework based on historical data, while the image-based model fuses local features from photos A and B for pixel-level deduction, making various assumptions and deductions about the possible forms of the missing areas. Therefore, there are multiple possibilities for the reconstruction.

[0091] To ensure the reasonableness of the restoration, the intelligent agent will screen and verify multiple sets of solutions, combining the authenticity of historical records, such as the era of clothing and scene layout; and the consistency of image logic, such as the direction of light and shadow and the proportion of figures; and then evaluate them. For example, if two photos are missing key figures from the same event, multiple figure completion solutions can be generated based on information such as the figures' clothing and positions in historical records. After comparative analysis, the optimal result that conforms to the historical scene is output, solving the problem of "blind assumptions" in traditional solutions. This makes the supplemented frame content not only visually coherent, but also highly consistent with the real scene in terms of historical background, era characteristics, and semantic logic, significantly improving the historical accuracy and credibility of the restoration results.

[0092] Finally, a cloud architecture combining block rendering and edge computing is adopted to achieve real-time preview and interactive adjustment of high-precision 3D scenes, and supports multiple output formats such as 3D models, VR / AR content, and dynamic videos, to meet the diverse application needs of cultural heritage digitization, film and television restoration, virtual reality, and other scenarios.

[0093] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An AI-driven method for local missing element frame completion, characterized in that, The method comprises the following steps: Collecting and preprocessing multi-modal data including text data, image data and video data; Building a multi-modal knowledge system, extracting directional features from the multi-modal data, converting them into structured data that can be associated, and realizing feature archiving and semantic association through an agent workflow, and establishing a retrieval engine in combination with a RAG system; Building an AI frame filling model that integrates text-to-image, image-to-image, and image-to-video large model generation capabilities, generates missing area filling content through feature constraints of the multi-modal knowledge system, and performs local frame filling on the filling content generated by the large model and the original image; The data obtained after filling the AI frame filling model is rendered in the cloud, and the results are output in multiple forms according to the application scenario requirements.

2. The AI-driven missing local element frame completion method of claim 1, wherein, The feature archiving and semantic association through the agent workflow specifically includes: Calling a classification model to automatically archive the extracted multi-modal features, and synchronizing the classification results to a distributed storage system through a model context protocol; Establishing cross-modal semantic association between text and visual data, mapping text feature vectors and image feature vectors, associating dynamic features in videos with static features in images, and forming a closed-loop verification relationship between text definitions and visual samples.

3. The AI-driven missing local element frame completion method of claim 1, wherein: The retrieval engine established in combination with the RAG system specifically includes: Combining keyword retrieval and vector similarity matching, extracting core keywords to locate relevant text features according to AI frame filling requirements, and further retrieving image visual features and video dynamic features with similar semantics based on the text features; When there is a knowledge conflict in the retrieval results, quantitatively score the candidate knowledge based on multiple dimensions such as multi-source cross-validation, feature consistency analysis and historical data correlation, and select the knowledge with the highest credibility as the basis for frame filling.

4. The AI-driven missing local element frame completion method of claim 3, wherein, The multi-modal knowledge system supports dynamic updating, specifically including: Converting user feedback repair and correction data into new multi-modal feature vectors, and archiving the new feature vectors into corresponding text, image, video or classification nodes through a data classification process driven by a model context protocol; After the new features are imported, automatically trigger the semantic association between the reconstructed knowledge, realize the synchronous evolution of the knowledge system and the actual application requirements, and continuously improve the accuracy and scene adaptability of subsequent AI frame filling.

5. The AI-driven missing local element frame completion method of claim 1, wherein, The large model generation capability includes: A text-to-image large model for receiving historical text descriptions retrieved by the RAG system and combining the style features of the target image to generate local scene detail materials consistent with historical reality; An image-to-image large model for extending or repairing missing areas by referring to multi-view image features in the knowledge base based on preprocessed target images, ensuring that the filling content is consistent with the original image in terms of light and shadow, texture and perspective; A video-to-video large model for generating multi-view dynamic frames with time sequence continuity based on static images and video dynamic features in the knowledge base, providing dynamic frame sequence support for three-dimensional scenes.

6. The AI-driven missing local element frame completion method of claim 5, wherein, The local frame filling process includes the following steps: Identify damaged, obscured or information missing areas in the target image through image segmentation algorithms, generate a frame filling area mask to determine the frame filling range and boundary features; According to the semantic type of the frame filling area, the intelligent agent workflow retrieves corresponding text descriptions, reference images and video dynamic characteristics from a multi-modal knowledge base, and dispatches a corresponding generation large model to construct a differentiated frame filling strategy; The frame filling content generated by each model is pixel-level fused with the original image to ensure high consistency with the overall scene in visual logic.

7. The AI-driven missing local element frame completion method of claim 6, wherein: The differentiated frame filling strategy is dynamically constructed according to the integrity state of the input image, and intelligent decision-making is performed, with the cooperation of text-to-image large models, image-to-image large models and image-to-video large models.

8. The AI-driven missing local element frame completion method of claim 7, wherein, The differentiated frame filling strategy specifically includes: When processing a single local damaged image, the intelligent agent generates a demand label based on the identified semantic type of the missing area, retrieves historical knowledge of the same period from a multi-modal knowledge base, and sequentially drives a text-to-image large model to generate a semantic outline and an image-to-image large model to perform texture-consistent pixel-level completion. After outputting multiple candidate repair schemes, the optimal result is selected based on style consistency and feature similarity. When processing multiple related images, at least one of which is complete, the intelligent agent takes the complete image as a reference benchmark, extracts its key visual features and fuses them with historical background information, spatially registers the complete image with the damaged image through image alignment technology, and collaboratively dispatches an image-to-image model to ensure structural consistency and a text-to-image model to supplement historical semantics, achieving high-reliability directional repair. When processing multiple related images that are all incomplete, the intelligent agent starts a multi-source knowledge fusion mode, constructs a scene semantic framework by integrating historical documents, contemporaneous images and local features of each image, drives a text-to-image model to deduce overall content hypotheses, an image-to-image model to fuse multi-perspective local features for pixel inference, and based on historical authenticity and visual logic consistency, quantitatively evaluates and optimizes multiple generated schemes, thereby reducing the blindness of repair hypotheses and improving the historical accuracy and visual rationality of the frame filling result.

9. The AI-driven missing local element frame completion method of claim 1, wherein, The cloud rendering method includes: Upload the image data generated by AI frame filling to a cloud high-performance graphics server cluster, dynamically allocate computing resources using a block rendering strategy, and perform fine rendering of architectural details, lighting effects and dynamic elements. Combined with edge computing technology, real-time preview and user interaction of the rendering result are realized, supporting dynamic adjustment of scene perspective and detail presentation.

10. The AI-driven missing local element frame completion method of claim 1, wherein, The multi-form output includes one or more of three-dimensional model files, immersive VR / AR content and dynamic video.

Images