A cross-device content synchronization and adaptive display method based on a multi-modal large model

Abstract

The application discloses a cross-device content synchronization and adaptive display method based on a multi-modal large model, and relates to the technical field of information processing and display. The method constructs a closed-loop system through three core steps: firstly, non-structured pixel flow is reconstructed into structured objects through cross-modal screen content recognition and deep semantic extraction; secondly, a platform-independent scene graph is constructed through generalized semantic encapsulation, realizing content and source hardware decoupling; finally, entity pruning, space arrangement and double-channel rendering are completed based on constraint tensor-driven topological mapping and hybrid rendering. The application breaks through the picture stream transmission limitation of traditional screen projection, realizes adaptive adaptation driven by structured semantic stream, considers text sharpness and media fluency, guarantees content consistency through a closed-loop quality control mechanism, effectively solves the rigid problem of heterogeneous device adaptation, and improves the adaptability and reliability of cross-device interaction.

Description

A method for cross-device content synchronization and adaptive display based on a multimodal large model Technical Field

[0001] This invention relates to the field of computer information processing and display technology, specifically to a cross-device content synchronization and adaptive display method based on a multimodal large model, applicable to multimodal content flow, semantic reconstruction and adaptive display scenarios among heterogeneous devices such as mobile phones, tablets, TVs, and large-screen displays. Background Technology

[0002] In the digital age of interconnected devices, users generally need to project content from source devices such as mobile phones and tablets onto target devices such as TVs and large-screen monitors to achieve efficient collaborative work or immersive viewing experiences. However, the significant differences between different devices in screen size, resolution, interaction methods (touch / remote control / voice), and display capabilities pose serious challenges to cross-device content synchronization.

[0003] Current mainstream cross-device synchronization methods have significant drawbacks: First, content stream mirroring and operation mapping only transmit pixel streams without parsing underlying semantics. Large-screen displays are simply scaled-down versions of small screens, failing to alter the content structure to leverage the advantages of large screens. Second, content-feature-based adaptive adjustments heavily rely on developer-defined rules, lacking generative UI reconstruction capabilities when facing heterogeneous devices, screens of special sizes, or dynamic content. Third, collaboration mechanisms based on network broadcasting or event triggering only focus on time synchronization of device functions, failing to achieve a deep transformation of content from visual to semantic perspectives and unable to dynamically adjust layout and visual elements according to terminal characteristics. These issues result in poor cross-device content presentation, unnatural interactions, and an inability to meet users' demands for a high-quality collaborative experience. Summary of the Invention

[0004] This invention aims to provide a cross-device content synchronization and adaptive display method based on a multimodal large model. By semantic-level parsing, standardized encapsulation, and constraint-driven adaptation of the source device content, it solves the problems of rigid adaptation, poor display effect, and insufficient semantic understanding in existing technologies, and achieves intelligent reconstruction and efficient adaptation of cross-device content, thereby improving display fidelity and natural interaction.

[0005] To solve the above-mentioned technical problems, the present invention adopts the following solution:

[0006] A method for cross-device content synchronization and adaptive display based on a multimodal large model, characterized by the following core steps:

[0007] S1. Cross-modal screen content recognition and deep semantic extraction: Through multimodal feature extraction and multimodal large model inference, analyze the visual, textual and logical association information of the source device screen content, and reconstruct the unstructured pixel stream into an interactive structured object;

[0008] S2. Generalized semantic encapsulation based on standard serialization protocol: The structured object is encapsulated into a platform-independent semantic description form, a scene graph is constructed and logical relationships between content are defined, thereby decoupling the content logic from the source physical attributes.

[0009] S3. Topology mapping and hybrid rendering based on constraint tensor: Obtain the multidimensional physical constraints of the target device and construct the constraint tensor. Based on the constraint tensor, perform adaptation processing on the scene graph and achieve adaptive display of content through distributed hybrid rendering.

[0010] Preferably, step S1 specifically includes the following steps:

[0011] S11. Spatiotemporally Saliency Panoramic Segmentation: A class-agnostic segmentation engine combined with a temporal difference algorithm is used to perform panoramic segmentation on the original image. The original image is deconstructed into several physically independent candidate region sets R = r1, r2, r3, r4, r5, where the geometric attributes of each candidate region are represented by r. i =(Mask i Box i Motion i ), where Mask i A pixel-level mask used to precisely delineate irregular areas; Box i The coordinates of the circumscribed rectangle; Motion i It is a temporal dynamic feature; by comparing the pixel change rate of adjacent frames, it marks whether the region is a dynamic refresh area;

[0012] S12. Collaborative annotation of multi-source heterogeneous features: the candidate regions r generated in step S11 i Based on this, by invoking the object detection, OCR, and semantic coding engines in parallel, each candidate region r is processed. i Inject semantic attributes into candidate regions r i Transform into a tuple V with multidimensional features i =(x i ,y i ,w i ,h i Type i ,f vis ,t ocr )=MultiModal - Feature - Extractor(r i); where x i ,y i Let w be the coordinates of the top-left corner of the bounding rectangle of the i-th candidate region. i h is the width of the bounding rectangle of the i-th candidate region. i Let Type be the height of the bounding rectangle of the i-th candidate region. i For modal types, identify the higher-level categories of video player containers and text display containers; f vis As a visual feature vector, extract color histogram and texture features; t ocr For text semantics, including content, font size, and layout information; MultiModal - Feature - Extractor is a multimodal feature extractor module;

[0013] S13. MLLM-based Intent Parsing and Semantic Verification: A multimodal large model is introduced to perform deep reasoning, analyzing the logical relationships between components through thought chain analysis; to quantify the accuracy of reasoning, an intent recognition confidence score γ is introduced.

[0014]

[0015] Wherein, γ represents the confidence level of intent recognition, which is used to quantify the reliability of the current multimodal large model's reasoning results on the logical relationships between components; , and These represent the visual feature matching degree, spatial layout rationality score, and semantic logical relevance score, respectively; β1, β2 and β3 are the weighting coefficients for the three scoring components, used to adjust the weight ratio of visual, spatial and semantic information in the final decision.

[0016] Preferably, step S2 specifically includes the following steps:

[0017] S21. Definition and Attribute Injection of Semantic Objects: The system maps the identified logical function blocks to standard JSON objects. i JSON object O i Injected cognitive attributes derived from multimodal large model inference; each JSON object O i The data structure is defined as follows:

[0018]

[0019] Among them, ID i For object numbering, Sem typeThis is an enumeration of functional intent values, allowing the receiving end to perform multimodal rendering based on its own hardware configuration; Cont i Prio is the specific content payload of this semantic object, including its associated business data, text content, and actual information about resource references; i For adaptive rendering weights; This provides contextual state information for the semantic object, encompassing dynamic state data, which includes the current interaction state and the associated environment state.

[0020] S22. Topology Reconstruction and State Packaging for Heterogeneous Content Scenarios: Construct a platform-independent scenario description structure, build a hybrid topology network based on the modal attributes of content objects, and establish parent-child nested associations, layer-overlay overlay associations, and linear-flow sequence associations; at the same time, serialize and encapsulate the instantaneous running state of the source end in multiple dimensions, including temporal state vectors, interaction state vectors, and spatial state vectors, and inject them into the protocol header.

[0021] Preferably, in step S22, the system defines an association function. Differentiated composition logic is executed for different types of content:

[0022] For UI interaction primitives: the system detects the geometric containment relationship between components and establishes parent-child nesting associations to maintain the consistency of logical groups; for streaming media and visualization primitives: the system establishes layer-overlay overlay associations; for document and text-image flow primitives: the system establishes linear-flow sequence associations, recording the logical order of paragraphs rather than their absolute positions, ensuring that content can achieve adaptive flow reflow on display screens with dynamically adjustable display size and aspect ratio.

[0023] Preferably, in step S22, the timing state vector is: Used for precise synchronization of audio and video playback progress and buffering status; t stamp This is the current playback timestamp for audio and video, used to synchronize playback progress across devices. play buf is in audio / video playback mode. status In audio / video buffer state;

[0024] Interaction state vector: Records the current user's input focus, unsubmitted draft text, and page scroll position, id focus The txt file contains the ID of the object currently receiving input focus. input For draft text content that the user has not submitted, scroll pos This is the current scroll position of the page;

[0025] Spatial state vector: For 3D maps or game scenes, record the camera view matrix and zoom level; mat view The viewpoint matrix of the camera in the 3D scene; level zoom For the zoom level of a 3D scene or map; ang camera This refers to the camera's orientation angle in a 3D scene.

[0026] Preferably, step S3 specifically includes the following steps:

[0027] S31. Target-end constraint modeling and semantic graph pruning: The system acquires the attributes of the target device in real time and constructs a high-dimensional constraint tensor C. target ={D dim I mode R cap P user},in, For screen physical size and resolution, For interactive mode, For local rendering computing power, The coordinates of the user's current attention focus; the retention threshold is dynamically calculated based on a high-dimensional constraint tensor. For each object O in the scene graph i Execute the judgment logic to filter out the set of objects that meet the conditions. ;

[0028] in, This is a mapping function based on the target device dimension, outputting a threshold base value that matches the device size; Provide an attention weight function for the user's focus area, and output the weight value for the corresponding area; This is a scaling factor function that varies with the device dimension; the smaller the device dimension, the smaller the coefficient value. For all semantic objects in the scene graph The set is the source of objects for pruning operations;

[0029] S32. Scene Entity Mapping and Spatial Arrangement Based on Multidimensional Physical Constraints: For discrete entities in documents and image / text streams, based on the screen width W in the constraint tensor... screen Calculate the number of columns N in the space. col According to the number of columns N in the space col The value can be used to dynamically construct parallel or sequential containers to achieve adaptive rearrangement of content;

[0030] For interactive entities such as buttons and list items, an interaction correction factor δ(I) is introduced. mode Physically correct its geometric parameters;

[0031] For 3D game graphics and high-definition video streams, continuous media entities are mapped to independent rendering surfaces or texture viewports, and generative AI is used to generate environmental extension textures in blank areas.

[0032] S33: Distributed Hybrid Rendering Scheduling and Visual Consistency Closed Loop: Adopts a dual-channel rendering mode, with the local vector rendering channel processing document text and discrete interactive entities such as vector icons, and the cloud pixel stream channel processing high-load 3D game scenes and immersive media entities such as 4K video streams; at the same time, it introduces cross-end semantic fingerprint comparison and rendering circuit breaker mechanism, and immediately switches to safe rendering mode if the similarity is lower than the safety threshold.

[0033] Preferably, in step S32: the number of columns N col Satisfy the following formula:

[0034]

[0035] Among them, W screen W represents the screen width parameter of the target device. minreadability Minimum width threshold to keep content readable.

[0036] Preferably, the physical modification in step S32 specifically includes:

[0037]

[0038] in, This serves as an identifier for the interaction mode type. The geometric correction factor bound to this interaction mode; These are the original width and height parameters of the interactive entity in the semantic object stage.

[0039] The beneficial effects of this invention are as follows:

[0040] 1. Achieve dual decoupling between content and hardware, rendering and computing power, and break through the bottleneck of heterogeneous device adaptation: By constructing a multimodal large model generalized scene graph, the dependence of content on the resolution and layout of the source device is removed; with the help of a distributed dual-channel rendering architecture, rendering tasks are allocated according to the computing power of the target end, avoiding the experience degradation of devices with weak computing power.

[0041] 2. Achieve differentiated hybrid rendering, balancing text sharpness and media smoothness: Strongly structured information uses local vector drawing to ensure clarity at any scaling; unstructured media uses lightweight compression and incremental updates to balance frame rate and bandwidth efficiency.

[0042] 3. Construct a closed-loop quality control mechanism to ensure the semantic consistency of generative content: when the source end generates the scene graph, it extracts the semantic fingerprint as a benchmark. After the target end reconstructs the graph, it compares the fingerprint similarity. If semantic deviation occurs, it triggers the circuit breaker, effectively curbing the uncertainty of the generation model.

[0043] 4. Coherent interaction logic and strong compatibility: Based on the platform-independent scene graph, the core logic of the content is preserved and the interactive function is not affected after cross-device synchronization; the unified generalized semantic encapsulation standard breaks down the system and hardware barriers of different devices and reduces the risk of format conflict. Attached Figure Description

[0044] Figure 1 shows the set of candidate regions for spatiotemporal saliency panoramic segmentation.

[0045] Figure 2 shows the attribute map of each candidate region.

[0046] Figure 3 is a schematic diagram of the collaborative annotation process for multi-source heterogeneous features.

[0047] Figure 4 shows the attribute diagram of the multi-source heterogeneous feature tuple set.

[0048] Figure 5 is a schematic diagram of the intent parsing and semantic verification process based on MLLM.

[0049] Figure 6 shows the confidence data for intent recognition of logical function blocks. Detailed Implementation

[0050] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0051] (a) Application scenario setting

[0052] This embodiment takes casting video app content from a smartphone (source device, screen resolution 1920×1080) to a smart TV (target device, screen resolution 3840×2160) as an example. The cast content includes a video preview area, a plot summary area, a purchase button area, an episode list and selection area, and a related video recommendation area, specifically demonstrating the implementation process of the present invention.

[0053] (II) Detailed Implementation of Steps

[0054] 1. Step S110: Spatiotemporal saliency panoramic segmentation (see Figure 1)

[0055] Perform spatiotemporal saliency panoramic segmentation on the source device's video app screen to obtain a set of candidate regions.

[0056] R = {r1, r2, r3, r4, r5}

[0057] The attributes of the five candidate regions are as follows (see Figure 2):

[0058] r1 (video preview area, real-time playback): Mask1 is a binary mask covering all pixels of the video preview area, with bounding rectangle coordinates Box1=(20, 20, 1100, 500), and the temporal dynamic feature Motion1 is the dynamic refresh area.

[0059] r2 (Synopsis Area): Mask2 is a binary mask covering all pixels of the synopsis area, Box2=(1100,20,1900,400), Motion2 is the static area;

[0060] r3 (Purchase Button Area): Mask3 is a binary mask covering all pixels of the purchase button area, Box3=(1100,400,1900,500), Motion3 is the static area;

[0061] r4 (set list and selection area): Mask4 is a binary mask covering all pixels of the set list and selection area, Box4=(20,500,1900,650), Motion4 is the static area;

[0062] r5 (Related Video Recommendation Area, Cover): Mask5 is a binary mask covering all pixels of the related video recommendation area, Box5=(20,650,1900,1060), and Motion5 is the static area.

[0063] 2. Step S120: Collaborative annotation of multi-source heterogeneous features (see Figure 3 and Figure 4)

[0064] Multi-source heterogeneous feature collaborative annotation is performed on the above 5 candidate regions to generate a feature tuple set {V1,V2,V3,V4,V5}, and the attributes of each feature tuple are as follows:

[0065] V1 (video player container): The top-left corner coordinates of the bounding rectangle of the first candidate region are x1=20, y1=20; the width of the bounding rectangle of the first candidate region is w1=1080; the height of the bounding rectangle of the first candidate region is h1=480; the visual feature vector is f. vis The values ​​are: color histogram [0.2, 0.1, 0.7, ...], texture features [0.3, 0.5, ...], and no textual semantics. ocr ;

[0066] V2 (text display container): The top-left corner coordinates of the bounding rectangle of the second candidate region are x2=1100, y2=20; the width of the bounding rectangle of the second candidate region is w2=800; the height of the bounding rectangle of the second candidate region is h2=380; the visual feature vector is f. visGiven a color histogram [0.8, 0.1, 0.1, ...], texture features [0.1, 0.2, ...], and text semantics t ocr The title is "Synopsis...", font size 18px, left-aligned.

[0067] V3 (interactive button container): The top-left corner coordinates of the bounding rectangle of the third candidate area are x3=1100, y3=400; the width of the bounding rectangle of the third candidate area is w3=800; the height of the bounding rectangle of the third candidate area is h3=100; and the visual feature vector is f. vis For color histogram [0.1, 0.8, 0.1, ...], texture features [0.4, 0.6, ...], and text semantics t ocr The text appears to be a list of search terms or tags related to purchasing individual episodes, the complete series, or VIP memberships. The font size is 16px, and the layout is centered.

[0068] V4 (List Selection Container): The top-left corner coordinates of the bounding rectangle of the 4th candidate region are x4=20, y4=500; the width of the bounding rectangle of the 4th candidate region is w4=1880; the height of the bounding rectangle of the 4th candidate region is h4=150; the visual feature vector is f. vis Given a color histogram [0.8, 0.1, 0.1, ...], texture features [0.2, 0.3, ...], and text semantics t ocr The text should be formatted as "Episode 1, Episode 2...", font size 18px, left-aligned.

[0069] V5 (Content Recommendation Container): The top-left corner coordinates of the bounding rectangle of the 5th candidate region are x5=20, y5=650; the width of the bounding rectangle of the 5th candidate region is w5=1880; the height of the bounding rectangle of the 5th candidate region is h5=410; the visual feature vector is f. vis Given a color histogram [0.8, 0.1, 0.1, ...], texture features [0.3, 0.2, ...], and text semantics t ocr Set the text to "Related Video Recommendations:...", font size 18px, and center alignment.

[0070] 3. Step S130: Intent parsing and semantic verification based on MLLM (see Figure 5 and Figure 6)

[0071] The multimodal large model is invoked to perform intent parsing and semantic verification on the feature tuple set, and weighting coefficients are set.

[0072] β1=0.3, β2=0.4, β3=0.3, the specific results are as follows:

[0073] Video playback functionality collection: associated components V1, V4, V5, visual matching degree Svisual =0.8, Spatial rationality S spatial =0.9, semantic relevance Ssemantic=0.95, intent recognition confidence γ=0.3×0.8+0.4×0.9+0.3×0.95=0.885, which is an effective functional block;

[0074] Purchase interactive function blocks: Associated components V2, V3, S visual =0.75, S spatial =0.85, Ssemantic=0.9, the confidence level of intent recognition γ=0.3×0.75+0.4×0.85+0.3×0.9=0.835, which is a valid functional block.

[0075] 4. Step S200: Generalized semantic encapsulation

[0076] The two effective functional blocks mentioned above are mapped to standardized JSON objects, and functional intents (video playback and purchase interaction) and rendering weights (video playback has a higher weight than purchase interaction) are injected. A platform-independent scene graph is constructed, and the logical relationship between "video playback function set - purchase interaction function block" is defined. Combined with the current video playback progress (00:25:10), the user's unsubmitted comment drafts, and other global session states, it is encapsulated into a standardized semantic data package in JSON format.

[0077] 5. Step S300: Topology Mapping and Hybrid Rendering

[0078] After the target device (smart TV) receives the semantic data packet, it constructs the constraint tensor Ctarget:

[0079] D dim =55 inches, resolution=3840*2160, mode=remote control, R cap =Supports 4K rendering, P user =Video preview area.

[0080] The scene graph is pruned based on constraint tensors to retain core functional blocks; the layout is adjusted according to the landscape TV screen characteristics, the video preview area is enlarged to a suitable size of 3840*2160, the plot summary area and purchase button area are arranged in a right-side double-column layout, and the episode list area is horizontally expanded to full screen width; a dual-channel rendering mode is adopted, the plot text, purchase button and other structured elements are drawn locally by vector, the video stream is rendered in the cloud and encoded into H.265 format and sent back, and the blank areas at the edge of the screen are filled by generative AI, finally outputting a high-fidelity, black-border-free adapted interface.

[0081] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Based on the technical essence of the present invention, any simple modifications, equivalent substitutions, and improvements made to the above embodiments within the spirit and principles of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A method for cross-device content synchronization and adaptive display based on a multimodal large model, characterized in that, The core steps include: S1, Cross-modal screen content recognition and deep semantic extraction: Through multimodal feature extraction and multimodal large model inference, the visual, textual, and logical association information of the source device screen content is analyzed, and the unstructured pixel stream is reconstructed into an interactive structured object; S2, Generalized semantic encapsulation based on standard serialization protocol: The structured object is encapsulated into a platform-independent semantic description form, a scene graph is constructed, and logical associations between content are defined to achieve decoupling between content logic and source physical attributes; Step S2 specifically includes the following steps: S21, Definition and attribute injection of semantic objects: The system maps the identified logical function blocks to standard JSON objects O i JSON object O i Injected cognitive attributes derived from multimodal large model inference; each JSON object O i The data structure is defined as follows: Among them, ID i For object numbering, Sem type This is an enumeration of functional intent values, allowing the receiving end to perform multimodal rendering based on its own hardware configuration; Cont i Prio is the specific content payload of this semantic object, including its associated business data, text content, and actual information about resource references; i For adaptive rendering weights; State i The context state information of the semantic object includes dynamic state data, including the current interaction state and the environment-related state; S22, topology reconstruction and state encapsulation of heterogeneous content scenes: Construct a platform-independent scene description structure, build a hybrid topology network based on the modal attributes of the content object, and establish parent-child nested association, layer-overlay superimposed association, and linear-flow sequence association; At the same time, the instantaneous running state of the source end is serialized and encapsulated in multiple dimensions, including temporal state vector, interaction state vector, and spatial state vector, and injected into the protocol header; S3, topology mapping and hybrid rendering based on constraint tensor: Obtain the multi-dimensional physical constraints of the target device and construct a constraint tensor, perform adaptation processing on the scene graph based on the constraint tensor, and achieve adaptive display of content through distributed hybrid rendering.

2. The method for cross-device content synchronization and adaptive display based on a multimodal large model according to claim 1, characterized in that, Step S1 specifically includes the following steps: S11, Spatiotemporal saliency panoramic segmentation: Using a class-agnostic segmentation engine combined with a temporal difference algorithm, panoramic segmentation is performed on the original image; the original image is deconstructed into several physically independent candidate region sets R=r1,r2,r3,r4,r5, and the geometric attributes of each candidate region are represented by r. i =(Mask i Box i Motion i ), where Mask i A pixel-level mask used to precisely delineate irregular areas; Box i The coordinates of the circumscribed rectangle; Motion i For temporal dynamic features; by comparing the pixel change rate of adjacent frames, the candidate region is marked as a dynamically refreshed area; S12, collaborative annotation of multi-source heterogeneous features: the candidate region r generated in step S11 i Based on this, by invoking the object detection, OCR, and semantic coding engines in parallel, each candidate region r is processed. i Inject semantic attributes into candidate regions r i Transform into a tuple V with multidimensional features i =(x i ,y i ,w i ,h i Type i ,f vis ,t ocr )=MultiModal - Feature - Extractor(r i ); where x i ,y i Let w be the coordinates of the top-left corner of the bounding rectangle of the i-th candidate region. i h is the width of the bounding rectangle of the i-th candidate region. i Let Type be the height of the bounding rectangle of the i-th candidate region. i For modal types, identify the higher-level categories of video player containers and text display containers; f vis As a visual feature vector, extract color histogram and texture features; t ocr For text semantics, including content, font size, and layout information; MultiModal - Feature - Extractor is a multimodal feature extractor module; S13, Intent parsing and semantic verification based on MLLM: Introducing a multimodal large model to perform deep reasoning, analyzing the logical relationships between components through thought chain analysis; To quantify the accuracy of reasoning, an intent recognition confidence score γ is introduced: Wherein, γ represents the confidence level of intent recognition, which is used to quantify the reliability of the current multimodal large model's reasoning results on the logical relationships between components; 、 and These represent the visual feature matching degree, spatial layout rationality score, and semantic logical relevance score, respectively; β1, β2 and β3 are the weighting coefficients for the three scoring components, used to adjust the weight ratio of visual, spatial and semantic information in the final decision.

3. The method for cross-device content synchronization and adaptive display based on a multimodal large model according to claim 1, characterized in that, In step S22, the system defines the correlation function. Differentiated composition logic is executed for different types of content: For UI interaction primitives: the system detects the geometric containment relationship between components and establishes parent-child nesting associations to maintain the consistency of logical groups; for streaming media and visualization primitives: the system establishes layer-overlay overlay associations; for document and text-image flow primitives: the system establishes linear-flow sequence associations, recording the logical order of paragraphs rather than their absolute positions, ensuring that the content achieves adaptive flow reflow on display screens with dynamically adjustable display size and aspect ratio.

4. The method for cross-device content synchronization and adaptive display based on a multimodal large model according to claim 1, characterized in that, In step S22, the timing state vector is: It is used to accurately synchronize the playback progress and buffering status of audio and video; t stamp This is the current playback timestamp for audio and video, used to synchronize playback progress across devices. play buf is in audio / video playback mode. status Audio / video buffer state; Interaction state vector: Records the current user's input focus, unsubmitted draft text, and page scroll position, id focus The txt file contains the ID of the object currently receiving input focus. input For draft text content that the user has not submitted, scroll pos The current scroll position of the page; spatial state vector: For 3D maps or game scenes, record the camera view matrix and zoom level; mat view This represents the viewpoint matrix of the camera in the 3D scene. level zoom For the zoom level of a 3D scene or map; ang camera This refers to the camera's orientation angle in a 3D scene.

5. The method for cross-device content synchronization and adaptive display based on a multimodal large model according to claim 1, characterized in that, Step S3 specifically includes the following steps: S31, Target-end constraint modeling and semantic graph pruning: The system acquires the attributes of the target device in real time and constructs a high-dimensional constraint tensor C. target ={D dim I mode R cap P user },in, For screen physical size and resolution, For interactive mode, For local rendering computing power, The coordinates of the user's current attention focus; the retention threshold is dynamically calculated based on a high-dimensional constraint tensor. For each object O in the scene graph i Execute the judgment logic to filter out the set of objects that meet the conditions. ;in, This is a mapping function based on the target device dimension, outputting a threshold base value that matches the device size; Provide an attention weight function for the user's focus area, and output the weight value for the corresponding area; This is a scaling factor function that varies with the device dimension; the smaller the device dimension, the smaller the coefficient value. For all semantic objects in the scene graph The set is the source of objects for pruning operations; S32, Scene entity mapping and spatial arrangement based on multidimensional physical constraints: For discrete entities in documents and text / image streams, based on the screen width W in the constraint tensor screen Calculate the number of columns N in the space. col According to the number of columns N in the space col The value dynamically constructs parallel or sequential containers to achieve adaptive content rearrangement; for interactive entities such as buttons and list items, an interaction correction factor δ(I) is introduced. mode Physically correct its geometric parameters; for 3D game graphics and continuous media entities in high-definition video streams, map them to independent rendering surfaces or texture viewports, and use generative AI to generate environmental extension textures in blank areas; S33: Distributed hybrid rendering scheduling and visual consistency closed loop: adopt a dual-channel rendering mode, with the local vector rendering channel processing document text and discrete interactive entities such as vector icons, and the cloud pixel stream channel processing high-load 3D game scenes and immersive media entities such as 4K video streams; at the same time, introduce cross-end semantic fingerprint comparison and rendering circuit breaker mechanism, and immediately switch to safe rendering mode if the similarity is lower than the safety threshold.

6. The method for cross-device content synchronization and adaptive display based on a multimodal large model according to claim 5, characterized in that, In step S32: the number of columns N col Satisfy the following formula: Among them, W screen W represents the screen width parameter of the target device. minreadability Minimum width threshold to keep content readable.

7. The method for cross-device content synchronization and adaptive display based on a multimodal large model according to claim 5, characterized in that, The physical correction in step S32 is as follows: in, This serves as an identifier for the interaction mode type. The geometric correction factor bound to this interaction mode; These are the original width and height parameters of the interactive entity in the semantic object stage.

Images