Multi-terminal collaborative intelligent interactive interface dynamic adaptation system

Abstract

The invention discloses a multi-terminal collaborative intelligent interactive interface dynamic adaptation system which comprises a multi-terminal terminal, a hardware capability abstraction layer, a hybrid transmission layer, a global hardware capability collaboration and low-delay adaptation core engine and an interface recombination layer, and all modules collaborate to achieve multi-terminal hardware resource scheduling and interactive interface dynamic adaptation. Specifically, the multi-terminal terminal comprises at least two types of intelligent terminals, and the intelligent terminals are selected from any combination of mobile phones, computers, tablet personal computers, intelligent wearable devices and vehicle-mounted terminals. Hardware resources of the multi-terminal terminal are converted into standardized schedulable capability units through a hardware capability abstraction layer; the unified calling interface shields hardware differences of different terminals, collaborative scheduling and multiplexing of cross-device hardware resources are achieved, hardware capability idling or repeated calling of a single device is avoided, and the overall utilization rate of multi-terminal hardware is remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the field of computer technology, and particularly relates to a multi-terminal collaborative intelligent interaction interface dynamic adaptation system. Background Art

[0002] In the related technologies of the existing multi-terminal collaborative interaction interface adaptation, the multi-terminal collaborative system has covered various intelligent terminals such as computers, mobile phones, tablets, smart wearable devices, and vehicle-mounted terminals, and can realize data interaction between terminals through a cloud server or a local area network, and has basic functions such as file synchronization, message transmission, and simple hardware call. For example, some systems support the transmission of pictures and documents between mobile phones and computers, or the cross-terminal use of a single terminal hardware (such as a mobile phone camera and a computer microphone) through specific software; at the same time, the existing technologies generally adopt a standardized data transmission protocol to ensure the basic compatibility of different terminals, and some solutions also preset a fixed interface layout template to meet the multi-terminal collaboration needs of users in simple scenarios such as daily office work and home entertainment, and realize basic multi-terminal linkage operations.

[0003] With the complication of multi-terminal collaborative scenarios, the existing technologies gradually show obvious limitations and are difficult to match the efficient and stable collaboration requirements. From the perspective of hardware scheduling, the existing systems lack a hardware capability standardization abstraction mechanism and cannot globally plan the hardware resources of multiple terminals, resulting in low hardware utilization rate, and often there is a situation where some terminal hardware is idle while some terminal hardware is overloaded; at the transmission level, most solutions rely on a single cloud centralized transmission architecture and cannot dynamically adjust the transmission channel according to the data type (low-latency real-time data, large-data-volume files), and it is easy to have a problem of too high transmission delay in real-time interaction scenarios; in terms of interface adaptation, the fixed interface layout template cannot be dynamically adjusted according to the hardware call status and transmission quality, and the adaptability is poor, and it is difficult to meet the operation habits and scenario requirements of different terminals; in addition, the existing technologies lack effective weak network adaptation and hardware offline degradation mechanisms, and it is easy to have a function interruption when the network fluctuates or the hardware is offline, and when multiple terminals compete for the same hardware resource, the conflict handling efficiency is low and the optimal decision cannot be quickly output, seriously affecting the fluency and stability of multi-terminal collaboration. Summary of the Invention

[0004] In order to overcome the above defects of the existing technology, the present invention provides a multi-terminal collaborative intelligent interaction interface dynamic adaptation system, which solves the problems of poor multi-terminal collaboration, high transmission delay, weak interface adaptation, and low hardware utilization rate in the existing technology.

[0005] To achieve the above object, the present invention provides the following technical solutions: A multi-terminal collaborative intelligent interactive interface dynamic adaptation system includes multi-terminal terminals, a hardware capability abstraction layer, a hybrid transmission layer, a global hardware capability collaboration and low-latency adaptation core engine, and an interface reorganization layer. These modules collaborate to achieve multi-terminal hardware resource scheduling and dynamic adaptation of the interactive interface, as detailed below: The multi-terminal includes at least two types of smart terminals, and the smart terminals are selected from any combination of mobile phones, computers, tablets, smart wearable devices, and vehicle terminals; The hardware capability abstraction layer is used to transform the hardware resources of each of the multi-terminal terminals into standardized and schedulable capability units, provide a unified calling interface to the global hardware capability collaboration and low-latency adaptation core engine, and support edge computing nodes of the hybrid transport layer to cache the global hardware capability status view. The hybrid transmission layer is used to provide low-latency transmission channels for scheduling instructions, hardware data and interface status data of the core engine for global hardware capability coordination and low-latency adaptation. It includes edge computing nodes deployed at the edge of the regional network and P2P direct connection networks between devices, and can dynamically allocate transmission channels according to data transmission requirements. The global hardware capability coordination and low-latency adaptation core engine, as the system decision center, integrates hardware capability scheduling, transmission strategy selection and interface adaptation rules. It can parse user operation behavior to generate hardware combination schemes and interface reorganization instructions, and realize the synchronization of interface status on each end. The interface reconfiguration layer receives interface reconfiguration instructions from the global hardware capability coordination and low-latency adaptation core engine, dynamically adjusts the interactive interface layout of each of the multi-terminal terminals, adapts the interface according to the terminal hardware capabilities and transmission status, and triggers interface degradation when the capability unit is offline or transmission is interrupted.

[0006] Preferably, the specific methods by which the hardware capability abstraction layer implements standardized capability unit conversion and unified calling interface include: automatically reporting hardware resource type, performance parameters, and current status when the device is connected, encapsulating them into capability units through the Protobuf protocol and registering them to the global hardware capability collaboration and low-latency adaptation core engine; synchronizing hardware status with the global hardware capability collaboration and low-latency adaptation core engine through heartbeat packets at 10ms intervals; the hardware resource types include keyboard, camera, microphone, sensor, display module, and stylus; the performance parameters include camera resolution, screen refresh rate, and sensor sampling rate; the current status includes hardware idle status, hardware occupancy status, hardware load rate, and terminal battery level.

[0007] Preferably, the specific structure of the hybrid transport layer includes: the edge computing node has a built-in transport strategy decision module, a global state cache module, and a conflict arbitration module; the P2P direct connection network is built based on WebRTC technology and supports NAT traversal; the global state cache module is used to cache the capability unit status of each terminal, user operation preference data, and commonly used interface reassembly templates.

[0008] Preferably, the dynamic allocation mechanism of the transmission channel in the hybrid transmission layer satisfies: For low-latency data, including hardware call commands, interface operation status data, and real-time interactive data, P2P direct connection channels are preferred, and the transmission latency does not exceed 50ms. For large volumes of data, including high-definition video streams, image materials, and large file data, edge node acceleration channels are used; When the packet loss rate of the P2P direct connection channel exceeds 10%, it will automatically switch to the edge node acceleration channel; when the load rate of the edge computing node exceeds 80%, some large data volumes will be diverted to the P2P direct connection channel. The P2P direct connection network supports simultaneous access by multiple terminals; when the P2P connection between any two terminals is interrupted, data is automatically forwarded through edge computing nodes to maintain interface state synchronization.

[0009] Preferably, the specific execution steps of the global hardware capability scheduling algorithm of the global hardware capability coordination and low-latency adaptation core engine include: Step 1: Requirements analysis. By identifying the user's operation behavior on any of the multi-terminal devices, the core hardware capabilities required for the current operation are extracted. Step 2: Capability matching. Query the global hardware capability status view of the edge computing node cache and filter out capability units that meet the performance parameter requirements and are in an idle state. Step 3: Optimal combination generation. The final hardware capability combination scheme is determined by combining the load rate of each terminal and the transmission cost. The load rate of a single terminal shall not exceed 70%. In terms of transmission cost, terminals within the same local area network are given priority, while also taking into account the user's historical operation preferences.

[0010] Preferably, the specific logic for the global hardware capability coordination and low-latency adaptation core engine to generate interface reconfiguration instructions includes: Each invoked capability unit is assigned a corresponding interface function module. When the terminal camera capability unit is invoked, the terminal interface displays the camera preview module and the shooting control module; when the terminal screen display capability unit is invoked, the terminal interface displays the data sharing module and the interactive list module. The interface loading priority is adjusted according to the real-time latency of the transmission channel: when the transmission latency exceeds 100ms, only the core functional modules are rendered; when the transmission latency does not exceed 50ms, the complete interface consisting of the core functional modules and auxiliary functional modules is loaded.

[0011] Preferably, the specific methods by which the interface reorganization layer dynamically adjusts the interface layout include: The interface adopts a modular architecture, splitting it into core functional modules and auxiliary functional modules, supporting on-demand loading. The core functional modules include shooting control buttons, data display canvas, and basic interactive controls. The auxiliary functional modules include parameter setting panel, historical operation record panel, and device connection list. The interface module's position, size, and interaction method are adaptively adjusted based on the terminal's screen size and hardware capabilities: for touch-screen terminals, the interface module displays large touch buttons; for keyboard and mouse-controlled terminals, the interface module displays a shortcut key prompt panel.

[0012] Preferably, the global hardware capability coordination and low-latency adaptation core engine also includes a user preference learning module. This module analyzes the user's selection of interface layout and hardware calling habits for each terminal, updates the decision weights of the global hardware capability scheduling algorithm, and optimizes the generation logic of interface reconfiguration instructions.

[0013] Preferably, in the conflict arbitration module of the hybrid transport layer edge computing node, when multiple terminals compete for the same capability unit, the arbitration priority order is as follows: the terminal currently associated with the user's operation takes priority over the historically frequently used terminal, and the historically frequently used terminal takes priority over the terminal with the lowest load rate.

[0014] Preferably, the specific method by which the interface reorganization layer triggers interface degradation includes: when a capability unit is detected to be offline or its transmission is interrupted, the failed interface module directly associated with the capability unit is first hidden, and the status of hardware offline or transmission interruption is explicitly marked on the interface; if the system detects that there is an idle backup capability unit that is functionally compatible with the failed capability unit, the backup capability unit is automatically associated and the interface layout is updated synchronously, and the function of the original failed module is migrated to the interface module corresponding to the backup capability unit; if there is no compatible backup capability unit, a prompt to connect available hardware is displayed on the interface, and 3-5 types of compatible hardware that match the current function are listed.

[0015] The technical effects and advantages of the multi-terminal collaborative intelligent interactive interface dynamic adaptation system of the present invention are as follows: 1. This invention transforms the hardware resources of multiple terminals into standardized and schedulable capability units through a hardware capability abstraction layer. It uses a unified calling interface to shield the hardware differences between different terminals, enabling collaborative scheduling and reuse of hardware resources across devices. This avoids idle or repeated use of hardware capabilities of a single device, significantly improving the overall utilization rate of multi-terminal hardware. At the same time, it supports edge node caching of the global hardware capability status view, ensuring that the core engine obtains the available status of hardware on each terminal in real time, providing data support for the optimal allocation of hardware resources, and completely solving the problems of difficult multi-terminal hardware coordination and resource waste in traditional solutions.

[0016] 2. This invention ensures real-time performance for low-latency data through a direct P2P connection channel, while improving transmission efficiency for large-volume data through an edge node acceleration channel. It also supports dynamic channel switching (switching to the edge channel when P2P packet loss exceeds the limit, and offloading to the P2P channel when edge nodes are overloaded). This architecture breaks through the latency bottleneck of traditional single-cloud transmission, reducing latency for multi-device data transmission. Furthermore, in weak network environments, edge node caching and P2P penetration ensure data transmission stability, preventing transmission interruptions or buffering due to network fluctuations.

[0017] 3. The global hardware capability scheduling algorithm of the core engine of this invention generates a globally optimal hardware combination scheme through a full-process logic of demand analysis, capability matching, and optimal combination generation, taking into account factors such as terminal load rate, transmission cost, and user preferences, thus avoiding overload of a single terminal. The conflict arbitration module of the edge computing nodes efficiently resolves conflicts according to priority when multiple terminals compete for the same hardware capability, shortening conflict processing time and ensuring that core functions are executed first. Compared with the problems of lack of global scheduling and inefficient conflict handling in traditional solutions, this system can significantly improve the operational efficiency and stability of multi-terminal collaboration.

[0018] 4. The interface reorganization layer of this invention adopts a modular architecture to split the interface. It adaptively adjusts the position, size and interaction method of the interface modules according to the terminal screen size, hardware capability type and transmission status to meet the operating habits of different terminals (such as large buttons on touch terminals and shortcut key prompts on keyboard and mouse control terminals). At the same time, when the capability unit is offline or the transmission is interrupted, a hierarchical interface degradation strategy is triggered. By hiding the failed module, switching to backup hardware, and guiding the access of compatible hardware, the core functions are ensured to remain uninterrupted. This avoids the problem of fixed interface and functional failure when hardware is offline in traditional solutions, and significantly improves the user interaction experience.

[0019] 5. The user preference learning module of the core engine of this invention analyzes users' choices of interface layout and hardware calling habits on various terminals, updates the scheduling algorithm decision weights and interface reorganization logic, and realizes the personalized adaptation of the system to user habits, reducing manual settings operations for users; at the same time, the system supports the access of multiple types of smart terminals (mobile phones, computers, tablets, smart wearable devices, vehicle terminals, etc.), and can flexibly adapt to multiple scenarios such as remote office, in-vehicle navigation, creative design, outdoor work, and home sharing. It has strong flexibility and wide applicability, breaking through the limitations of traditional solutions in terms of single scenario adaptation and insufficient personalization. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating the implementation of a multi-terminal collaborative intelligent interactive interface dynamic adaptation system proposed in this invention. Detailed Implementation

[0021] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0022] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include," "contain," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "includes..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0023] refer to Figure 1This invention provides a multi-terminal collaborative intelligent interactive interface dynamic adaptation system. Its core architecture (multi-terminal terminal + hardware capability abstraction layer + hybrid transmission layer + global hardware capability collaboration and low-latency adaptation core engine + interface reconfiguration layer) strictly adheres to the system's core technical features: the hardware capability abstraction layer uses the Protobuf protocol to encapsulate capability units (10ms heartbeat packet synchronization status); the hybrid transmission layer constructs a P2P direct connection network via WebRTC (dynamically allocating edge / direct connection channels); the interface reconfiguration layer implements a three-level degradation mechanism of "failure concealment - backup migration - guidance prompts"; the core engine integrates user preference learning and global scheduling algorithms; and edge computing nodes perform conflict arbitration (priority: current operating terminal > historically frequently used terminal > terminal with the lowest load rate), comprehensively verifying the system's collaborative adaptation capabilities.

[0024] Example 1 Remote work video conferencing scenarios: Real-time scenario: Users need to participate in a 2-hour customer needs communication meeting from home using a laptop to share PPT and meeting minutes, a smartphone to present their face (ensuring clear visuals), and wireless headphones to receive / transmit audio (avoiding ambient noise). The meeting must be smooth throughout (transmission latency ≤50ms), with real-time synchronization of interface status (e.g., PPT page turning and presentation screen synchronization), and quick adaptation to alternative hardware in case of unexpected headphone disconnection.

[0025] Implementation terminal: Laptop: Windows operating system, 2.5K resolution screen, supports screen sharing, connected to a home 500M Wi-Fi network, CPU load rate 15%, memory usage 20%; Smartphone: Android system, high-resolution rear main camera (supports 1080P@30fps video capture), connected to the same home Wi-Fi network, CPU load rate 10%, battery 85%; Wireless headphones: Supports active noise cancellation microphone, Bluetooth connection to smartphones, within Wi-Fi coverage area, audio sampling rate of 48kHz, and connection stability of 98%.

[0026] System components: The hardware capability abstraction layer (responsible for standardized encapsulation and state synchronization of hardware resources), the hybrid transmission layer (including urban edge nodes 1.2km away from the user's residential area, supporting edge acceleration and P2P direct connection), the global hardware capability coordination and low latency adaptation core engine (responsible for requirement parsing, hardware scheduling and interface instruction generation), and the interface reorganization layer (responsible for dynamic adjustment and degradation processing of the interface of each terminal).

[0027] Implementation steps: (1) Hardware capability registration and edge caching: After the laptop is connected to the system, the hardware capability abstraction layer automatically collects and reports hardware resources: "display module (2.5K resolution, 60Hz refresh rate), screen sharing capability, keyboard and mouse input capability", which are encapsulated into capability units through the Protobuf protocol and registered to the core engine; the edge node caches the status of the capability unit and marks it as "idle, load rate 15%"; After the smartphone connects, it reports the hardware resources: "camera (high pixel, 1080P capture), built-in microphone, Bluetooth module", and the edge node synchronizes the cache status as "idle, load rate 10%"; After the wireless headphones are connected to the smartphone via Bluetooth, the mobile phone agent reports the hardware resources: "noise-canceling microphone, audio output capability", and the edge node cache status is "idle, load rate 5%"; The system updates the hardware status of each terminal in real time through heartbeat packets at 10ms intervals, and the edge nodes dynamically refresh the global hardware capability view to ensure that the core engine obtains the latest resource information.

[0028] (2) Dynamic allocation of transmission channels: When a user clicks the "Start a video conference" button on their laptop, the core engine identifies the user's needs based on the interaction: "Screen sharing (large data volume, bitrate 4Mbps) + facial recognition (low latency, bitrate 2Mbps) + audio transmission (low latency, bitrate 128kbps)". The hybrid transport layer allocates channels based on data type and network conditions: Laptop screen sharing data (large data volume): Using edge node acceleration channel, the actual transmission latency is 35ms (the network link bandwidth between the edge node and the user is sufficient). Smartphone camera data and wireless headphone audio data (low latency requirement): Using a laptop-smartphone P2P direct connection channel, the actual measured packet loss rate was 2% and the transmission latency was 28ms. The system started a channel monitoring thread to check the P2P channel status every 500ms. When the meeting was 45 minutes in, the P2P channel experienced a brief interference from home Wi-Fi, and the packet loss rate rose to 12%. The system automatically switched to the edge node channel, and the transmission latency was maintained at 42ms (≤50ms threshold), with no interface lag.

[0029] (3) Interface reorganization and degradation triggering: During a normal meeting, the interface restructuring layer adjusts the interfaces of each terminal according to the core engine's instructions: On laptops: The left 30% of the screen displays the "Meeting List Module" (including attendee avatars and mute status), the right 70% of the screen displays the "PPT Sharing Module", and the bottom is fixed with "Audio Control Buttons" (mute, volume adjustment); On smartphones: 80% of the screen displays the "Camera Preview Module" (live portrait mode), and the bottom 20% of the screen displays the "Shooting Angle Adjustment Slider" (supports 15° vertical adjustment). Wireless headset: No visual interface, only receives "mute / unmute" commands and outputs conference audio synchronously; The wireless headset unexpectedly disconnected during the meeting (Bluetooth connection interrupted): The interface reorganization layer immediately hides the "audio control module" directly associated with the headphones and displays a "headphones offline" status message on the laptop and smartphone interfaces; The system detected that the laptop's built-in microphone was idle (compatible with the headphone microphone function), automatically associated the backup capability unit, and synchronously updated the laptop interface, adding a "Built-in Microphone Enable" button; If the user does not activate the backup microphone in time, the interface will further display a prompt: "Please connect available hardware. Compatible types: Bluetooth noise-canceling headphones, device built-in microphone".

[0030] Implementation results: Technical specifications: The 2-hour meeting was smooth without any interruptions, the interface status synchronization success rate was 100%, the transmission latency was stable at 28-42ms, the maximum load rate of each terminal was ≤30%, and the downgrade processing time after the headset disconnected was 80ms. User experience: No need to manually switch device functions, PPT page turning and voice explanation synchronization latency ≤20ms, customer feedback "the smoothness of the screen is 60% higher than traditional meeting software", headphone noise cancellation reduces ambient noise by 45dB, audio transmission is quickly restored after disconnection, and there is no obvious experience gap.

[0031] Example 2 Navigation scenarios: Real-time scenario: When driving from a main urban road to a highway service area (50km in total, 70% of which is highway), the in-vehicle terminal needs to display a navigation map and real-time traffic conditions, the smartphone needs to collect video of the road ahead (to identify congestion / accidents), and the smartwatch needs to provide turn signals (to avoid distraction while driving). On highways, there may be weak 4G signals (bandwidth ≤1Mbps) or brief interruptions, so it is necessary to ensure that the navigation does not lag. If the smartphone camera is occupied by the dashcam, the navigation needs should be prioritized.

[0032] Implementation terminal: In-vehicle terminal: Linux kernel system, 10.2-inch touch screen, supports GPS positioning and 4G network connection, and is connected to the vehicle's 4G network; Smartphone: iOS system, high-pixel rear main camera (supports road condition video recognition), 8GB memory, connected to vehicle 4G network; Smartwatch: HarmonyOS operating system, 1.3-inch screen, supports vibration alerts, Bluetooth connectivity to smartphones, 70% battery level.

[0033] System components: The hardware capability abstraction layer (responsible for hardware resource reporting and status synchronization of vehicle terminals, mobile phones, and watches), the hybrid transmission layer (including two edge nodes along the highway, pre-caching map tiles of commonly used road sections), the global hardware capability coordination and low-latency adaptation core engine (responsible for scheduling algorithm execution and interface instruction generation), the edge computing node (with built-in conflict arbitration module), and the interface reorganization layer (interface interaction optimization adapted to driving scenarios).

[0034] Implementation steps: (1) Handling terminal startup and hardware conflicts: After the vehicle is powered on, the vehicle terminal, smartphone, and smartwatch automatically connect to the system. The hardware capability abstraction layer reports the resources of each terminal: the vehicle terminal has "GPS positioning capability + 10.2-inch display capability + 4G communication capability", the smartphone has "high-pixel camera + 4G communication capability", and the smartwatch has "vibration alert capability + small-size display capability". When a smartphone simultaneously launches both the "Navigation Traffic Data Collection" and "Dashcam App," both request access to the camera, triggering a hardware conflict. After receiving a conflict request, the conflict arbitration module of the edge computing node makes a decision based on the terminal usage priority: "Navigation traffic data collection" is associated with the user's current core need (navigation), and "Dashcam APP" is a background task, so the camera is prioritized for navigation needs, and the dashcam APP automatically switches to "background low frame rate recording" (15fps) to avoid affecting the navigation function.

[0035] (2) Transmission adaptation in weak network environments: Once the vehicle enters the highway section, the 4G network bandwidth drops to 800kbps, and the hybrid transmission layer activates a weak network adaptation strategy: Edge nodes push pre-cached highway map tiles to vehicle terminals to avoid repeatedly downloading map data and reduce network usage; Road condition video data (bitrate 500kbps) collected by smartphone is transmitted via P2P direct connection between vehicle terminal and smartphone (NAT traversal is achieved using WebRTC technology, with a measured transmission latency of 45ms). When the vehicle enters a tunnel (4G signal interruption for 20 seconds), the system triggers an interface downgrade: the in-vehicle terminal hides the "real-time traffic video module" and only retains the "core map module + turn prompt module"; the smartwatch maintains the vibration prompt function, and the interface is not adjusted to ensure driving safety.

[0036] (3) Interface adaptation and interaction optimization: In-vehicle terminal (touch operation adapted to driving scenarios): 90% of the screen displays the "navigation map module" (enlarging road details and improving visibility), and the top 10% of the screen displays "minimalist turn prompts" (such as "keep right after 500m to enter the service area"). The touch button size is enlarged to 50×50mm (to avoid accidental touches while driving). On smartphones (running in the background, not actively operated): 20% of the screen displays a "road condition collection window" (real-time preview of the road ahead), and the bottom displays "network status prompts" (such as "weak network, P2P transmission has been switched"). On the smartwatch: 60% of the screen displays a "turn arrow icon" (to intuitively indicate the driving direction), and the bottom 40% of the screen displays a "distance indicator" (such as "3km to the next exit"). A moderate vibration is triggered 5 seconds before turning (to avoid interfering with driving while ensuring user awareness).

[0037] Implementation results: Technical specifications: The navigation interface is smooth even in weak network conditions; hardware conflict resolution takes 85ms; the smartwatch's turn prompt accuracy is 100%; offline navigation error in tunnels is ≤50m; and navigation function is normal during 4G signal interruptions. User experience: No need to look down at the smartphone throughout the entire process. Turn prompts are accurately transmitted via vibration from the smartwatch. Navigation efficiency on highways is 30% higher than using the car navigation system alone. The device operation does not distract the driver's attention and meets the requirements for safe driving.

[0038] Example 3 Creative poster design production scenarios: Real-time scenario: The designer needs to create an e-commerce promotional poster (size 800×1200px). The core requirements are: to draw the main poster pattern using a tablet with a stylus (pressure sensitivity support required), to adjust color values ​​and layer parameters using a desktop computer (high-precision display required), and to use a drawing tablet for color adjustment (keyboard shortcuts required). The system needs to learn the designer's usage habits (85% of the time spent drawing on a tablet and 90% of the time spent adjusting parameters on a desktop computer in the past week), and automatically allocate functions the next time the design software is launched, without repeated settings. The drawing data needs to be synchronized in real time (delay ≤20ms).

[0039] Implementation terminal: Desktop computer: macOS system, 27-inch 4K screen (color accuracy ΔE≤1), supports multi-window management, connected to Wi-Fi 6 network, CPU load rate 12%; Tablet: iPadOS system, 12.9-inch screen, supports stylus (4096 levels of pressure sensitivity), on the same Wi-Fi 6 network, CPU load rate 8%; Graphics tablet: Supports pressure sensitivity adjustment and 16 customizable shortcut keys, connects to desktop computer via USB-C, 200pps sampling rate, and 100% connection stability.

[0040] System components: The hardware capability abstraction layer (responsible for encapsulating resources such as tablet touch pen pressure sensitivity and computer display capabilities), the hybrid transmission layer (including the edge node of the building where the designer's studio is located, 300m away from the studio, caching commonly used color value parameter templates and layer structure templates), the global hardware capability collaboration and low-latency adaptation core engine (including the user preference learning module, responsible for scheduling algorithms and interface command generation), and the interface reorganization layer (responsible for refined interface layout and interaction adaptation).

[0041] Implementation steps: (1) User preference learning and template generation: The core engine's user preference learning module statistically analyzed designers' operational data over the past week: tablet drawing accounted for 85% of the time, desktop computer parameter adjustment accounted for 90% of the time, and drawing tablet color adjustment accounted for 70% of the time. The system generates a "user preference weight matrix" based on statistical data: tablet (drawing weight 0.85), desktop computer (parameter tuning weight 0.9), and drawing tablet (color adjustment weight 0.7), and caches the matrix to edge nodes; The core engine has been updated with a new scheduling algorithm: the "drawing canvas" function will be automatically assigned to the tablet and the "color value parameter adjustment" function will be assigned to the desktop computer the next time the design software is launched, eliminating the need for designers to manually set these functions and improving operational efficiency.

[0042] (2) Hardware capability registration and collaborative scheduling: The hardware capability abstraction layer collects and reports resources from each terminal: Tablet: "Stylus pressure sensitivity (4096 levels), display capability (2732×2048 resolution)", status "Idle, load rate 8%"; Desktop computer: "4K color accuracy display capability, multi-window management capability", status "idle, load rate 12%"; Graphics tablet: "Pressure-sensitive color adjustment capability, 16 customizable shortcut keys", status "Idle, load rate 5%"; The core engine performs global scheduling: tablet drawing data (low latency requirement) is transmitted directly from desktop computer to tablet via P2P connection (measured latency 18ms) to ensure real-time synchronization; color value parameter data (including template files, large data volume) from desktop computer is transmitted through edge nodes (measured latency 32ms) to reduce repeated loading; shortcut key commands from the drawing tablet are directly synchronized to desktop computer without network transmission, with a response time ≤100ms.

[0043] (3) Refined interface refactoring: Tablet version (compatible with stylus): 100% screen display of the "Drawing Canvas Module" and hiding all parameter panels (to avoid interfering with drawing); when designers zoom the screen with two fingers, they can call up the "Brush Thickness Adjustment" shortcut function to meet the needs of quick operation; Desktop PC version (compatible with keyboard and mouse): The left 20% of the screen displays the "Color Value Parameter Panel" (including RGB value adjustment and color curve tools), the right 20% of the screen displays the "Layer Management Panel" (supports layer hiding / showing and transparency adjustment), and the middle 60% of the screen displays the "Canvas Preview Module" (synchronizing with tablet drawing content); the top of the panel has a fixed "Drawing Tablet Shortcut Key Tips" (such as "Shortcut Key 1 = Red Tone Adjustment, Shortcut Key 2 = Color Level +5"), which is convenient for designers to quickly refer to; On the drawing tablet: There is no visual interface. Shortcut key operations are synchronized to the "color value parameter panel" of the desktop computer in real time. For example, if the designer presses the shortcut key 2, the desktop computer will automatically increase the color level parameter by 5, without the need for manual adjustment.

[0044] Implementation results: Technical specifications: After learning user preferences, the hardware scheduling accuracy is 92%, the drawing data synchronization latency is stable at 18-22ms, the desktop computer color value adjustment response time is ≤500ms, and the graphics tablet shortcut key synchronization response time is ≤100ms. Design efficiency: Poster production time has been reduced from 2.5 hours in traditional single-device design to 1.5 hours. Designers report that "there is no need to frequently switch device functions, and concentration is improved by 50%." The shortcut keys on the drawing tablet improve color adjustment efficiency by 40%, and the high-precision display of the 4K screen ensures accurate color value adjustment.

[0045] Example 4 Outdoor power line inspection scenario: Real-time scenario: Power engineers inspecting high-voltage lines in the suburbs (8km in total, no 5G signal, only 3G network coverage) need to complete the following tasks: collect equipment temperature data with an infrared thermometer, enter inspection records (including temperature values ​​and equipment status) on a portable tablet, and locate and mark fault points with a smartphone; data must be synchronized in real time (synchronization success rate ≥95% under weak network conditions), core data (such as temperature values) must be retained when equipment is offline, and automatic data transmission must be performed after reconnection, while avoiding lag on the portable tablet due to multitasking (load rate ≤70%).

[0046] Implementation terminal: Smartphone: Android system, supports 4G / 3G / 2G networks, GPS positioning accuracy 10m, 8GB memory, operating on 3G network, CPU load rate 25%; Portable tablet: Windows operating system, 8-inch touchscreen, supports data entry, 4GB RAM, connected to the same 3G network, CPU load rate 15%; Infrared thermometer: Industrial-grade equipment, supports temperature measurement from -20℃ to 650℃, connects to a portable tablet via Bluetooth 5.0, sampling rate 1 time / second, and buffer capacity ≤100 data entries.

[0047] System components: The hardware capability abstraction layer (responsible for hardware resource reporting and status synchronization of infrared thermometers, tablets, and mobile phones), the hybrid transmission layer (including edge nodes of towns near the inspection area, covering a radius of 8km, and pre-caching power inspection record templates), the global hardware capability coordination and low-latency adaptation core engine (responsible for load balancing scheduling and offline data management), and the interface reorganization layer (responsible for offline interface adaptation and data retransmission prompts).

[0048] Implementation steps: (1) Transmission configuration in weak network environment: Engineers start all terminals and connect them to the system. The hardware capability abstraction layer reports the resources of each terminal: smartphones have "GPS positioning capability + 3G communication capability", portable tablets have "touch input capability + Bluetooth communication capability", and infrared thermometers have "temperature acquisition capability". The hybrid transport layer allocates transmission channels based on the 3G network bandwidth (≤500kbps): Temperature data from the infrared thermometer (small data volume, 100 bytes / data): transmitted via P2P direct connection between a portable tablet and a smartphone, with a measured latency of 65ms; Inspection records of portable tablets (large data volume, 2MB / copy, including pre-cached templates): transmitted in blocks through edge nodes (500KB per block, supports breakpoint resumption) to avoid excessive bandwidth consumption in a single transmission. Edge nodes pre-cache "power inspection record templates" (including fixed fields such as equipment number and inspection items). After the portable tablet is connected to the system, it can directly call the locally cached template without repeated downloads, saving network resources.

[0049] (2) Offline adaptation and data caching: When an engineer enters a signal dead zone (3G network interruption), the system automatically triggers offline mode: On smartphones: Hide the "Real-time Synchronization Status Module" and keep the "Data Cache Module" (which displays the temperature values ​​of the 12 collected devices). Display an "Offline Prompt" at the top (e.g., "No network currently, data is cached"). Portable tablet: 70% of the screen displays the "Temperature Data Input Module" (including the temperature value input box and the device status drop-down menu), and the bottom 30% of the screen displays the "Offline Cache Progress" (e.g., "5 records have been cached, totaling 1.2MB"). Infrared thermometer: Continues to collect temperature data and caches the data to a portable tablet via Bluetooth. The cached data is sorted by collection time to avoid data loss.

[0050] (3) Hardware load balancing scheduling: During peak inspection periods (when engineers simultaneously input 3 inspection records and receive temperature measurement data in real time), the portable tablet runs 3 tasks: "data input", "temperature measurement data parsing", and "map positioning", and the CPU load rate rises to 65% (close to the 70% threshold). The core engine detected that the portable tablet's load was approaching its limit and automatically started a load balancing strategy: transferring the "map positioning" capability to the smartphone (current load rate is 25%, resources are sufficient). The smartphone is responsible for locating and marking the fault point. The location data is directly synchronized to the portable tablet via P2P connection, reducing the load rate of the portable tablet to 42% to avoid lag. The positioning error is ≤10m, which meets the accuracy requirements for fault point marking.

[0051] (4) Data retransmission after network connection: When the engineer moves out of the signal dead zone (reconnects to the 3G network), the system automatically initiates the data retransmission process. The portable tablet retransmitted 5 offline cached inspection records through the edge node, and the actual time was 28 seconds (the transmission time of a single record is ≤6 seconds). The smartphone synchronizes location data to the portable tablet, and the portable tablet interface returns to "real-time synchronization status" and displays the message "Re-transfer complete, 5 records in total"; Temperature data cached by the infrared thermometer is synchronously uploaded to the edge node to complete data backup.

[0052] Implementation results: Technical specifications: 98% data synchronization success rate in weak network environment, offline data retransmission time ≤30s, maximum load rate of portable tablet 65% (≤70% threshold), GPS positioning error ≤10m, data entry error rate reduced from 5% of traditional manual recording to 0; Inspection efficiency: The inspection time for an 8km high-voltage line has been reduced from 4 hours of traditional manual recording to 2.5 hours. Engineers no longer need to return to the office to re-enter data. After the inspection is completed, they can submit a report directly through the system, improving work efficiency by 37.5%.

[0053] Example 5 Home video sharing scenarios: Real-time scenario: Family members can share video resources on weekends: Parents can use a smart screen to play family-friendly animations (4K quality required) and use their smartphones to control playback progress and content permissions (blocking violent scenes in the animations); children can use tablets to watch the animations (the tablets must be compatible with children's operating habits, such as large icons); the system is required to learn family usage habits (parents use the smart screen 90% of the time and children use the tablet 80% of the time). If the smart screen is used simultaneously (parents watch TV series + children watch animations), the needs of parents will be prioritized, and the animation content will be 100% anonymized.

[0054] Implementation terminal: Smart screen: HarmonyOS system, 55-inch 4K screen, supports video playback and 4K decoding, connected to a home 100M Wi-Fi network, CPU load rate 20%; Parental smartphone: Android system, supports permission management, 12GB memory, on the same Wi-Fi network, CPU load rate 15%; Kids tablet: iPadOS system, 8.3-inch touchscreen, supports Kids mode, 6GB memory, on the same Wi-Fi network, CPU load rate 10%.

[0055] System components: The hardware capability abstraction layer (responsible for hardware resource reporting and status synchronization of smart screens, mobile phones, and tablets), the hybrid transmission layer (including the edge node of the community where the home is located, 500m away from the home, caching the desensitized template of parent-child animation), the global hardware capability collaboration and low-latency adaptation core engine (including user preference learning module and permission management module), and the interface reorganization layer (responsible for child-friendly interface adaptation and content desensitization display).

[0056] Implementation steps: (1) Permission settings and content anonymization: Parents can set rules through the "permission management module" on their smartphones: "Block all violent scenes in the animation that are ≥3 seconds long" and "Only show the animation playback interface on the child's tablet and hide the settings button." Edge nodes load "parent-child animation desensitization templates" (marking the timeline and alternative scenes for violent scenes). After the smart screen starts playing the animation, the core engine automatically identifies the violent scenes (based on timeline matching) and triggers content desensitization. Parent's smartphone app: Displays the full animation and includes "Pause / Fast Forward" and "Permission Modification" buttons at the bottom for easy parental control of playback and permission adjustments; On the children's tablet: violent scenes are automatically replaced with "cartoon sticker screen" (to avoid the influence of inappropriate content), and only a large "play / pause" icon (30×30mm in size, adapted to children's operating habits and reducing accidental touches) is displayed at the bottom.

[0057] (2) User preference learning and conflict resolution: The core engine's user preference learning module collects family usage data within one week: parents use smart screens to play videos 90% of the time, and children use tablets to watch videos 80% of the time, generating "family usage preference weights". On a weekend evening, a parent clicked "Play TV series" on the smart screen, while the child simultaneously clicked "Play animation" on the tablet. Both requested to use the smart screen's playback capabilities, triggering a hardware conflict. The conflict arbitration module of the edge computing node makes decisions based on priority: parents use the smart screen more frequently (related to historical preferences), and "playing TV series" is the current active operation of parents, so the smart screen is given priority to parents; The child's tablet automatically enters the "playback queue," and the interface displays "Queuing, estimated waiting time 25 minutes (remaining duration of the parent's episode)," while also recommending "other currently available animations" to enhance the child's user experience.

[0058] (3) Hardware capability adaptation and interface adjustment: Smart screen hardware capability adaptation: After registering "4K display capability + stereo output capability", it will automatically match the "video playback" requirement, and the interface will display "4K drama screen (95% screen coverage) + playback progress bar (bottom 5%)", which supports parents to pause / fast forward via voice control; Parent's smartphone interface: Displays "TV series control module" (including pause / fast forward and resolution adjustment buttons) + "permission status prompt" (such as "violent scenes have been blocked, currently in child safety mode"); Children's tablet interface: When waiting to play, it displays "animated poster + estimated waiting time"; when playing, it displays "simplified playback interface" (only retaining large play / pause buttons and animation screen, without complicated operation options); The system synchronizes the status of each terminal with heartbeat packets at 10ms intervals: if the smart screen loses power unexpectedly (goes offline), the core engine automatically transfers the playback task to the parent's smartphone, and the phone interface switches to the "TV series playback module" to ensure that video playback is uninterrupted.

[0059] Implementation results: Technical specifications: 100% accuracy in anonymizing animation content; 95% of user preference scheduling matches family usage habits; 180ms time to resolve hardware conflicts; and ≤2s time to switch from offline smart screen to mobile phone. Family Experience: Parents can remotely control playback and permissions via smartphone without manually switching devices. Children are safe and controllable when operating the tablet without the risk of accidental touches. The convenience of family video sharing is 80% higher than the traditional "single device playback". Parents report that "there is no need to worry about children being exposed to inappropriate content, and we feel more at ease using it".

[0060] Comparative Example 1 Traditional cloud-based centralized adaptation solutions: Solution Overview: The mainstream "cloud-based centralized adaptation" solution in the existing technology is adopted: multiple terminals (laptops, smartphones, and wireless headphones of the same type as in Example 1) achieve data synchronization and interface adaptation through a remote cloud server. There is no hardware capability abstraction layer, the transmission relies only on a single cloud channel, there are no edge nodes and P2P direct connection functions, and the interface layout is fixed (it cannot be dynamically reorganized according to hardware calls).

[0061] Test scenario: Completely consistent with Example 1: 2-hour remote video conference at home, sharing PPT from a laptop, presenting via smartphone, transmitting audio through wireless headphones, and a 500M Wi-Fi network environment at home.

[0062] Key Indicator Comparison: Regarding transmission latency, the transmission latency of this invention is stable at 28-42ms, while the transmission latency of traditional cloud solutions is ≥85ms. This invention reduces latency by more than 50% through edge node acceleration and direct P2P connection, ensuring a smooth and uninterrupted process. Regarding interface synchronization success rate, this invention achieves 100%, while traditional cloud solutions only reach 82%. The real-time status synchronization mechanism of this invention effectively avoids operational disruptions caused by synchronization failures. Regarding hardware conflict resolution time, this invention's solution is ≤80ms, while traditional cloud solutions are ≥500ms. The edge arbitration module of this invention significantly improves conflict handling efficiency. Regarding hardware offline degradation success rate, this invention achieves 100%, while traditional cloud solutions only reach 30%. The three-level degradation strategy of this invention ensures functional continuity when hardware is offline. Regarding weak network lag rate (bandwidth reduced to 2Mbps), this invention's solution has a lag rate of 0%, while traditional cloud solutions have a lag rate of 35%. The dynamic channel switching mechanism of this invention better adapts to weak network environments.

[0063] Traditional centralized cloud-based adaptation solutions suffer from drawbacks such as high latency, poor synchronization, hardware waste, and offline function interruption due to "single transmission channel, lack of hardware scheduling, fixed interface, and no offline adaptation". This invention completely solves the above problems through a full-link collaborative design of "hardware capability abstraction + edge-P2P hybrid transmission + global scheduling + dynamic interface reorganization". It has significant advantages in the smoothness, stability, resource utilization and user experience of multi-terminal collaboration, and can meet the actual use needs of various scenarios such as remote office, in-vehicle navigation, creative design, outdoor inspection, and home sharing.

[0064] Compared with Examples 1-5 and Comparative Example 1, the core difference between the present invention (Examples 1-5) and the traditional cloud-based centralized adaptation solution (Comparative Example 1) lies in the fact that the former solves the common pain points of the latter, such as "high transmission latency, poor synchronization, hardware waste, and weak network / offline failure", through the end-to-end collaboration of "hardware capability abstraction + edge-P2P hybrid transmission + global scheduling + dynamic interface reorganization", and shows stable advantages in different application scenarios.

[0065] In terms of transmission latency, the solution of this invention controls the latency to 18-65ms in various scenarios, such as 28-42ms in remote office scenario (Example 1), 45ms in vehicle navigation scenario (Example 2), and 18-22ms in creative design scenario (Example 3). In contrast, Comparative Example 1 relies on a single cloud channel, and the latency is generally ≥85ms, which can only meet the basic data synchronization and cannot support low latency requirements (such as video conferencing screen synchronization and navigation turn prompts).

[0066] Regarding the interface and data synchronization success rate, the solution of this invention achieves a 100% synchronization success rate in all embodiments thanks to P2P direct connection state synchronization and edge node caching, and maintains 98% even in the weak network environment of outdoor inspection scenario (Example 4); Comparative Example 1, due to the cloud transmission being easily affected by network fluctuations, has a synchronization success rate of only 82%, and often suffers from problems such as unsynchronized page turning in office meeting PPTs and loss of design data.

[0067] In terms of hardware scheduling and conflict handling efficiency, the solution of this invention uses a global scheduling algorithm and a conflict arbitration mechanism to resolve hardware conflicts in 80-180ms (e.g., 85ms for camera conflicts in vehicle scenarios and 180ms for conflicts in home-shared smart screens), and the hardware utilization rate reaches 90% (no idle resources). In contrast, Comparative Example 1 has no hardware scheduling capability and can only work independently with a single device. The hardware utilization rate is less than 55% (e.g., idle mobile phone camera), and the conflict handling time is ≥500ms, which can easily lead to function interruption.

[0068] The ability to adapt to weak networks and offline conditions is a significant advantage of the present invention: In outdoor inspection (Example 4), data retransmission time is ≤30s under weak 3G network conditions through block transmission and offline caching; in vehicle tunnel (Example 2), the navigation error is ≤50m when offline; while in contrast, Comparative Example 1 has a 35% stuttering rate in weak network (bandwidth 2Mbps) and the function is directly interrupted when offline, which cannot meet the needs of complex scenarios.

[0069] In terms of user experience, the solution of this invention adapts the interface and interaction according to the scenario (such as enlarging the touch button in the car scenario and the large icon in the children's tablet), and ensures the continuity of function through hierarchical degradation (hiding the failed module, switching the backup hardware, and guiding the reconnection) when the hardware is offline; compared with Comparative Example 1, the interface is fixed and there is no adaptation and degradation mechanism. The operation is laggy when the network is weak and it cannot be used when offline. The user operation efficiency is 30%-60% lower than that of the solution of this invention.

[0070] In summary, the solution of this invention is not a single-scenario optimization, but rather achieves a comprehensive surpassing of traditional cloud solutions in multiple scenarios such as remote work, in-vehicle, design, outdoor, and home through the flexible combination of core technology modules, and has strong versatility and industrial application value.

[0071] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of protection of the claims.

[0072] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A multi-terminal collaborative intelligent interactive interface dynamic adaptation system, characterized in that, It includes a multi-terminal, hardware capability abstraction layer, hybrid transmission layer, global hardware capability coordination and low-latency adaptation core engine, and interface reorganization layer. The modules work together to achieve multi-terminal hardware resource scheduling and dynamic adaptation of the interactive interface, as detailed below: The multi-terminal includes at least two types of smart terminals, and the smart terminals are selected from any combination of mobile phones, computers, tablets, smart wearable devices, and vehicle terminals; The hardware capability abstraction layer is used to transform the hardware resources of each of the multi-terminal terminals into standardized and schedulable capability units, provide a unified calling interface to the global hardware capability collaboration and low-latency adaptation core engine, and support edge computing nodes of the hybrid transport layer to cache the global hardware capability status view. The hybrid transmission layer is used to provide low-latency transmission channels for scheduling instructions, hardware data and interface status data of the core engine for global hardware capability coordination and low-latency adaptation. It includes edge computing nodes deployed at the edge of the regional network and P2P direct connection networks between devices, and can dynamically allocate transmission channels according to data transmission requirements. The global hardware capability coordination and low-latency adaptation core engine, as the system decision center, integrates hardware capability scheduling, transmission strategy selection and interface adaptation rules. It can parse user operation behavior to generate hardware combination schemes and interface reorganization instructions, and realize the synchronization of interface status on each end. The interface reconfiguration layer receives interface reconfiguration instructions from the global hardware capability coordination and low-latency adaptation core engine, dynamically adjusts the interactive interface layout of each of the multi-terminal terminals, adapts the interface according to the terminal hardware capabilities and transmission status, and triggers interface degradation when the capability unit is offline or transmission is interrupted.

2. The multi-terminal collaborative intelligent interactive interface dynamic adaptation system as described in claim 1, characterized in that, The specific methods by which the hardware capability abstraction layer implements standardized capability unit transformation and unified calling interface include: automatically reporting hardware resource type, performance parameters, and current status when the device is connected, encapsulating them into capability units through the Protobuf protocol and registering them to the global hardware capability collaboration and low-latency adaptation core engine; synchronizing hardware status with the global hardware capability collaboration and low-latency adaptation core engine through heartbeat packets at 10ms intervals; the hardware resource types include keyboard, camera, microphone, sensor, display module, and stylus; the performance parameters include camera resolution, screen refresh rate, and sensor sampling rate; the current status includes hardware idle status, hardware occupancy status, hardware load rate, and terminal battery level.

3. The multi-terminal collaborative intelligent interactive interface dynamic adaptation system as described in claim 1, characterized in that, The specific structure of the hybrid transport layer includes: the edge computing node has a built-in transport strategy decision module, a global state cache module, and a conflict arbitration module; the P2P direct connection network is built based on WebRTC technology and supports NAT traversal; the global state cache module is used to cache the capability unit status of each terminal, user operation preference data, and commonly used interface reassembly templates.

4. The multi-terminal collaborative intelligent interactive interface dynamic adaptation system as described in claim 1, characterized in that, The dynamic allocation mechanism of the transmission channels in the hybrid transmission layer satisfies: For low-latency data, including hardware call commands, interface operation status data, and real-time interactive data, P2P direct connection channels are preferred, and the transmission latency does not exceed 50ms. For large volumes of data, including high-definition video streams, image materials, and large file data, edge node acceleration channels are used; When the packet loss rate of the P2P direct connection channel exceeds 10%, it will automatically switch to the edge node acceleration channel. When the load rate of the edge computing node exceeds 80%, some large amounts of data will be diverted to the P2P direct connection channel. The P2P direct connection network supports simultaneous access by multiple terminals; when the P2P connection between any two terminals is interrupted, data is automatically forwarded through edge computing nodes to maintain interface state synchronization.

5. The multi-terminal collaborative intelligent interactive interface dynamic adaptation system as described in claim 1, characterized in that, The specific execution steps of the global hardware capability scheduling algorithm of the core engine for global hardware capability coordination and low-latency adaptation include: Step 1: Requirements analysis. By identifying the user's operation behavior on any of the multi-terminal devices, the core hardware capabilities required for the current operation are extracted. Step 2: Capability matching. Query the global hardware capability status view of the edge computing node cache and filter out capability units that meet the performance parameter requirements and are in an idle state. Step 3: Optimal combination generation. The final hardware capability combination scheme is determined by combining the load rate of each terminal and the transmission cost. The load rate of a single terminal shall not exceed 70%. In terms of transmission cost, terminals within the same local area network are given priority, while also taking into account the user's historical operation preferences.

6. The multi-terminal collaborative intelligent interactive interface dynamic adaptation system as described in claim 1, characterized in that, The specific logic for the global hardware capability coordination and low-latency adaptation core engine to generate interface reconfiguration instructions includes: Each invoked capability unit is assigned a corresponding interface function module. When the terminal camera capability unit is invoked, the terminal interface displays the camera preview module and the shooting control module; when the terminal screen display capability unit is invoked, the terminal interface displays the data sharing module and the interactive list module. The interface loading priority is adjusted according to the real-time latency of the transmission channel: when the transmission latency exceeds 100ms, only the core functional modules are rendered; when the transmission latency does not exceed 50ms, the complete interface consisting of the core functional modules and auxiliary functional modules is loaded.

7. The multi-terminal collaborative intelligent interactive interface dynamic adaptation system as described in claim 1, characterized in that, The specific methods by which the interface reorganization layer dynamically adjusts the interface layout include: The interface adopts a modular architecture, splitting it into core functional modules and auxiliary functional modules, supporting on-demand loading. The core functional modules include shooting control buttons, data display canvas, and basic interactive controls. The auxiliary functional modules include parameter setting panel, historical operation record panel, and device connection list. The interface module's position, size, and interaction method are adaptively adjusted based on the terminal's screen size and hardware capabilities: for touch-screen terminals, the interface module displays large touch buttons; for keyboard and mouse-controlled terminals, the interface module displays a shortcut key prompt panel.

8. The multi-terminal collaborative intelligent interactive interface dynamic adaptation system as described in claim 1, characterized in that, The core engine for global hardware capability coordination and low-latency adaptation also includes a user preference learning module. This module analyzes users' choices of interface layouts and hardware calling habits for various terminals, updates the decision weights of the global hardware capability scheduling algorithm, and optimizes the generation logic of interface reconfiguration instructions.

9. The multi-terminal collaborative intelligent interactive interface dynamic adaptation system as described in claim 1, characterized in that, When multiple terminals compete for the same capability unit, the conflict arbitration module of the hybrid transport layer edge computing node has the following arbitration priority order: the terminal currently associated with the user's operation takes priority over the historically frequently used terminal, and the historically frequently used terminal takes priority over the terminal with the lowest load rate.

10. The multi-terminal collaborative intelligent interactive interface dynamic adaptation system as described in claim 1, characterized in that, The specific methods by which the interface reorganization layer triggers interface degradation include: when a capability unit is detected to be offline or its transmission is interrupted, the failed interface module directly associated with that capability unit is first hidden, and a hardware offline or transmission interruption status is explicitly displayed on the interface; if the system detects that there is an idle backup capability unit that is functionally compatible with the failed capability unit, the system automatically associates the backup capability unit and synchronously updates the interface layout, migrating the function of the original failed module to the interface module corresponding to the backup capability unit; if there is no compatible backup capability unit, a prompt to connect available hardware is displayed on the interface, and 3-5 compatible hardware types that match the current function are listed.

Images