Broadcasting three-end collaborative AI scheduling system and unloading method based on mobile phone surplus computing power

Abstract

The application discloses a kind of based on mobile phone surplus computing power broadcast three end collaborative AI scheduling system and task unloading method, it is related to broadcast terminal side computing power scheduling and multi-terminal AI collaborative technical field.This system includes broadcast operator service end, intelligent terminal and broadcast large screen audio-visual terminal, and broadcast operator service end is responsible for account authentication, equipment binding, user side computing power resource pool construction and dynamic task scheduling;Intelligent terminal includes at least one smart phone as AI computing power bearing node, relies on INT4 grouping quantization lightweight big model multiplex terminal surplus computing power to complete all AI intelligent body inference and multi-modal task calculation;Broadcast large screen audio-visual terminal carries lightweight message analysis plug-in and is responsible for video decoding playing.The system and method provided by the application do not need to deploy edge computing power equipment, do not need to iterate inventory large screen audio-visual terminal hardware, can effectively reduce the cost of broadcast AI business landing, solve video bandwidth squeeze, offline function failure, computing power waste and other problems.

Description

Technical Field

[0001] This invention belongs to the field of computing power scheduling and multi-terminal AI collaboration technology for broadcasting terminals. Specifically, it relates to an AI intelligent agent dynamic scheduling system and task offloading method that utilizes the surplus computing power of smartphones to achieve collaboration among broadcasting servers, smartphones, and large-screen broadcasting audiovisual terminals. It is mainly applicable to low-cost AI empowerment, multi-terminal AI collaborative interaction, offline intelligent agent interaction, and broadcasting multimodal audiovisual and smart home integration business scenarios in scenarios where the computing power of various types of large screens such as existing broadcasting set-top boxes, split large-screen TVs, and integrated smart TVs is insufficient. Background Technology

[0002] Currently, when cable television networks promote value-added services such as AI-powered intelligent agent dialogue, multimodal content recognition, and intelligent interaction in homes and government / enterprise offices, they generally face the problem of insufficient computing power in existing large-screen audiovisual terminals, making them unable to support large-scale model inference. Existing computing power expansion solutions in the industry mainly fall into three categories:

[0003] The first category involves deploying dedicated edge computing servers for residential communities. This method involves deploying edge computing hardware in the broadcasting community's data center to offload set-top box AI tasks to edge servers for computation. This solution requires additional hardware, data center upgrades, network upgrades, and long-term maintenance investment, resulting in high construction costs, long deployment cycles, and resource waste issues such as severe idleness during off-peak hours and insufficient computing power during peak hours.

[0004] The second category is set-top box hardware upgrade and expansion solutions. This involves replacing set-top boxes with new ones equipped with NPU chips in batches to enhance local AI inference capabilities. This solution requires iterative replacement of a massive number of existing set-top boxes nationwide, resulting in extremely high terminal replacement costs, the inability to reuse existing equipment, and significant difficulties in implementation.

[0005] The third category comprises existing dual-screen interaction solutions for mobile phones and set-top boxes. Existing patents and products only implement basic signaling interactions such as mobile phone QR code binding, remote control playback, and screen projection. The mobile phone merely serves as a remote control input device and does not participate in any AI intelligent agent computing tasks. There are also a few general-purpose TV computing power offloading solutions, but these are only suitable for point-to-point direct connection to general smart TV LANs. They lack a dedicated broadcast server scheduling architecture, broadcast network adaptation capabilities, and home computing power pool management mechanisms. They cannot adapt to standardized broadcast IPTV service systems and are incompatible with multi-form large-screen hybrid networking scenarios involving split-screen TVs and traditional set-top boxes. Existing single-mobile phone computing power scheduling solutions only optimize computing power fluctuations for a single device, lacking multi-terminal computing power pool weighted balancing and standardized LAN offline communication protocols, thus failing to support concurrent AI interaction scenarios across multiple screens throughout the house.

[0006] The existing technical solutions have the following core defects:

[0007] First, traditional computing power expansion relies heavily on new dedicated hardware, which involves large hardware investments, extensive network upgrades, and high barriers to implementation, making it difficult for small and medium-sized broadcasting branches to deploy on a large scale.

[0008] Secondly, existing solutions fail to utilize the vast surplus computing power of home smartphones. Current mainstream smartphones are generally equipped with high-performance NPUs and large amounts of memory. Actual testing shows that in typical user scenarios such as WeChat, short videos, and web browsing, the average computing power utilization of the entire device is only about 9%. Even with a lightweight AI agent running simultaneously, the average surplus computing power can still be maintained in the range of 65% to 75%, indicating a significant waste of surplus computing power.

[0009] Furthermore, existing technologies lack a dedicated three-terminal collaborative scheduling architecture for broadcasting, and lack unified server binding, home computing resource pool management, and dynamic load balancing mechanisms for AI tasks, making it impossible to achieve whole-house collaborative AI computing across multiple mobile phones and set-top boxes. Simultaneously, traditional solutions suffer from issues such as video services and AI computing tasks running concurrently, bandwidth congestion, and playback stuttering. Moreover, large amounts of raw user audio, video, and image data are uploaded to edge data centers, resulting in high interaction latency and a significant risk of user privacy leaks.

[0010] Finally, existing technologies lack robust offline fallback capabilities. When there are network failures or fluctuations in the private network, the AI ​​interaction function of the broadcast screen becomes completely ineffective, resulting in poor system robustness and user experience. Summary of the Invention

[0011] To address the shortcomings of existing technologies, the present invention aims to provide a broadcasting three-terminal collaborative AI scheduling system and task offloading method based on the surplus computing power of mobile phones. The system and method can improve the shortcomings of existing broadcasting AI computing power expansion solutions, such as high hardware cost, great difficulty in transformation, low utilization rate of computing power resources, lack of three-terminal collaborative scheduling mechanism, poor privacy protection, and weak offline availability.

[0012] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0013] A collaborative AI scheduling system for broadcasting three terminals based on the surplus computing power of mobile phones, the system architecture includes: a broadcasting operator server, a smart terminal and a large-screen audio-visual terminal;

[0014] The broadcasting operator's server serves as the global scheduling hub, and is equipped with an account binding module, a computing power scheduling module, a message gateway module, and a CA authentication module. These modules are used to complete the binding of accounts on the three devices, build a user-end computing power resource pool, dynamically allocate AI tasks, forward lightweight messages, and perform security authentication for broadcasting services.

[0015] The intelligent terminal includes at least one smartphone as an AI computing power carrier terminal. It is equipped with a broadcasting collaboration APP and has a built-in local INT4 quantized AI inference engine and computing power monitoring module. It is used to report its own CPU and NPU surplus computing power ratio, remaining power and network status in real time, and undertake all broadcasting AI intelligent agent computing tasks and multimodal recognition inference tasks.

[0016] The large-screen audio-visual terminal is used to complete broadband media video decoding, encryption and decryption, and large-screen image rendering services; the large-screen audio-visual terminal is equipped with a lightweight message parsing plugin, which is used to receive and execute structured control commands pushed by the smart terminal.

[0017] Furthermore, in the aforementioned AI scheduling system for collaborative broadcasting across three terminals based on surplus computing power of mobile phones, the large-screen audiovisual terminal can be any one or more of the following: an existing DVB+IPTV dual-mode broadcasting set-top box with a display screen, a split-screen TV with external broadcasting authentication peripherals, or an integrated smart TV with a broadcasting CA authentication communication module; the lightweight message parsing plugin of the existing DVB+IPTV dual-mode set-top box is a non-intrusive software plugin; the split-screen TV completes network access verification through external broadcasting compliance authentication peripherals; and the integrated smart TV directly accesses the user-end computing power resource pool by relying on its built-in CA chip and local area network communication module.

[0018] Furthermore, as described above, the broadcasting three-terminal collaborative AI scheduling system based on the surplus computing power of mobile phones also includes various portable smart devices with NPU computing power and Wi-Fi local area network communication capabilities as extended terminals. These extended terminals and the smartphone are jointly incorporated into the user-end computing power resource pool, serving as supplementary computing power nodes when the smartphone's computing power is insufficient, thereby realizing a multi-type terminal hybrid computing power collaborative architecture.

[0019] Furthermore, as described above, in the broadcasting three-terminal collaborative AI scheduling system based on the surplus computing power of mobile phones, the broadcasting operator server has an independent video service channel and an AI intelligent agent computing task channel. The video service uses the broadcasting private network link throughout, and the AI ​​intelligent agent computing task is exclusively processed by the intelligent terminal, realizing the hierarchical isolation between streaming media video tasks and AI computing tasks.

[0020] The intelligent terminal completes all AI inference calculations locally and only sends byte-level structured result messages back to the broadcast operator's server or the broadcast large-screen audio-visual terminal, without uploading original voice, image and video sensitive data;

[0021] The system supports an offline closed-loop working mode when the external network is disconnected. It completes device discovery, encrypted communication, and offline task return through a dedicated local area network direct communication protocol between the smartphone and the large-screen audio-visual terminal. After the external network is restored, it automatically completes offline data deduplication and synchronization, realizing seamless switching between online and offline services.

[0022] The AI ​​inference engine on the smart terminal uses a grouped INT4 quantization model, which maps the weights of the original FP32 floating-point model to four-bit integers for storage, and completes the quantization restoration through grouped independent scaling coefficients and zero-point offset correction.

[0023] Furthermore, in the aforementioned AI scheduling system for broadcasting terminals based on surplus computing power of mobile phones, the computing power scheduling module employs a dynamic load balancing algorithm. It collects four core parameters in real time: CPU idle rate, NPU utilization rate, remaining battery power, and network latency of the smart terminal. Intelligent task allocation is completed through a weighted calculation model, with the preset task allocation weight formula being:

[0024] Task allocation weight = (Percentage of surplus computing power × 0.5–0.6) + (Remaining battery power × 0.2–0.3) + (Network quality score × 0.15–0.25)

[0025] The network quality score is obtained by comprehensively quantifying network latency and packet loss rate, with a score range of 0 to 1; the surplus computing power ratio is calculated based on the real-time computing power occupancy of smart terminals; when the surplus computing power ratio of smart terminals drops to 30% to 40%, the system automatically triggers task migration, switching to other bound smart terminals of the user to undertake the task.

[0026] Furthermore, in the aforementioned broadcasting three-terminal collaborative AI scheduling system based on the surplus computing power of mobile phones, the dedicated local area network direct connection communication protocol is used to realize the three-terminal closed-loop operation in the scenario of external network disconnection and offline operation. The protocol includes a device discovery mechanism, a handshake negotiation mechanism, an encrypted channel mechanism, an offline task return mechanism, and a server-side recovery and synchronization mechanism.

[0027] Furthermore, in the above-mentioned AI scheduling system for broadcasting three terminals based on the surplus computing power of mobile phones, the device discovery mechanism adopts UDP multicast, the large-screen audio-visual terminal periodically broadcasts device information and account hash values, and the smart terminal filters out unrelated user devices through account hash matching.

[0028] The handshake negotiation mechanism establishes a reliable long connection based on TCP to complete device identity verification, protocol version, and offline service capability negotiation.

[0029] Furthermore, in the aforementioned AI scheduling system for broadcasting three terminals based on the surplus computing power of mobile phones, the encryption channel mechanism adopts a lightweight encryption scheme. During the online binding phase, a unique user pre-shared key is issued by the broadcasting operator's server. In offline scenarios, the smart terminal and the large-screen audiovisual terminal generate a temporary session key through the pre-shared key and a random salt value.

[0030] Furthermore, in the aforementioned collaborative AI scheduling system for broadcasting terminals based on the surplus computing power of mobile phones, the offline task feedback mechanism is as follows: when the external network is disconnected, the large-screen audiovisual terminal stores the user's AI interaction request in a local offline cache queue and sends it to the smart terminal through a local area network encrypted channel; the smart terminal calls the local INT4 quantized AI model to complete inference, generates structured instructions, and sends them back to the large-screen audiovisual terminal for execution; the large-screen audiovisual terminal retains a task record with a unique request ID.

[0031] The server-side recovery synchronization mechanism is as follows: After the external network is restored, the smart terminal and the large-screen audio-visual terminal start the segmented breakpoint resume synchronization, complete global data deduplication based on the unique request ID, clear the local cache after synchronization is completed, and the system automatically switches back to the server-side online unified scheduling mode; during the offline LAN operation phase, when a user's smartphone is added or offline, the local computing power classification file is synchronized in real time through UDP multicast, and the local computing power resource pool is dynamically updated without the need for the cable TV operator's server to issue synchronization instructions.

[0032] A method for offloading AI tasks collaboratively across three broadcasting terminals based on surplus computing power of mobile phones, based on the broadcasting three-terminal collaborative AI scheduling system as described above, the method includes the following steps:

[0033] S1. Three-terminal binding registration: The smartphone completes CA authentication through the Broadcasting Collaboration APP, and scans the code to complete the three-party binding of the smartphone, large-screen audio-visual terminal and broadcasting operator server. The server establishes a user-end computing power resource pool and issues the user's encryption key.

[0034] S2. Computing power status reporting: Smartphones periodically collect and report local CPU and NPU computing power usage, remaining power and network latency data, and the server updates the computing power resource pool status of the user terminal in real time.

[0035] S3, AI Task Layered Scheduling: The large-screen audio-visual terminal collects user voice and image interaction requests and only uploads request signaling. Based on the computing power pool status and load balancing algorithm, the server offloads AI inference tasks to smartphones with sufficient spare computing power. If the spare computing power of a single smartphone drops to the threshold range of 30% to 40%, the high-load multimodal inference tasks it carries are migrated to other smartphones with sufficient spare computing power in the computing power resource pool for continued execution.

[0036] S4, Local Lightweight Inference: The smartphone completes speech recognition, semantic understanding, and multimodal content recognition inference through the local INT4 quantized AI model, generating byte-level structured result messages;

[0037] S5. Lightweight result feedback and display: Smartphones encrypt and send back structured results, which are then forwarded by the server to the corresponding large-screen audio-visual terminal. The large-screen audio-visual terminal performs pop-up windows, subtitle overlay, video linkage, and device control operations. The video service runs independently and stably throughout the entire process.

[0038] S6. Offline Adaptive Switching and Synchronization: When the network is normal, online three-terminal collaborative scheduling is performed; when the server heartbeat timeout or external network link is detected, the local area network offline closed-loop mode is automatically switched; after the network is restored, offline data synchronization is completed through breakpoint resume and data deduplication mechanism to unify the operating status of the three-terminal devices.

[0039] Compared with existing technologies, the collaborative AI scheduling system and task offloading method for broadcasting terminals based on surplus computing power of mobile phones provided by this invention have the following beneficial effects:

[0040] 1. Unlike the simple interactive solutions of traditional dual-screen projection and local area network point-to-point direct connection, this invention constructs a three-layer closed-loop scheduling system for broadcasting exclusive server-smartphone-large screen audio-visual terminal, realizing unified account binding, intensive management of computing power pool, and dynamic load balancing of multiple terminals;

[0041] 2. Based on real terminal test data, this invention reuses 65% to 75% of the surplus computing power of smartphones, transforming the massive surplus computing power of personal terminals into standardized AI computing power resources for broadcast users. The resource utilization rate is significantly higher than that of traditional fixed edge servers, greatly improving the efficiency of computing power utilization and effectively revitalizing the surplus computing power resources of users.

[0042] 3. By completely separating video services and AI tasks through a dual-channel isolation mechanism, the computing power tasks are prevented from crowding out video transmission bandwidth, ensuring stable and smooth core video playback services of broadcasting, without stuttering, screen tearing, or delay.

[0043] 4. All sensitive data such as raw audio, video, and images are processed locally on the mobile phone, and only lightweight structured results are transmitted over the external network, thus avoiding the risk of privacy leakage from the source and ensuring privacy and security; relying on the INT4 quantization model, it realizes offline AI interaction in all scenarios, and the system has strong fault tolerance and scenario adaptability.

[0044] 5. This invention does not require the deployment of community edge servers or the batch replacement of existing large-screen audio-visual terminal hardware. It can expand computing power simply by upgrading software and deploying an APP, reducing hardware investment by more than 90%. This effectively reduces the cost of AI transformation for broadcasting and improves the industry pain points of traditional solutions, such as high investment, difficulty in implementation, and limited large-scale deployment.

[0045] 6. Based on lightweight, non-intrusive plug-in modification, it is compatible with all types of existing DVB+IPTV dual-mode set-top boxes. The network modification scope is small and the adaptation cost is low, making it suitable for rapid large-scale promotion and deployment in the national broadcasting system. Attached Figure Description

[0046] To further illustrate the above and other advantages and features of this application, the specific embodiments of this application will be described in more detail below with reference to the accompanying drawings. The accompanying drawings, together with the following detailed description, are included in and form a part of this specification. It should be understood that these drawings only depict typical examples of this application and should not be considered as limiting the scope of this application.

[0047] Figure 1 This is a schematic diagram of the overall three-layer collaborative architecture of a broadcasting three-terminal collaborative AI scheduling system based on the surplus computing power of mobile phones, provided in a specific embodiment of the present invention.

[0048] Figure 2 This is a sequence diagram of the three-terminal binding and registration process;

[0049] Figure 3 A complete business process diagram for unloading AI interaction tasks;

[0050] Figure 4 This is a schematic diagram illustrating the dynamic scheduling of a multi-terminal computing power pool in a home environment. Detailed Implementation

[0051] Exemplary embodiments of this application will be described below with reference to the accompanying drawings. For clarity and brevity, not all features of actual implementations are described in the specification. However, it should be understood that many implementation-specific decisions must be made in the development of any such actual embodiment to achieve the developer's specific goals, such as meeting constraints related to the system and business, and these constraints may vary depending on the implementation. Furthermore, it should be understood that while development work can be very complex and time-consuming, such development work is merely a routine task for those skilled in the art who benefit from the content of this application.

[0052] The embodiments or examples disclosed below are used to implement this application. They are merely examples and are not intended to limit this application.

[0053] To address the shortcomings of existing broadcast AI computing power expansion methods, such as reliance on hardware upgrades, high upgrade costs, poor compatibility with existing terminals, interference between video services and AI tasks, lack of offline interaction capabilities, low utilization of computing resources, and insufficient user privacy and security, this invention provides a broadcast three-terminal collaborative AI scheduling system based on the surplus computing power of mobile phones. The overall architecture of this system is as follows: Figure 1 As shown in the diagram, the system consists of three core layers: the broadcast operator's server, smartphones, and large-screen audio-visual terminals. It also includes the overall architecture relationships of the corresponding functional modules of each terminal, the isolation of video and AI dual business channels, and the hierarchical scheduling of computing power.

[0054] Specifically, the system architecture comprises three main components: the broadcast operator's server, smart terminals, and large-screen broadcast audiovisual terminals.

[0055] The cable TV operator's server is deployed at the cable TV center, serving as the global scheduling hub of the system. It is equipped with an account binding module, a computing power scheduling module, a message gateway module, and a CA authentication module, and is responsible for completing the binding of accounts and devices on the three terminals, the construction of user-end computing power resource pools, the hierarchical scheduling of AI tasks, message forwarding, and security authentication.

[0056] The intelligent terminal includes at least one smartphone, which serves as the AI ​​computing power carrier terminal. It is equipped with a broadcasting collaboration APP, and has a built-in local AI inference engine and computing power monitoring module. It is responsible for undertaking all broadcasting AI intelligent agent inference, multimodal recognition, and semantic interaction computing tasks.

[0057] Large-screen audio-visual terminals are deployed in various user scenarios such as homes and government / enterprise offices. As professional video presentation terminals, they only have core video decoding and playback capabilities, and are responsible for IPTV / DVB video decoding, image rendering, and audio and video output services. They are equipped with a lightweight message parsing plugin, which is used to receive and execute lightweight structured instructions processed by the smart terminal, and does not undertake any AI inference computing tasks.

[0058] It should be noted that the computing power nodes of this system are not limited to smartphones, but can be expanded to adapt to portable smart devices with NPU computing power and Wi-Fi local area network communication capabilities, such as Android tablets, iPads, smart learning tablets, and portable AI terminals. When the user's main mobile phone has insufficient computing power, low battery, excessive background usage, or is offline, the system automatically offloads AI tasks to idle portable smart terminals such as tablets, achieving hybrid computing power collaboration among multiple types of terminals.

[0059] The large-screen audio-visual terminal can be any one or more of the following: an existing DVB+IPTV dual-mode cable set-top box with a display screen, a split-screen TV with an external cable authentication peripheral, or an integrated smart TV with a cable CA authentication and communication module. The lightweight message parsing plugin for the existing DVB+IPTV dual-mode set-top box is a non-intrusive software plugin; the split-screen TV completes network access verification through an external cable compliance authentication peripheral; and the integrated smart TV directly connects to the user's computing power pool via its built-in CA chip and LAN communication module.

[0060] Cable TV set-top boxes, split-screen TVs, and integrated smart TVs share the same local area network (LAN) encrypted communication protocol, CA authentication and verification logic, and offline task interaction process. They are uniformly adapted to a two-tiered dynamic computing power management strategy: single-task real-time adaptive degradation and upgrade, and global business cycle adaptive degradation and upgrade. The scheduling logic, encryption mechanism, and offline interaction logic of the three types of large-screen hardware are fully reused. All extended terminals reuse a unified account binding, computing power reporting, LAN communication, and offline scheduling mechanism, and are uniformly incorporated into the user-end computing power resource pool.

[0061] The system achieves efficient AI scheduling across the three terminals of broadcast operators' servers, smart terminals, and large-screen audiovisual terminals through the coordinated operation of the following core supporting mechanisms.

[0062] Mechanism 1: Task Layering and Isolation Mechanism

[0063] This system employs a dual-service layered isolation mechanism, dividing the broadcast user's services into two completely independent task systems: streaming video tasks and AI intelligent agent computing tasks. Streaming video tasks, such as live broadcasting, video-on-demand, time-shift playback, and video decryption and rendering, are exclusively processed by the large-screen audiovisual terminal, utilizing the existing broadcast network video transmission link throughout. AI intelligent agent computing tasks, such as voice question answering, multimodal recognition, intelligent agent conversation, and content understanding, are exclusively processed by the smartphone. The broadcast operator's server constructs independent video service channels and AI message scheduling dual channels, effectively preventing AI data transmission from crowding out video bandwidth and ensuring zero-lag and zero-interference video playback on large screens.

[0064] Mechanism 2: Lightweight Result Transmission Compression Mechanism

[0065] This system employs a "local full computation, lightweight result return" transmission mechanism. After completing all AI inference calculations on the smartphone, it does not upload large amounts of sensitive data such as original audio, original images, and video frames. Instead, it returns only a structured, lightweight message containing text results, device control commands, screen display coordinates, resource IDs, and confidence scores, with the message size compressed to the byte level. This mechanism significantly reduces uplink bandwidth consumption at the terminal, shortens interaction latency, and reduces the external network transmission links for user privacy data, thereby improving data security from the source.

[0066] Mechanism 3: Offline computing power fallback strategy based on INT4 quantization

[0067] This invention provides a pre-installed lightweight AI agent model with grouped INT4 quantization in smart terminals. The model parameters are not limited to specific specifications and can be adapted to lightweight large models with different parameter scales based on the terminal's computing power. The core principle of grouped INT4 quantization is as follows: the weights of the original FP32 floating-point model are mapped to 4-bit integers for storage. The model weights are grouped, with each group configured with an independent scaling factor and zero-point offset parameter. High-precision quantization restoration is achieved through group calibration. With extremely low precision loss, the model memory usage is compressed to 12.5% ​​of the original FP32 model, and NPU inference computing power consumption is reduced by more than 60%. The system adapts to different parameter scales of INT4 quantization models for smartphones with varying computing power: high-end smartphones are adapted to 6B-7B parameter multimodal INT4 quantization models, handling high-load tasks such as video content recognition and posture detection; mid-range smartphones are adapted to 2B-3B parameter lightweight INT4 models, handling medium-load tasks such as speech recognition and program retrieval; and low-end entry-level phones are adapted to ≤1B parameter ultra-lightweight INT4 models, performing only lightweight tasks such as text instruction parsing. Both mid-range and entry-level smartphones can run lightweight large models smoothly locally.

[0068] When the broadcasting private network fails, the external network is disconnected, or the server link times out, the system automatically switches to offline working mode. The smart terminal independently completes all AI inference tasks based on the local INT4 quantization model and completes data interaction with the large-screen audio-visual terminal through direct connection of the local area network, ensuring the uninterrupted operation of broadcasting AI services.

[0069] Mechanism 4: Full Compatibility and Adaptation Mechanism for Existing Terminals

[0070] This invention provides a layered compatibility solution for large-screen audiovisual terminals: For existing DVB+IPTV dual-mode set-top boxes, a non-intrusive, pure software modification solution is adopted, requiring no hardware replacement; separate large-screen TVs can complete network adaptation simply by pairing them with standardized broadcast authentication peripherals; integrated smart TVs have built-in integrated grouping and natively support the system's collaborative scheduling from the factory. None of the three types of large-screen terminals require chip upgrades or additional NPU hardware. Only a lightweight message parsing plugin is added to the large-screen audiovisual terminal side. This plugin has extremely low resource consumption, runs as a low-priority process, does not preempt video decoding threads, and does not interfere with existing core broadcast services such as CA authentication and video decryption. This achieves seamless adaptation for all types of existing broadcast terminals, with a small modification scope, low implementation cost, and strong adaptability.

[0071] Mechanism 5: Dynamic Computing Power Balancing Scheduling Strategy

[0072] This invention constructs a dynamic load balancing algorithm based on measured surplus computing power data from mobile phones. Extensive testing on various terminals has verified that the daily overall surplus computing power of mainstream smartphones is consistently between 65% and 75%, with an optimal average of 70%. This system collects four core parameters in real time: CPU idle rate, NPU utilization rate, remaining battery power, and network latency. A weighted calculation model is used to complete intelligent task allocation. The formula for calculating task allocation weights is as follows:

[0073] Task allocation weight = surplus computing power percentage × 0.5~0.6 + remaining battery power × 0.2~0.3 + network quality score × 0.15~0.25; where each coefficient can take any value within the range and can be adaptively adjusted according to the phone hardware level and user network environment.

[0074] The network quality score is generated based on a comprehensive quantification of network latency and packet loss rate, with a score range of 0 to 1. When the proportion of surplus computing power of a terminal is between 30% and 40% of the threshold, the system determines that the terminal's computing power is insufficient and automatically triggers a task migration mechanism to switch unfinished AI tasks to other backup terminals within the user's system, with a preferred migration threshold of 35%. The system prioritizes allocating multimodal high-load tasks to terminals with sufficient surplus computing power and automatically downgrades terminals with insufficient computing power to be assigned lightweight tasks, achieving dynamic and balanced scheduling of computing power throughout the system. If the surplus computing power of all smart terminals in the computing power pool is below the 30% to 40% migration threshold, the AI ​​inference tasks are automatically and progressively downgraded and lightweighted, prioritizing the retention of basic voice on-demand and channel switching core audiovisual instruction inference, while shutting down high-load value-added tasks such as image recognition and posture detection. When multiple terminals compete for the same inference task, the task is allocated based on its weight value, with the terminal with the highest weight value taking the task first.

[0075] Mechanism Six: Local Area Network Direct Connection Communication Protocol System

[0076] To achieve efficient and reliable collaboration among the three terminals in offline scenarios, this invention designs a dedicated LAN direct connection communication protocol system for large-screen audio-visual terminals and smartphones, used to realize a closed-loop service for the three terminals in offline scenarios. This LAN direct connection communication protocol system includes five core mechanisms: device discovery, handshake negotiation, encrypted channel, offline task return, and server-side recovery synchronization.

[0077] Device discovery mechanism: UDP multicast communication is adopted, with the preferred multicast address 239.255.10.10 and port 7612. The large-screen audio-visual terminal periodically broadcasts device information and account hash values. Mobile phones accurately match the devices through the account hash and filter out unrelated user devices, enabling fast and accurate discovery of terminals within the home.

[0078] Device handshake mechanism: After successful device discovery, a reliable long connection is preferably established based on TCP port 7613. Both parties exchange device ID, protocol version, and offline service capability information to complete device identity verification and service capability negotiation.

[0079] Encrypted channel mechanism: During the online binding phase, the server issues a unique pre-shared key to each user. In offline scenarios, both ends generate a temporary session key based on the pre-shared key and a random salt value. This invention uses the AES-128-CBC symmetric encryption algorithm, which has low computational overhead and is suitable for the computing power of low-performance large-screen audio-visual terminals. It is paired with the SM3 domestic hash algorithm to complete message digest integrity verification. While ensuring data transmission security, it does not put computational load pressure on existing large-screen audio-visual terminals, demonstrating excellent adaptability.

[0080] Offline task feedback mechanism: When the external network is disconnected, the large-screen audio-visual terminal stores the user's AI interaction request in the local offline cache queue and sends it to the smartphone through the local area network encrypted channel; after the smartphone completes the AI ​​inference locally, it sends back the structured result, the large-screen audio-visual terminal executes the corresponding operation and completes the screen display, and at the same time retains the task record with a unique request ID locally.

[0081] Server-side recovery synchronization mechanism: After the external network link is restored, the mobile phone and the large-screen audio-visual terminal automatically trigger the segmented breakpoint resume synchronization mechanism. Data deduplication is completed based on the unique request ID to avoid duplicate data entering the database. After synchronization is completed, the local cache data is cleared, and the system automatically switches back to the server-side online unified scheduling mode, thus achieving seamless switching between online and offline business.

[0082] The multicast address and port number used in this invention are parameters of the preferred embodiment. They can be replaced by equivalent multicast network segments and equivalent communication ports, and are not limited to the fixed values ​​disclosed in this embodiment.

[0083] The method for task offloading using the above-mentioned broadcasting three-terminal collaborative AI scheduling system based on mobile phone surplus computing power provided by the present invention is as follows:

[0084] Step 1: Three-terminal binding and registration. The smartphone APP completes the cable TV CA authentication, and the smartphone, large-screen audio-visual terminal, and cable TV operator server are bound together by scanning a QR code. The server establishes a user-exclusive computing power resource pool and issues the user's encryption key. Figure 2 This is a sequence diagram of the three-terminal binding and registration process of the present invention, illustrating the complete interactive sequence logic between the broadcast server, smartphone, and large-screen broadcast audiovisual terminal, including CA authentication, device binding, user key distribution, and creation of the user-end computing power resource pool. The broadcast collaborative APP can only obtain exclusive permissions for system computing power scheduling and local area network encrypted communication after completing local CA device authentication; ordinary third-party applications do not have permission to modify the scheduling weight of the broadcast AI inference thread or preempt the broadcast's exclusive computing power and transmission channels.

[0085] Step 2: Computing power status reporting. Smartphones periodically collect and report local computing power usage, remaining battery power, and network status information, while the server updates the dynamic data of the user's computing power resource pool in real time.

[0086] Step 3: Layered scheduling of AI tasks. The large-screen audio-visual terminal collects user interaction requests and uploads them to the server. The server, based on the real-time status of the computing power pool, dynamically allocates AI tasks to smartphone terminals with sufficient surplus computing power.

[0087] Step 4: Local AI Inference Calculation. The smartphone calls upon the local INT4 quantized offline model to complete multimodal inference of speech, image, and semantics, generating lightweight structured results.

[0088] Step 5: Result Feedback and Display. The structured results are uploaded to the mobile phone in encrypted form. The server accurately forwards them to the corresponding large-screen audiovisual terminal. The large-screen audiovisual terminal completes the image rendering, pop-up display, and business linkage operations. The video service runs independently and stably throughout the entire process. Figure 3 This is a complete business process diagram of the AI ​​interactive task unloading of the present invention, which shows the full process execution steps of computing power reporting, task scheduling, local AI inference, lightweight result feedback, and large screen display in online mode.

[0089] Step Six: Offline Adaptive Switching and Data Synchronization. When the network is normal, online three-terminal collaborative scheduling is performed; when a server heartbeat timeout or external network link disconnection is detected, the system automatically switches to LAN offline closed-loop mode; after network recovery, offline data interruption resume transmission and deduplication synchronization are automatically completed, unifying the operating status of the three devices.

[0090] The present invention will now be further described in detail with reference to specific embodiments. Each embodiment progresses from simple to complex, covering all scenarios including basic single-terminal interaction, multi-terminal collaboration, offline / network disconnection, and multimodal integrated value-added services. The interaction latency of each embodiment is consistently within the range of 150–200 ms, with an optimal latency of 180 ms. Compared to traditional edge computing solutions, the interaction latency is significantly reduced, and there is no need for continuous transmission of raw data over the external network, resulting in lower resource consumption.

[0091] Example 1: Single mobile phone bound to a single set-top box, basic AI voice intelligent agent scenario

[0092] This embodiment applies to a basic cable TV AI voice interaction scenario in a typical single-household household. The system consists of a cable TV server, a smartphone, and an existing DVB+IPTV dual-mode set-top box. The smartphone is pre-installed with an INT4 quantized lightweight voice intelligence model, and its daily idle computing power is stably between 65% and 70%.

[0093] Users initiate commands such as on-demand playback, program search, and channel switching via the set-top box's voice remote control. The set-top box only collects raw voice data and uploads request signaling; it does not perform any AI computing tasks. The server schedules the AI ​​inference task to the bound smartphone. The smartphone locally performs speech recognition, semantic understanding, and program retrieval inference, generating structured data containing program IDs, playback control commands, and pop-up display coordinates, which is then transmitted back. Upon receiving the commands, the set-top box displays the interface pop-up, program on-demand playback, and video feed. The entire video live streaming link is independently isolated, without lag or interference.

[0094] This embodiment not only adapts to existing set-top boxes, but also ensures that the entire interaction process and computing power scheduling logic remain completely unchanged after being replaced by a split-screen TV or an integrated smart TV. Only the hardware authentication and network access method for the large screen is adapted differently, thus achieving zero hardware modification and low-cost AI voice empowerment for the large screen.

[0095] Example 2: Multi-terminal computing power collaboration scenario in a household with two mobile phones and two set-top boxes.

[0096] This embodiment constructs a multi-device home computing power pool architecture, with one flagship mobile phone and one mid-range smartphone bound to the same cable TV account, corresponding to two existing set-top boxes in the living room and bedroom. The server collects computing power usage, remaining battery power, and network latency data of the two mobile phones in real time, and dynamically allocates AI tasks through a weighted load balancing algorithm.

[0097] The living room set-top box simultaneously initiates a high-load multimodal task involving video image recognition, with the server prioritizing flagship phones with ample computing power to handle it. Simultaneously, the bedroom set-top box initiates lightweight tasks such as weather inquiries and time announcements, which are then processed in parallel by mid-range phones. When the flagship phone is running a high-load application in the foreground and its surplus computing power drops to 35% or below, the system automatically migrates unfinished high-load tasks to mid-range phones for continued execution, ensuring seamless task interruption and smooth interaction. This embodiment effectively improves the blocking problem of concurrent AI requests across multiple screens in the home, increases the utilization rate of surplus computing power, and achieves dynamic collaborative scheduling of computing power throughout the house.

[0098] Example 3: Running AI agent offline locally on mobile phone, without external network access

[0099] This embodiment is applied to extreme scenarios such as broadcast network failures, home internet outages, and server link interruptions. The device pre-completes three-terminal binding, local key caching, and offline model pre-installation. After network outage, the set-top box and mobile phone automatically initiate the device discovery mechanism, quickly address via UDP multicast, and establish an encrypted local area network communication channel based on TCP handshake.

[0100] When a user initiates commands such as voice playback, channel switching, or program query, the set-top box stores the interaction request in a local offline cache queue and sends it to the mobile phone via an encrypted local area network channel. The mobile phone uses its local INT4 quantization model to complete the entire AI inference process offline, generating standardized control commands that are sent back to the set-top box for execution. The set-top box retains a task record with a unique request ID locally. Once the external network link is restored, the mobile phone and set-top box automatically initiate a breakpoint resume synchronization mechanism, the server completes data deduplication and database entry, and the system automatically switches back to online scheduling mode. This embodiment ensures uninterrupted availability of broadcast AI services in a completely offline scenario, improving system robustness and user privacy security.

[0101] Example 4: Multimodal Fusion AI Scenarios (Video Content Understanding + Pose Recognition + Smart Home Integration)

[0102] This embodiment expands the scope of value-added broadcasting services by leveraging the high-performance NPU computing power of flagship mobile phones and employing an INT4+INT8 hybrid quantization multimodal model. It simultaneously handles three types of AI tasks: video content understanding, user posture recognition, and smart home control for broadcasting. The multimodal model parameters of this invention are not limited to fixed specifications and can be flexibly adapted to the terminal's computing power.

[0103] The set-top box uses an external camera to capture user gestures and voice commands, uploading only lightweight feature data and not transmitting raw video or audio data. The mobile phone performs image gesture reasoning, video content tag recognition, and smart home voice command parsing in parallel, integrating and generating structured data such as video overlay commands, home device control codes, and content tags. The set-top box simultaneously renders the image recognition frame, displays video tags, and controls smart home devices. This embodiment expands the traditional single audiovisual service of broadcasting to a multimodal AI interaction and whole-house smart home integration scenario, enhancing the added value of broadcasting services and user experience.

[0104] Example 5: Multi-type terminal hybrid computing power collaborative scenario

[0105] Figure 4 This is a schematic diagram of the dynamic scheduling of the home multi-terminal computing power pool of the present invention, which shows the load balancing execution mechanism of multi-terminal computing power status collection, weight formula calculation, intelligent task allocation, and dynamic migration of high-load tasks.

[0106] Meanwhile, multiple large-screen terminals, including split-screen TVs, integrated smart TVs, and traditional set-top boxes, can be deployed in a mixed manner within the home. Each screen can independently initiate AI interaction requests, and the server can uniformly allocate computing power tasks. The two-level adaptive upgrade and downgrade control strategy applies uniformly to all large-screen terminals, and there are no scene compatibility limitations.

[0107] This invention provides a collaborative AI scheduling system and offloading method for broadcasting terminals based on surplus computing power of mobile phones. The system architecture includes a broadcasting operator server, smart terminals, and large-screen audiovisual terminals. The broadcasting operator server is responsible for account authentication, device binding, user-end computing power resource pool construction, and dynamic task scheduling. The smart terminal includes at least one smartphone as an AI computing power carrying node. Relying on the INT4 group quantization lightweight large model, it reuses the terminal's surplus computing power to complete all AI agent inference and multimodal task calculations, only sending back structured instructions to avoid the transmission of raw sensitive data. Various broadcasting large-screen audiovisual terminals achieve zero hardware modification by relying on lightweight message parsing plugins or built-in broadcasting modules, exclusively occupying video decoding and playback services, and achieving task layering and isolation. At the same time, through a dedicated mobile phone and large-screen audiovisual terminal LAN direct connection communication protocol, it realizes closed-loop interaction during network outages and data deduplication and synchronization after network recovery. Combined with a multi-terminal dynamic load balancing algorithm, it can be expanded to adapt to portable smart terminals such as tablets to supplement computing power. The system and method provided by this invention do not require the deployment of edge computing devices or the iteration of existing large-screen audio-visual terminal hardware. They can effectively reduce the cost of implementing AI services in the broadcasting industry, solve industry pain points such as video bandwidth congestion, offline function failure, computing power waste, and privacy leakage, and have good compatibility, stability, and scalability.

[0108] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention is also intended to include these modifications and variations.

Claims

1. A collaborative AI scheduling system for broadcasting terminals based on the surplus computing power of mobile phones, characterized in that, The system architecture includes: a broadcasting operator server, a smart terminal, and a large-screen audio-visual terminal; The broadcasting operator's server serves as the global scheduling hub, and is equipped with an account binding module, a computing power scheduling module, a message gateway module, and a CA authentication module. These modules are used to complete the binding of accounts on the three devices, build a user-end computing power resource pool, dynamically allocate AI tasks, forward lightweight messages, and perform security authentication for broadcasting services. The intelligent terminal includes at least one smartphone as an AI computing power carrier terminal. It is equipped with a broadcasting collaboration APP and has a built-in local INT4 quantized AI inference engine and computing power monitoring module. It is used to report its own CPU and NPU surplus computing power ratio, remaining power and network status in real time, and undertake all broadcasting AI intelligent agent computing tasks and multimodal recognition inference tasks. The large-screen audio-visual terminal is used to complete broadband media video decoding, encryption and decryption, and large-screen image rendering services; the large-screen audio-visual terminal is equipped with a lightweight message parsing plugin, which is used to receive and execute structured control commands pushed by the smart terminal.

2. The broadcasting three-terminal collaborative AI scheduling system based on surplus computing power of mobile phones according to claim 1, characterized in that, The large-screen audio-visual terminal is any one or more of the following: an existing DVB+IPTV dual-mode cable set-top box + display screen, a split large-screen TV with external cable authentication peripherals, or an integrated smart TV with a cable CA authentication communication module; the lightweight message parsing plugin of the existing DVB+IPTV dual-mode set-top box is a non-intrusive software plugin; the split large-screen TV completes the device network access verification through an external cable compliance authentication peripheral; the integrated smart TV directly accesses the user-end computing resource pool by relying on the built-in CA chip and local area network communication module.

3. The broadcasting three-terminal collaborative AI scheduling system based on surplus computing power of mobile phones according to claim 2, characterized in that, The smart terminal also includes various portable smart devices with NPU computing power and Wi-Fi local area network communication capabilities as extended terminals. These extended terminals and the smartphone are jointly incorporated into the user-end computing power resource pool, serving as supplementary computing power nodes when the smartphone's computing power is insufficient, thereby realizing a multi-type terminal hybrid computing power collaborative architecture.

4. The broadcasting three-terminal collaborative AI scheduling system based on surplus computing power of mobile phones according to claim 3, characterized in that, The broadcasting operator's server has an independent video service channel and an AI intelligent agent computing task channel. The video service uses the broadcasting dedicated network link throughout, and the AI ​​intelligent agent computing task is exclusively processed by the intelligent terminal, realizing the layered isolation between streaming video tasks and AI computing tasks. The intelligent terminal completes all AI inference calculations locally and only sends byte-level structured result messages back to the broadcast operator's server or the broadcast large-screen audio-visual terminal, without uploading original voice, image and video sensitive data; The system supports an offline closed-loop working mode when the external network is disconnected. It completes device discovery, encrypted communication, and offline task return through a dedicated local area network direct connection communication protocol between the smartphone and the large-screen audio-visual terminal. After the external network is restored, it automatically completes offline data deduplication and synchronization, realizing seamless switching between online and offline services. The AI ​​inference engine on the smart terminal uses a grouped INT4 quantization model, which maps the weights of the original FP32 floating-point model to four-bit integers for storage, and completes the quantization restoration through grouped independent scaling coefficients and zero-point offset correction.

5. The broadcasting three-terminal collaborative AI scheduling system based on surplus computing power of mobile phones according to claim 3, characterized in that, The computing power scheduling module employs a dynamic load balancing algorithm, collecting four core parameters in real time: CPU idle rate, NPU utilization rate, remaining battery power, and network latency of the smart terminal. It then uses a weighted calculation model to allocate intelligent tasks. The preset task allocation weight formula is as follows: Task allocation weight = (Percentage of surplus computing power × 0.5–0.6) + (Remaining battery power × 0.2–0.3) + (Network quality score × 0.15–0.25) The network quality score is obtained by comprehensively quantifying network latency and packet loss rate, with a score range of 0 to 1; the surplus computing power ratio is calculated based on the real-time computing power occupancy of smart terminals; when the surplus computing power ratio of smart terminals drops to 30% to 40%, the system automatically triggers task migration and switches other bound smart terminals of the user to undertake the task.

6. The broadcasting three-terminal collaborative AI scheduling system based on surplus computing power of mobile phones according to claim 4, characterized in that, The dedicated LAN direct connection communication protocol is used to realize three-terminal closed-loop operation in the scenario of external network disconnection and offline operation. The protocol includes device discovery mechanism, handshake negotiation mechanism, encrypted channel mechanism, offline task return mechanism and server-side recovery and synchronization mechanism.

7. The broadcasting three-terminal collaborative AI scheduling system based on surplus computing power of mobile phones according to claim 5, characterized in that, The device discovery mechanism adopts UDP multicast, and the large-screen audio-visual terminal periodically broadcasts device information and account hash values. The smart terminal filters out unrelated user devices by matching account hashes. The handshake negotiation mechanism establishes a reliable long connection based on TCP to complete device identity verification, protocol version, and offline service capability negotiation.

8. The broadcasting three-terminal collaborative AI scheduling system based on surplus computing power of mobile phones according to claim 6, characterized in that, The encrypted channel mechanism adopts a lightweight encryption scheme. During the online binding phase, a unique user pre-shared key is issued by the cable TV operator's server. In offline scenarios, the smart terminal and the large-screen audio-visual terminal generate a temporary session key through the pre-shared key and a random salt value.

9. The broadcasting three-terminal collaborative AI scheduling system based on surplus computing power of mobile phones according to claim 7, characterized in that, The offline task feedback mechanism is as follows: when the external network is disconnected, the large-screen audio-visual terminal stores the user's AI interaction request in the local offline cache queue and sends it to the smart terminal through the local area network encrypted channel; the smart terminal calls the local INT4 quantized AI model to complete inference, generates structured instructions and sends them back to the large-screen audio-visual terminal for execution, and the large-screen audio-visual terminal retains the task record with a unique request ID. The server-side recovery synchronization mechanism is as follows: After the external network is restored, the smart terminal and the large-screen audio-visual terminal start the segmented breakpoint resume synchronization, complete global data deduplication based on the unique request ID, clear the local cache after synchronization is completed, and the system automatically switches back to the server-side online unified scheduling mode; during the offline LAN operation phase, when a user's smartphone is added or offline, the local computing power classification file is synchronized in real time through UDP multicast, and the local computing power resource pool is dynamically updated without the need for the cable TV operator's server to issue synchronization instructions.

10. A method for offloading AI tasks collaboratively across three broadcasting terminals based on surplus computing power of mobile phones, characterized in that, Based on the broadcasting three-terminal collaborative AI scheduling system according to any one of claims 1-9, the method includes the following steps: S1. Three-terminal binding registration: The smartphone completes CA authentication through the Broadcasting Collaboration APP, and scans the code to complete the three-party binding of the smartphone, large-screen audio-visual terminal and broadcasting operator server. The server establishes a user-end computing power resource pool and issues the user's encryption key. S2. Computing power status reporting: Smartphones periodically collect and report local CPU and NPU computing power usage, remaining power and network latency data, and the server updates the computing power resource pool status of the user terminal in real time. S3, AI Task Layered Scheduling: The large-screen audio-visual terminal collects user voice and image interaction requests and only uploads request signaling. Based on the computing power pool status and load balancing algorithm, the server offloads AI inference tasks to smartphones with sufficient spare computing power. If the spare computing power of a single smartphone drops to the threshold range of 30% to 40%, the high-load multimodal inference tasks it carries are migrated to other smartphones with sufficient spare computing power in the computing power resource pool for continued execution. S4, Local Lightweight Inference: The smartphone completes speech recognition, semantic understanding, and multimodal content recognition inference through the local INT4 quantized AI model, generating byte-level structured result messages; S5. Lightweight result feedback and display: Smartphones encrypt and send back structured results, which are then forwarded by the server to the corresponding large-screen audio-visual terminal. The large-screen audio-visual terminal performs pop-up windows, subtitle overlay, video linkage, and device control operations. The video service runs independently and stably throughout the entire process. S6. Offline Adaptive Switching and Synchronization: When the network is normal, online three-terminal collaborative scheduling is performed; when the server heartbeat timeout or external network link is detected, the local area network offline closed-loop mode is automatically switched; after the network is restored, offline data synchronization is completed through breakpoint resume and data deduplication mechanism to unify the operating status of the three-terminal devices.

Images