Intelligent global cloud live broadcast system based on AI and multi-operator and multi-cloud optimization

Abstract

The invention relates to the technical field of network communication and video processing, in particular to an intelligent global cloud live broadcast system based on AI and multi-operator and multi-cloud optimization, and the system comprises a data collection module which is used for collecting live broadcast video streams; the video compression module adopts H.265 coding; the content review module is used for performing multi-modal review; the RPMTS packaging module is used for accessing three operator cloud private line POP nodes through a public network after packaging; the decoding module is used for decoding POP nodes in the middle area; the translation module is used for generating multi-language subtitles or dubbing; the multi-operator and multi-cloud adaptability matrix module calculates an optimal path in combination with an AI algorithm and a multi-cloud cost adaptability matrix; the multi-branch access module is used for realizing branch node cooperative forwarding; and the multi-language distribution module is used for generating multi-language subtitles and dubbles and pushing the multi-language subtitles and dubbles to each global live broadcast platform, so that synchronous distribution is realized, and the system can reduce global live broadcast delay, improve multi-language coverage capability and realize high reliability and cost optimization.

Description

Technical Field

[0001] The present invention relates to the field of network communication and video processing technology, and in particular to an intelligent global cloud live broadcast system based on AI and multi-operator and multi-cloud optimization. Background Art

[0002] As globalization accelerates, demand for high-quality, low-latency live video streaming continues to grow globally across applications such as e-commerce shopping, online education, media communications, and enterprise-level video conferencing. However, existing cloud live streaming technologies still face the following key technical bottlenecks in cross-border and cross-regional distribution and management:

[0003] 1. Insufficient global network adaptability

[0004] Current cloud live streaming systems typically rely on a single cloud service provider or static CDN nodes, lacking the ability to dynamically optimize paths based on users' real-time geographic location, operator network status, and international backbone network congestion. This leads to the following issues:

[0005] When Southeast Asian users access Middle Eastern nodes, high latency and packet loss rates occur due to fixed routing.

[0006] When South American users access North American nodes, the user experience fluctuates because they fail to differentiate between local carrier network quality.

[0007] Data transmission from Africa to Asia lacks intelligent load balancing of satellite links and submarine cable resources, resulting in low link utilization.

[0008] 2. Limited multi-operator and multi-cloud resource scheduling capabilities

[0009] The current multi-cloud architecture is primarily used for disaster recovery and backup, but lacks the ability to coordinate and dispatch costs, performance, and operator dedicated line resources in real time. This is manifested in the following ways:

[0010] Unable to dynamically select the optimal node based on the network quality and bandwidth prices of different cloud service providers, as well as the cost structure of operator dedicated lines such as China Telecom CN2, China Unicom AS4837, and China Mobile CMI;

[0011] For example, the stability and latency of AWS Singapore nodes differ from those of Alibaba Cloud Singapore nodes, and the performance of dedicated lines between different clouds and operators varies greatly. However, the existing system lacks integrated scheduling.

[0012] When user traffic surges during major e-commerce promotions in the Middle East, it's impossible to offload back-to-source traffic to domestic operators' Point-of-Place (POP) nodes (such as China Telecom's Dubai POP) in advance, leading to network congestion and live streaming freezes.

[0013] 3. Poor adaptability of encoding and content distribution

[0014] Traditional live streaming systems typically use fixed encoding configurations, making it difficult to dynamically adjust to user needs and network conditions in different regions around the world. This results in:

[0015] Middle Eastern users prefer subtitle regions, while Southeast Asian users focus on product details, but the existing system cannot provide differentiated encoding;

[0016] When the network bandwidth of African users dropped sharply, the bitrate adjustment delay exceeded 3 seconds, resulting in a serious decline in image quality and live broadcast freezes.

[0017] 4. Insufficient global content compliance review

[0018] Existing content review systems primarily rely on keyword filtering and lack multimodal AI analysis of video, audio, and text. This makes them unable to meet regulatory requirements in various countries, as shown by the following:

[0019] Lack of real-time performance, unable to detect illegal content within seconds during live broadcasts;

[0020] There is a lack of targeted testing for sensitive symbols and food labels in the Middle East;

[0021] The implementation of the EU GDPR's "right to be forgotten" has been delayed, posing compliance risks;

[0022] China's new online audiovisual regulations and compliance requirements in Southeast Asia and Indonesia have not formed a differentiated strategy.

[0023] 5. Lack of coordinated optimization between AI and network systems

[0024] The existing AI algorithm module and network routing control system are independent of each other, resulting in:

[0025] Live content recognition results (such as breaking news or celebrity appearances) cannot trigger real-time adjustments to network bandwidth and routing policies;

[0026] Different business scenarios (such as educational screen sharing and e-commerce product close-ups) use the same transmission strategy, and resource scheduling cannot be dynamically linked to the user experience.

[0027] In summary

[0028] Existing global cloud live streaming systems generally suffer from insufficient cross-regional network adaptability, lack of intelligent multi-operator and multi-cloud resource scheduling, poor adaptability in encoding and content distribution, insufficient depth and real-time content compliance review, and lack of coordinated optimization of AI algorithms and network systems. These problems make it difficult to meet the global user demand for low-latency, high-definition, compliant, controllable, and cost-effective video live streaming services.

[0029] Therefore, there is an urgent need to propose an intelligent global cloud live broadcast system based on AI and multi-operator and multi-cloud optimization. By deeply integrating AI algorithms with multi-cloud scheduling, operator dedicated lines, dynamic encoding optimization, and multimodal content review, a comprehensive upgrade of global video live broadcast services in terms of stability, low latency, multi-language adaptation, content compliance, and cost control can be achieved, helping e-commerce, education, media, and enterprise-level applications to achieve global deployment and enhance competitiveness. Summary of the Invention

[0030] To achieve the above objectives, the present invention proposes an intelligent global cloud live broadcast system based on AI and multi-operator and multi-cloud optimization, including:

[0031] The data acquisition module is used to obtain the original video stream data generated by the local live broadcast device;

[0032] A video compression module is used to compress the original video stream based on a preset H.265 encoding algorithm to generate a compressed video stream;

[0033] The content review module is used to perform video frame image recognition, audio ASR transcription, and NLP semantic analysis on the compressed video stream through the content security API to implement multimodal content review of the video stream and obtain the video stream that has passed the review;

[0034] RPMTS encapsulation module: used to perform RPMTS real-time encapsulation on the video stream that has passed the review, generating an encapsulated data stream suitable for public network transmission;

[0035] The public network access module is used to connect the RPMTS encapsulated data stream to the nearest point-of-presence (POP) node of the cloud-to-cloud private line of operators such as China Telecom, China Unicom, and China Mobile, and realize the bridge connection with the operator's private line.

[0036] The decoding module is used to perform H.265 decoding on the received compressed video stream at the intermediate POP node and restore it to the original video frame data;

[0037] The translation module is used to translate the decoded video and audio content in real time based on AI speech recognition and neural network machine translation algorithms, generating subtitles and dubbing streams in the target language;

[0038] Multi-operator and multi-cloud adaptability matrix module: This module intelligently calculates and selects the optimal live broadcast distribution path based on the user's egress operator attributes and the cost and performance of multi-cloud nodes.

[0039] The multi-branch access module supports multiple branch nodes to access the live broadcast system simultaneously, forming distributed collaboration and multi-point cloud forwarding capabilities;

[0040] The multilingual distribution module generates multilingual subtitles and dubbing through AI real-time translation, and supports API access to live broadcast platforms to achieve global multilingual synchronous distribution.

[0041] In one example, the content review module includes:

[0042] The video frame detection unit is used to capture each frame of the compressed video stream, call the OCR algorithm to recognize text content, and detect illegal content such as political, pornographic, and brand infringement;

[0043] The audio recognition unit is used to perform ASR speech transcription on the audio part of the video stream and convert the speech into text content;

[0044] The NLP semantic analysis unit is used to perform natural language processing and sensitive word analysis on the transcribed text to determine whether there are any illegal semantics;

[0045] The review result output unit is used to output the review results and push the video stream that has passed the review to the RPMTS packaging module.

[0046] In one example, the RPMTS packaging module includes:

[0047] The fragment encapsulation unit is used to perform real-time fragment encapsulation of the video stream that has passed the review with a granularity of 100ms;

[0048] The encapsulation protocol unit adopts UDP over RTP protocol and supports packet loss retransmission mechanism;

[0049] The encapsulation control unit is used to control the number of encapsulation retries. The default retry is 3 times to ensure the stability of cross-region public network transmission.

[0050] In one example, the translation module includes:

[0051] The speech recognition subunit uses the Conformer structural acoustic model and the Transformer language model to recognize video and audio content in real time;

[0052] The neural network translation subunit, based on the M2M-100 and Transformer neural network machine translation models, translates the recognized text into the target language text;

[0053] The subtitle generation subunit converts the translated text into an SRT standard subtitle file and synchronizes the time code with the video stream;

[0054] The dubbing generation sub-unit calls the AI ​​speech synthesis interface to generate the target language dubbing audio stream from the translated text and synchronously encapsulate it with the video stream.

[0055] In one example, the multi-operator and multi-cloud adaptability matrix module includes:

[0056] The operator attribute identification unit is used to identify the operator ASN and geographical location corresponding to the branch egress IP address and generate user operator attribute information;

[0057] Cost and performance data collection unit, used to obtain bandwidth prices, rental prices, traffic billing models, and real-time performance indicators of each cloud service provider's POP nodes through API interfaces, including bandwidth utilization, average latency, packet loss rate, and availability;

[0058] The adaptation coefficient calculation unit is used to calculate the adaptation coefficient based on whether the user's exit operator is consistent with the operator of each target POP node, and give priority to nodes directly connected to the same operator;

[0059] A matrix construction unit, used to combine operator adaptation coefficients, cost data, and performance indicators to build a multi-operator and multi-cloud adaptability matrix;

[0060] The path scoring calculation unit is used to calculate the comprehensive score of each POP node based on the constructed multi-operator and multi-cloud adaptability matrix and preset weight parameters;

[0061] The optimal node output unit is used to select the POP node with the highest score as the optimal path based on the comprehensive scoring results, and output other nodes with scores higher than the preset threshold as redundant paths.

[0062] In one example, the matrix construction unit includes:

[0063] The weight configuration subunit is used to configure the weight parameters of cost, delay, packet loss rate and adaptation coefficient according to the business scenario;

[0064] The comprehensive score calculation subunit normalizes the cost, performance, and adaptability of each POP node based on the configured weight parameters and calculates a comprehensive score;

[0065] The node sorting subunit sorts all POP nodes according to the comprehensive scores and generates the optimal POP node list.

[0066] The path score calculation subunit is used to calculate the path score based on the matrix results combined with the weight parameters;

[0067] The optimal node output subunit is used to select the optimal node according to the path score and output a redundant node list.

[0068] In one example, the multi-branch access module includes:

[0069] Branch identification unit, used to collect the public network IP, ASN, geographic location, egress bandwidth and computing resource information of branch nodes;

[0070] Access control unit, used for API key authentication, access permission verification and registration management of branch nodes;

[0071] The stream access and forwarding unit is used to receive the live stream and play it locally on the branch. It forwards the stream to other branch nodes according to system scheduling to achieve distributed collaborative distribution.

[0072] In one example, the multilingual distribution module includes:

[0073] The multi-language generation unit is used to synchronously encapsulate the target language subtitles and dubbing streams generated by the AI ​​translation module with the original video stream;

[0074] The API interface access unit is used to push multi-language live streams to various mainstream live broadcast platforms through the API interface, realizing global multi-language synchronous distribution.

[0075] The intelligent global cloud live broadcast system based on AI and multi-operator and multi-cloud optimization proposed by this invention can bring the following benefits:

[0076] Beneficial effects:

[0077] Intelligent routing: Using AI scoring algorithms and multi-carrier and multi-cloud SD-WAN dynamic routing, the optimal path is selected in real time, avoiding delays and packet loss caused by fixed paths.

[0078] Improved content compliance: Multimodal content review (OCR+ASR+NLP) enables violation detection within seconds, meeting regulatory requirements in multiple regions around the world.

[0079] Improved real-time translation: The AI ​​translation module supports multi-language subtitles and dubbing, with an overall delay of less than 1 second, eliminating cross-language viewing barriers;

[0080] Optimal multi-operator and multi-cloud cost and performance: Combining a multi-operator and multi-cloud adaptability matrix, we achieve coordinated optimization of node costs and network performance, improving stability by 50% and reducing bandwidth costs by 10% to 25%.

[0081] Enhanced stability and reliability: The path switching module switches to redundant paths based on real-time network status, ensuring uninterrupted live streaming and improving user experience.

[0082] System intelligence and business adaptability: Supports business scenario weight configuration to achieve global optimization in multiple dimensions including encoding, routing, content compliance, and translation. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0084] Figure 1 This is a diagram of the intelligent global cloud live broadcast system architecture based on AI and multi-operator and multi-cloud optimization;

[0085] Figure 2 Schematic diagram of the multi-operator and multi-cloud adaptability matrix module of this intelligent global cloud live broadcast system based on AI and multi-operator and multi-cloud optimization. DETAILED DESCRIPTION

[0086] In order to more clearly illustrate the overall concept of the present invention, a detailed description is given below in an exemplary manner in conjunction with the accompanying drawings.

[0087] In the description of the present invention, it should be understood that the terms "center", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "axial", "radial", "circumferential", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore cannot be understood as limiting the present invention.

[0088] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature identified as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0089] In the present invention, unless otherwise expressly specified or limited, terms such as "mounted," "connected," "connect," and "fixed" should be understood broadly. For example, they may refer to fixed connection, detachable connection, or integration; mechanical connection, electrical connection, or communication; direct connection or indirect connection through an intermediate medium; and internal communication between two components or interaction between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0090] In the present invention, unless otherwise clearly specified and limited, a first feature "above" or "below" a second feature may be that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. In the description of this specification, the descriptions with reference to the terms "one scheme", "some schemes", "examples", "specific examples", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the scheme or example are included in at least one scheme or example of the present invention. In this specification, the schematic expressions of the above terms do not necessarily refer to the same scheme or example. Moreover, the specific features, structures, materials or characteristics described may be combined in an appropriate manner in any one or more schemes or examples.

[0091] like Figures 1 to 2 As shown,

[0092] Example 1

[0093] The embodiment of the present invention provides an intelligent global cloud live broadcast system based on AI and multi-cloud optimization, including:

[0094] (1) Data acquisition module:

[0095] Configured on the local live broadcast device, such as an encoder, capture card, or edge streaming server, it is used to collect the original video stream data generated by the camera or streaming device.

[0096] Interface specifications: Support HDMI2.0 and SDI input.

[0097] Resolution and frame rate: Supports 3840×2160@60fps acquisition.

[0098] Color sampling: Support YUV42010bit.

[0099] Protocol compatibility: compatible with RTMP, RTSP, and SRT streaming protocols.

[0100] Network jitter buffer: The setting range is 200-500ms, which can be dynamically adjusted to adapt to different network environments and ensure collection stability.

[0101] Privacy protection measures:

[0102] The explicit consent pop-up window is automatically triggered when face collection is performed, in compliance with Article 26 of the Personal Information Protection Law

[0103] Automatically downsample to 720p before outbound data transmission to meet the requirements of the "Measures for Data Outbound Security Assessment"

[0104] (2) Video compression module:

[0105] Perform H.265 (HEVC) encoding compression on the collected original video stream. The specific parameters are as follows:

[0106] Dynamic bitrate control: Adaptively adjusts the bitrate between 1-8Mbps based on network conditions.

[0107] Key frame interval: GOP = 30 frames (optimized for e-commerce live streaming scenarios).

[0108] ROI encoding: allocates an additional 30% bitrate to the subtitle area (bottom 20% pixels) in the picture.

[0109] Technical effect: Under the same subjective image quality, it saves 45.7% of bandwidth compared with the H.264 encoding scheme (test sequence: HEVC Class B).

[0110] (3) Content review module:

[0111] Used to perform multimodal content security review on compressed video streams, including:

[0112] Multi-level review process:

[0113] Video frame OCR detection: Sampling 5 frames per second, the OCR algorithm is used to detect illegal text content such as political, pornographic, and brand infringement.

[0114] Audio ASR transcription: Calls the Conformer structural acoustic model to support real-time speech transcription in Chinese and English.

[0115] NLP semantic analysis: Based on the BERT sensitive word model, the transcribed text is judged for semantic violations.

[0116] Special detection of religious content:

[0117] Built-in 12 categories of religious symbol detection models (98.3% accuracy)

[0118] Automatically block close-up shots of non-halal food

[0119] Audit mechanism:

[0120] Suspected illegal content will be automatically transferred to the manual review queue

[0121] Manual review response time <30 minutes

[0122] Maintain complete audit logs for more than 180 days

[0123] (4)RPMTS package module

[0124] Perform RPMTS real-time encapsulation on the video stream that has passed the review. The configuration is as follows:

[0125] Sharding window size: 100ms.

[0126] Encapsulation protocol: UDP over RTP, the packet header structure includes:

[0127]

[0128] Dynamic retransmission algorithm:

[0129] Number of retransmissions = base value 3 + floor (current packet loss rate / 5%), with a maximum of 8 times.

[0130] (5) Public network access module

[0131] The encapsulated video stream is connected to the cloud dedicated line POP nodes of the three major operators through the public network, including:

[0132] China Telecom;

[0133] China Unicom;

[0134] China Mobile;

[0135] Access strategy: When traffic is detected, the system automatically switches to the operator's corresponding POP node to achieve optimal path access.

[0136] (6) Decoding module

[0137] Deployed on the intermediate POP node to perform H.265 decoding on the received compressed video stream. The configuration is as follows:

[0138] Hardware acceleration solution: Using NVIDIA T4 GPU, supporting 16-channel 1080p@60fps parallel decoding.

[0139] Fault-tolerance mechanism: When packet loss is detected, damaged frames are compensated through time-domain interpolation to ensure continuous and stable playback.

[0140] (7) Translation module

[0141] Based on AI speech recognition (ASR), neural machine translation (NMT), and text-to-speech (TTS) algorithms, multilingual subtitles and dubbing streams are generated in real time. The quantitative indicators are as follows:

[0142]

[0143] (8) Multi-operator and multi-cloud adaptability matrix module

[0144] This module includes:

[0145] Operator attribute identification unit: collects user egress IP, queries ASN and geographic location, and verifies through ping fingerprint.

[0146] The calculation formula of adaptation coefficient a is:

[0147] If it is a direct connection with the same operator, α = 1.0;

[0148] If it is a cross-operator call, then:

[0149] α = 0.8 - 0.1 × number of cross-border hops.

[0150] Among them, the number of cross-border hops refers to the number of cross-border segments in the path from the user's export IP to the target POP node, and the minimum adaptation coefficient is not less than 0.5.

[0151] Dynamic weight configuration example (Python pseudocode):

[0152] If business type == "e-commerce live streaming":

[0153] Weight = {"delay":0.6,"cost":0.2,"packet loss rate":0.2}

[0154] elif Service Type == "Video Conferencing":

[0155] Weight = {"delay":0.3,"cost":0.3,"packet loss rate":0.4}

[0156] Matrix output: Build a matrix using multi-cloud cost and performance data, calculate a comprehensive score, and output the optimal path node.

[0157] (9) Multi-branch access module

[0158] Supports multiple branch node access, the configuration is as follows:

[0159] Node health monitoring: Detects the network status from branch nodes to end users every 30 seconds, including:

[0160] Latency (ICMPPing);

[0161] Available bandwidth (iperf3 test);

[0162] Packet loss rate (UDP Flood detection).

[0163] Load balancing algorithm:

[0164] Traffic distribution ratio = node score / sum of all node scores

[0165] in:

[0166] Traffic distribution ratio: the actual traffic ratio allocated to each branch node;

[0167] Node score: The score is calculated based on node latency, bandwidth, and packet loss rate;

[0168] Sum of all node scores: The sum of scores of all available nodes in the system.

[0169] (10) Multi-language distribution module

[0170] Platform adaptation solution:

[0171] Live streaming platform APILive: uses DASH / HLS packaging;

[0172] QoS guarantee mechanism:

[0173] When congestion is detected on the target platform, the system automatically reduces the bit rate of non-key frames by up to 40%.

[0174] (11) Test data

[0175] Table 1 Comparison of cross-border live streaming performance

[0176]

[0177] Table 2 Content review effectiveness

[0178]

[0179] Example 2:

[0180] The embodiment of the present invention provides an intelligent global cloud live broadcast system based on AI and multi-cloud optimization, a multi-operator and multi-cloud adaptability matrix module, such as Figure 2 Shown, including:

[0181] (1) Operator attribute identification unit

[0182] Function: Identify the carrier ASN and geographic location corresponding to the user or branch node egress IP address, providing input for subsequent multi-cloud POP node selection.

[0183] Specific steps:

[0184] Collect the public network exit IP address, for example: 221.176.0.1.

[0185] Call the IP2ASN database (such as MaxMindASNDB) to query the ASN number, for example:

[0186] Query result: ASN9808 (China Mobile Communications Corporation, China Mobile).

[0187] Use the GeoIP database to resolve the country, province, and city information of the export IP.

[0188] Perform latency fingerprint verification and ping the target cloud POP node to confirm the consistency of the operator and physical network path.

[0189] (2) Cloud vendor preferred matching unit

[0190] Function: Based on the operator's attributes, the POP node of the same operator's cloud is selected to achieve the shortest physical path and direct access to the operator.

[0191] Specific steps:

[0192] Query the operator-cloud vendor POP node mapping table. The example is as follows:

[0193] Operator cloud vendor POP node priority list

[0194] China Mobile CMI Mobile Cloud POP Node > Alibaba Cloud CMI Private Line POP Node

[0195] China Unicom Unicom Cloud AS4837 POP Node > Alibaba Cloud Unicom Dedicated Line POP Node

[0196] China Telecom Tianyi Cloud CN2 POP Node > Alibaba Cloud Telecom Private Line POP Node

[0197] Output the preferred POP node based on the query results. For example:

[0198] User export operator: China Mobile

[0199] Preferred POP node: CMI nearest POP node

[0200] (3) Cost and performance data collection unit

[0201] Function: Collect the cost and real-time network performance indicators of each cloud service provider's POP nodes to form a multi-dimensional input matrix.

[0202] Specific steps:

[0203] Cost data collection:

[0204] Obtain the bandwidth price, rental price, and traffic billing mode of each cloud POP node through the API interface.

[0205] Example:

[0206] Alibaba Cloud Singapore Node: $0.072 / GB

[0207] AWS Singapore Node: $0.085 / GB

[0208] Performance indicator collection:

[0209] Obtain real-time information through the SD-WAN control layer:

[0210] Bandwidth availability

[0211] Average latency

[0212] Packet loss rate

[0213] Node availability

[0214] Example:

[0215] Alibaba Cloud Singapore: 42ms latency, 0.1% packet loss

[0216] AWS Singapore: 45ms latency, 0.2% packet loss

[0217] (4) Adaptation coefficient calculation unit

[0218] Function: Calculates the adaptation coefficient between the user's egress carrier and the carrier of each target POP node.

[0219] Calculation formula:

[0220] α = {1.0, same operator, 0.8 - 0.1 × number of cross-border hops, cross-operator}

[0221] Formula Description:

[0222] When the user and the POP node are operated by the same operator, the adaptation coefficient α = 1.0;

[0223] When it is across operators, α=0.8-0.1×number of international hops (the minimum number of international hops is 1 and the maximum number does not exceed 3).

[0224] (5) Matrix construction unit

[0225] Function: Build a multi-cloud adaptability matrix that combines cost, performance, and adaptation coefficient.

[0226] Example of matrix structure:

[0227]

[0228] (6) Path scoring calculation unit

[0229] Function:

[0230] Dynamically configure the weights of each indicator based on the business scenario, and calculate a comprehensive score for each POP node in the multi-operator and multi-cloud adaptability matrix.

[0231] Specific steps:

[0232] Dynamic weight configuration example (Python pseudocode):

[0233] If business type == "e-commerce live streaming":

[0234] Weight = {"delay":0.6,"cost":0.2,"packet loss rate":0.2}

[0235] elif Service Type == "Video Conferencing":

[0236] Weight = {"delay":0.3,"cost":0.3,"packet loss rate":0.4}

[0237] Comprehensive scoring formula:

[0238] Score = (normalized cost * weight _ cost) + (normalized delay * weight _ delay) + (normalized packet loss rate * weight _ packet loss rate) + (adaptation coefficient * weight _ adaptation coefficient)

[0239] Technical effects:

[0240] Implement quantitative scoring of each POP node based on business scenario requirements, providing a calculation basis for optimal node selection.

[0241] (7) Optimal node output unit

[0242] Function:

[0243] According to the scoring results of the path scoring calculation subunit, the POP node with the highest score is selected as the optimal path, and a redundant path list is output.

[0244] Specific steps:

[0245] Sort all POP nodes by comprehensive score;

[0246] Output the POP node with the highest score as the optimal path;

[0247] POP nodes with output scores higher than the preset threshold are used as redundant paths to improve system stability and cost-effectiveness.

[0248] Technical effects:

[0249] Through the optimal node output unit, the system can flexibly select the optimal path and configure backup paths based on real-time calculation results, thereby improving the stability and reliability of the global live broadcast system.

[0250] Technical Effect: Optimal access with the same operator reduces latency by 20% to 30% and packet loss by 50%;

[0251] Combining multi-cloud costs and performance, bandwidth utilization is reduced by 10% to 25%; dynamic weight configuration meets different business scenarios and improves system adaptability and stability.

[0252] Example 3

[0253] This embodiment provides an implementation method for a multi-branch access module in an intelligent global cloud live broadcast system based on AI, multi-operator, and multi-cloud optimization, including:

[0254] (1) Branch identification unit

[0255] Function: Collect and identify the network and computing resource attributes of branch nodes to form the system's global access and scheduling input.

[0256] Specific steps:

[0257] Public IP collection

[0258] Automatically collect the public network exit IP address of the branch node through the SD-WAN edge device or RPMTS streaming client.

[0259] ASN Lookup

[0260] Call the IP2ASN database to query the ASN to which the egress IP belongs and determine the operator information.

[0261] Example:

[0262] Egress IP: 223.104.18.1

[0263] Query results: ASN9808 (China Mobile)

[0264] Geographic location analysis

[0265] Use the GeoIP database to resolve the country, province, and city corresponding to the export IP.

[0266] Computing resource acquisition

[0267] Collect node CPU utilization, memory utilization, and GPU capabilities (such as whether H.265 hardware decoding is supported) for subsequent stream allocation and transcoding scheduling.

[0268] (2) Access control unit

[0269] Function: Perform secure access control and authentication for branch nodes.

[0270] Specific steps:

[0271] API Key Authentication

[0272] Each branch node is assigned a unique APIKey through the management platform. The Key and ClientID must be carried for authentication when pushing streams or registering.

[0273] Permission Verification

[0274] The backend calls the authentication service interface to verify the APIKey, IP whitelist, and branch registration status.

[0275] Registration Management

[0276] The successfully authenticated branch node information is entered into the node registry, including:

[0277] Node ID

[0278] Public IP

[0279] ASN

[0280] Geographical location

[0281] CPU / GPU capabilities

[0282] Current access status

[0283] (3) Stream access and forwarding unit

[0284] Function: Realize branch node access, decoding, forwarding and distributed collaboration of live streams.

[0285] Specific steps:

[0286] Stream Access

[0287] The branch node receives the main stream from the system (such as the decoded output stream of the POP node), decodes it locally and forwards it.

[0288] Local playback

[0289] If the branch node is a content consumption end, it directly plays the decoded stream and supports 4K@60fps decoding.

[0290] Branch forwarding

[0291] If configured as a relay node, it will re-encapsulate the received stream according to system scheduling and push it to other downstream branch nodes to achieve multi-level distributed collaborative forwarding.

[0292] (4) Node health monitoring

[0293] Function: Regularly monitor the network and resource status of branch nodes to provide a basis for load balancing and scheduling.

[0294] Specific monitoring indicators:

[0295] Latency: Run ICMPPing to the target user area and record the mean and variance of the round-trip latency.

[0296] Available bandwidth: Use iperf3TCP test to collect the available upstream / downstream bandwidth.

[0297] Packet loss rate: Calculate the packet loss rate through the UDP Flood test.

[0298] Detection frequency: once every 30 seconds, the results are written to the node status database.

[0299] (5) Load balancing algorithm

[0300] Function: Dynamically allocate traffic based on node health status and score calculation.

[0301] Calculation method:

[0302] Traffic distribution ratio = node score / sum of all node scores

[0303] Node score: Calculated based on the node latency (the lower the higher the score), available bandwidth (the larger the higher the score), and packet loss rate (the lower the higher the score).

[0304] Sum of all node scores: The sum of the scores of all currently online and healthy branch nodes.

[0305] (6) Technical effects

[0306] The multi-branch access module described in this embodiment achieves:

[0307] Global multi-point collaborative forwarding: Supports collaborative streaming and forwarding among headquarters, branches, and edge nodes, improving system coverage and user access stability.

[0308] Access security and authentication: APIKey+ClientID dual authentication mechanism ensures branch node access security;

[0309] Intelligent load balancing: Dynamically distributes traffic based on real-time network and computing resource status, improving overall system utilization and user experience.

[0310] Example 4

[0311] This embodiment provides an implementation of a multi-language distribution module in an intelligent global cloud live broadcast system based on AI and multi-operator and multi-cloud optimization, including the following units and functions:

[0312] (1) Multilingual Generation Unit

[0313] Function: Encapsulate the target language subtitles and dubbing streams generated by the AI ​​translation module in a time-synchronized manner with the original video stream.

[0314] Specific steps:

[0315] Subtitle generation:

[0316] Input: target language text after NMT translation.

[0317] Processing: Generate a subtitle file that complies with the SRT / VTT standard, including the start timestamp, end timestamp, and subtitle text content.

[0318] Output: subtitle track file that matches the original video stream.

[0319] Dubbing generation:

[0320] Input: target language text after NMT translation.

[0321] Processing: Call the TTS (such as FastSpeech2) speech synthesis service to generate the target language dubbing audio stream. The output format supports PCM / WAV / AAC.

[0322] Output: An audio stream file that is strictly aligned with the original video stream length.

[0323] Synchronous package:

[0324] Embed the subtitle file and dubbing audio stream into the original video stream container (mp4 / ts / flv), keep the PTS / DTS time code consistent, and ensure decoding synchronization between the live broadcast end and the player end.

[0325] (2) API interface access unit

[0326] Function: Push multi-language live streams to major live streaming platforms through API interfaces, achieving global multi-language synchronous distribution.

[0327] Specific steps:

[0328] Platform interface adaptation:

[0329] Live streaming platform API Live: Call the live streaming platform API to create and push streams, and the encapsulation format is DASH (MPEG-DASH) / HLS (HTTP Live Streaming).

[0330] Branch-owned APP / Web player: Push multi-language packaged streams via WebSocket+HLS or WebRTC, and the player parses multi-language audio and subtitle tracks.

[0331] Streaming certification:

[0332] Obtain and cache the AccessToken for each platform, and refresh it regularly to ensure the stability of streaming API calls.

[0333] Real-time status monitoring:

[0334] Call the platform stream status API to monitor the streaming status, number of concurrent viewers, and playback quality (buffer ratio, bitrate, etc.) in real time, and write the results to the monitoring database.

[0335] (3) QoS guarantee mechanism

[0336] Function: Ensure the quality of multi-language distribution and enhance the user viewing experience.

[0337] Specific measures:

[0338] Dynamic bitrate adjustment:

[0339] When network congestion on the target platform is detected (such as when RTMP / HLSACK latency increases beyond a threshold), the bitrate of non-critical frames is automatically reduced by up to 40%.

[0340] Multi-language audio track downgrade switching:

[0341] If the target language dubbing audio stream experiences continuous frame loss, the system automatically switches to subtitle mode to ensure content accessibility.

[0342] User-adaptive multi-language selection:

[0343] With the HLS multi-audio track + multi-subtitle track function, players can switch languages ​​independently without refreshing the page.

[0344] (4) Test data

[0345]

[0346] Technical Effects

[0347] The multilingual distribution module described in this embodiment achieves the following technical effects:

[0348] Multi-language global synchronization: Supports multi-language subtitles and dubbing for the same live stream, covering major languages ​​such as English, Arabic, Hindi, French, and Chinese.

[0349] Improved distribution efficiency: Platform API access and packaging are synchronized and automated to avoid repeated streaming and improve distribution efficiency by more than 30%.

[0350] User experience optimization: Provides adaptive switching of multiple audio tracks and subtitles to meet the personalized viewing needs of users in different countries and regions, and increases user retention rate by 40%.

[0351] Implementation Case 5

[0352] This embodiment provides verification of the overall solution of an intelligent global cloud live broadcast system based on AI, multi-operator and multi-cloud optimization. It combines the modules of Examples 1 to 4 to form an end-to-end system, and verifies the system's technical effectiveness through actual scenario testing.

[0353] (1) Verification environment

[0354] Test scenario: Cross-border e-commerce live streaming, the host is located in Shenzhen, China, and the target users are distributed in:

[0355] North America (San Francisco)

[0356] Middle East (Dubai)

[0357] Southeast Asia (Singapore)

[0358] Access the network:

[0359] China Telecom CN2 export

[0360] AWS Singapore Node

[0361] Alibaba Cloud Singapore Node

[0362] China Mobile CMI Node

[0363] Testing tools:

[0364] SD-WAN console (network status collection)

[0365] FFmpeg (streaming and transcoding)

[0366] OBSStudio (local streaming)

[0367] Self-developed multimodal AI content review API

[0368] ASR-NMT-TTSAI Translation Framework

[0369] Live API SDK for live streaming platforms

[0370] (2) Verification steps

[0371] Data collection and compression verification

[0372] The camera's original stream (3840×2160@60fps) is captured and input into the H.265 compression module. The bit rate is dynamically adjusted between 3-6Mbps, and the PSNR is stabilized above 38dB.

[0373] Content review and verification

[0374] Compared with the H.264 encoding scheme, under the same subjective image quality (PSNR ≥ 38dB) conditions, the average bandwidth savings is 45.7% (test sequence: HEVC Class B)

[0375] The live stream bitrate dropped from an average of 6.5Mbps (H.264) to 3.5Mbps (H.265)

[0376] OCR detects sensitive words (political, brand, religious symbols) with a detection accuracy rate of >98%.

[0377] ASR transcribes Chinese and English with a WER of less than 5.3%.

[0378] NLP semantic analysis correctly identified 23 cases of politically banned words, with a recall rate of 99.2%.

[0379] RPMTS encapsulation and public network access verification

[0380] UDP over RTP encapsulation is enabled, with 100ms fragmentation, and the packet loss rate at the Chinese POP node is controlled within 0.2%.

[0381] China Telecom CN2 path latency is stable at 48ms; CMI Mobile Cloud path latency is stable at 52ms.

[0382] Multi-operator and multi-cloud adaptability matrix verification

[0383] The system automatically identifies the egress as China Telecom CN2, preferring the Alibaba Cloud Singapore node with an adaptation coefficient of 1.0.

[0384] Weight configuration (e-commerce live broadcast scenario): delay 0.6, cost 0.2, packet loss rate 0.2.

[0385] The node with the highest comprehensive output score is Alibaba Cloud Singapore, with a latency of 42ms and a bandwidth price of $0.072 / GB.

[0386] Translation module verification

[0387] The end-to-end delay of the ASR-NMT-TTS link is controlled within 900ms.

[0388] Generate bilingual subtitles and dubbing audio tracks in English and Arabic, with a MOS score of ≥4.0.

[0389] Multi-branch access verification

[0390] Beijing, Dubai, and Frankfurt branch nodes collaborate to forward:

[0391] Beijing → Dubai latency: 129ms

[0392] Dubai → Frankfurt latency: 85ms

[0393] System traffic distribution is automatically adjusted based on node scores, increasing bandwidth utilization by 32%.

[0394] Multilingual distribution verification

[0395] Live streaming platform: DASH encapsulation and streaming, multi-language switching delay <1s.

[0396] Live streaming platform: HLS encapsulation and streaming, subtitle switching without black screen or lag.

[0397] (3) Verification results

[0398]

[0399] Summary of technical effects

[0400] The overall system combination verification of this embodiment proves that the solution of the present invention has the following beneficial effects in terms of low latency, high stability, multi-language adaptation and operation cost optimization for global live broadcast:

[0401] End-to-end latency is reduced by more than 34%, significantly improving the user viewing experience.

[0402] Video stream compression verifies an average bandwidth saving of 45.7%.

[0403] Multi-cloud + multi-operator intelligent scheduling reduces bandwidth costs by 10% to 25% and improves stability by 50%.

[0404] The AI ​​translation module generates multilingual subtitles and dubbing in real time, with end-to-end latency controlled within 1 second.

[0405] The modular architecture supports collaborative forwarding among multiple branch nodes, and the system is highly scalable and adaptable to global business deployment.

[0406] The various embodiments in this specification are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the description of the method embodiments.

[0407] The foregoing is merely an embodiment of the present invention and is not intended to limit the present invention. It will be apparent to those skilled in the art that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention are intended to be included within the scope of the claims of the present invention.

Claims

1. An intelligent global cloud live broadcast system based on AI and multi-operator and multi-cloud optimization, characterized by: include: The data acquisition module is used to obtain the original video stream data generated by the local live broadcast device; A video compression module is used to compress the original video stream based on a preset H.265 encoding algorithm to generate a compressed video stream; The content review module is used to perform video frame image recognition, audio ASR transcription, and NLP semantic analysis on the compressed video stream through the content security API to implement multimodal content review of the video stream and obtain the video stream that has passed the review; RPMTS encapsulation module: used to perform RPMTS real-time encapsulation on the video stream that has passed the review, generating an encapsulated data stream suitable for public network transmission; The public network access module is used to connect the RPMTS encapsulated data stream to the nearest cloud-to-cloud dedicated line POP node of the three major operators (such as China Telecom, China Unicom, and China Mobile) through the public network, realizing a bridge connection with the operator's dedicated line; The decoding module is used to perform H.265 decoding on the received compressed video stream at the intermediate POP node and restore it to the original video frame data; The translation module is used to translate the decoded video and audio content in real time based on AI speech recognition and neural network machine translation algorithms, generating subtitles and dubbing streams in the target language; Multi-operator and multi-cloud adaptability matrix module: This module intelligently calculates and selects the optimal live broadcast distribution path based on the user's egress operator attributes and the cost and performance of multi-cloud nodes. Multi-branch access module supports multiple branch nodes to access the live broadcast system simultaneously, forming distributed collaboration and multi-point cloud forwarding capabilities; The multilingual distribution module generates multilingual subtitles and dubbing through AI real-time translation, and supports API access to live broadcast platforms to achieve global multilingual synchronous distribution.

2. The intelligent global cloud live broadcast system based on AI and multi-operator and multi-cloud optimization according to claim 1 is characterized by: The content review module includes: The video frame detection unit is used to capture each frame of the compressed video stream, call the OCR algorithm to recognize text content, and detect illegal content such as political, pornographic, and brand infringement; The audio recognition unit is used to perform ASR speech transcription on the audio part of the video stream and convert the speech into text content; The NLP semantic analysis unit is used to perform natural language processing and sensitive word analysis on the transcribed text to determine whether there are any illegal semantics; The review result output unit is used to output the review results and push the video stream that has passed the review to the RPMTS packaging module.

3. The intelligent global cloud live broadcast system based on AI and multi-operator and multi-cloud optimization according to claim 1 is characterized by: The RPMTS package module includes: The fragment encapsulation unit is used to perform real-time fragment encapsulation of the video stream that has passed the review with a granularity of 100ms; The encapsulation protocol unit adopts UDP over RTP protocol and supports packet loss retransmission mechanism; The encapsulation control unit is used to control the number of encapsulation retries. The default retry is 3 times to ensure the stability of cross-region public network transmission.

4. The intelligent global cloud live broadcast system based on AI and multi-operator and multi-cloud optimization according to claim 1 is characterized by: The translation module includes: The speech recognition subunit uses the Conformer structural acoustic model and the Transformer language model to recognize video and audio content in real time; The neural network translation subunit, based on the M2M-100 and Transformer neural network machine translation models, translates the recognized text into the target language text; The subtitle generation subunit converts the translated text into an SRT standard subtitle file and synchronizes the time code with the video stream; The dubbing generation sub-unit calls the AI ​​speech synthesis interface to generate the target language dubbing audio stream from the translated text and synchronously encapsulate it with the video stream.

5. The intelligent global cloud live broadcast system based on AI and multi-operator and multi-cloud optimization according to claim 1 is characterized by: The multi-operator and multi-cloud adaptability matrix module includes: The operator attribute identification unit is used to identify the operator ASN and geographical location corresponding to the branch egress IP address and generate user operator attribute information; Cost and performance data collection unit, used to obtain bandwidth prices, rental prices, traffic billing models, and real-time performance indicators of each cloud service provider's POP nodes through API interfaces, including bandwidth utilization, average latency, packet loss rate, and availability; The adaptation coefficient calculation unit is used to calculate the adaptation coefficient based on whether the user's exit operator is consistent with the operator of each target POP node, and give priority to nodes directly connected to the same operator; A matrix construction unit, used to combine operator adaptation coefficients, cost data, and performance indicators to build a multi-operator and multi-cloud adaptability matrix; The path scoring calculation unit is used to calculate the comprehensive score of each POP node based on the constructed multi-operator and multi-cloud adaptability matrix and preset weight parameters; The optimal node output unit is used to select the POP node with the highest score as the optimal path based on the comprehensive scoring results, and output other nodes with scores higher than the preset threshold as redundant paths.

6. The intelligent global cloud live broadcast system based on AI and multi-operator and multi-cloud optimization according to claim 1 is characterized by: The matrix construction unit includes: The weight configuration subunit is used to configure the weight parameters of cost, delay, packet loss rate and adaptation coefficient according to the business scenario; The comprehensive score calculation subunit normalizes the cost, performance, and adaptability of each POP node based on the configured weight parameters and calculates a comprehensive score; The node sorting subunit sorts all POP nodes according to the comprehensive scores and generates the optimal POP node list. The path score calculation subunit is used to calculate the path score based on the matrix results combined with the weight parameters; The optimal node output subunit is used to select the optimal node according to the path score and output a redundant node list.

7. The intelligent global cloud live broadcast system based on AI and multi-operator and multi-cloud optimization according to claim 1 is characterized by: The multi-branch access module includes: Branch identification unit, used to collect the public network IP, ASN, geographic location, egress bandwidth and computing resource information of branch nodes; Access control unit, used for API key authentication, access permission verification and registration management of branch nodes; The stream access and forwarding unit is used to receive the live stream and play it locally on the branch. It forwards the stream to other branch nodes according to system scheduling to achieve distributed collaborative distribution.

8. The intelligent global cloud live broadcast system based on AI and multi-operator and multi-cloud optimization according to claim 1 is characterized by: The multilingual distribution module includes: The multi-language generation unit is used to synchronously encapsulate the target language subtitles and dubbing streams generated by the AI ​​translation module with the original video stream; The API interface access unit is used to push multi-language live streams to various mainstream live broadcast platforms through the API interface, realizing global multi-language synchronous distribution.

Images