A flow address unique key multi-tenant live commentary pipeline management method and system

Abstract

The application discloses a kind of flow address unique key multi-tenant live commentary pipeline management method and system, server is divided into access, control, media AI three planes, with stream identification code as whole link unique isolation key.Passage is only authenticated to flow, and stream is not automatically started commentary pipeline;Control plane is explicitly started and stopped live through interface, and broadcast verification source flow prevents ghost stream, and intelligent splicing stream address adapts multiple platforms.Media AI layer is isolated based on code, process, knowledge base, six-dimensional physical isolation such as barrage, and each tenant pipeline is completely independent.The system sets watchdog every 15 seconds to inspect, and automatically recycles zombie stream when flow is broken for more than 180 seconds;Broadcast empty audio cache to avoid tail sound residue, and barrage link can be delayed after broadcast Delay is issued.Control plane supports horizontal expansion, and media plane uses node fixed node distributed architecture.The application solves the problems of traditional scheme tenant crosstalk, resource waste and frequent misoperation, and can scale to carry multiple concurrent AI live commentary.

Description

Technical Field

[0001] This invention belongs to the technical fields of multi-tenant cloud media stream processing, AI real-time live commentary, and server-side pipeline scheduling, specifically involving a multi-tenant live commentary pipeline management method and system with a unique stream address key. Background Technology

[0002] With the rapid popularization of cloud services for AI-powered live streaming commentary, the industry is gradually shifting from a single-channel trial model to a multi-tenant, multi-channel concurrent large-scale operation model. Traditional live streaming commentary server architectures, during large-scale commercialization, have gradually exposed numerous technical shortcomings, such as poor multi-tenant isolation, lack of business orchestration, susceptibility to accidental pipeline start / stop, wasteful use of botnet resources, and weak platform compatibility, failing to meet the demands of stable commercial operation.

[0003] In existing technologies, the mainstream live commentary service architecture is mainly divided into three categories, and the corresponding technical shortcomings are as follows: The first type is a single-machine, single-channel commentary architecture. The server uses a global singleton variable to manage the live streaming pipeline, and globally uniformly occupies frame extraction, mixing, and audio queue resources. A single server can only support a single live commentary service. If multi-channel concurrency is required, it can only be achieved through multiple instances and multiple deployments. This results in extremely low server resource utilization, high deployment costs, and the knowledge bases, commentator personas, and output push links of each live stream cannot be isolated at all. This easily leads to data crosstalk and garbled commentary content, making it completely unsuitable for multi-tenant commercial scenarios.

[0004] The second type is a multi-user architecture based on session login state. This architecture relies on front-end web page login session identifiers to distinguish different users, and the live streaming pipeline is strongly bound to the browser session and login account. This architecture is highly dependent on front-end interaction and login state, and cannot achieve pure back-end interface orchestration, nor can it be automatically called by the upper-level management platform. At the same time, the session identifier has no natural binding relationship with the media stream, and the media stream cannot be accurately matched to the corresponding tenant after being pushed. It cannot achieve fine-grained isolation of tenant knowledge base, bullet screen source, and output push target, and the independence of tenant business cannot be guaranteed.

[0005] The third type is the push-stream automatically triggered architecture. The media gateway automatically starts the commentary pipeline when it detects a push-stream action and stops the pipeline when the stream is interrupted. This architecture has a high degree of business uncontrollability. The timing of the broadcast, the commentator's persona, the output platform, and the tone style cannot be customized and can only use default parameters. At the same time, short-term network jitter and instantaneous disconnection and reconnection will cause the pipeline to start and stop frequently, which will seriously affect the stability of the live broadcast. An even more prominent drawback is that when the streamer disconnects or the device is shut down, the server-side pipeline cannot automatically detect the complete disconnection. It will retain idle frame-skipping and mixing subprocesses for a long time, forming a large number of zombie streams. These streams continuously occupy the server's CPU, memory, and bandwidth resources, causing resource waste, system lag, and long-term operation can easily lead to service crashes.

[0006] In addition, existing technologies cannot achieve multi-dimensional physical isolation of tenants, and multiple tenants share a global knowledge base and configuration parameters, resulting in low tenant data security. At the same time, there is a lack of standardized multi-platform push address adaptation mechanisms, leading to poor compatibility with address formats of different third-party live streaming platforms. There is no pre-verification mechanism for starting a broadcast, which can easily result in ghost commentary streams without a source. There is no cache cleanup mechanism for ending a broadcast, which can easily lead to the problem of residual audio being pushed after the broadcast ends. Overall, the system has significant shortcomings in stability, compatibility, and self-healing.

[0007] In summary, existing live commentary pipeline technologies suffer from several technical defects, including poor multi-tenant isolation, weak concurrency capabilities, lack of business orchestration, susceptibility to accidental pipeline start-up and shutdown, inability to automatically reclaim zombie streams, poor platform compatibility, and low resource utilization. There is an urgent need to develop a multi-tenant live commentary pipeline management solution based on unique stream address keys, multi-dimensional physical isolation, interface-based orchestration, and self-healing capabilities. Summary of the Invention

[0008] This invention aims to solve the technical problems of poor isolation, weak concurrent scheduling capabilities, inability to automatically orchestrate services, easy accidental start / stop due to network jitter, inability to automatically reclaim zombie stream resources, poor multi-platform adaptability, and cache residue interfering with the live broadcast effect in existing technologies. It provides a method and system for managing multi-tenant live broadcast pipelines with unique stream address keys, realizing six-dimensional physical isolation of multi-tenants, decoupling and controllable push and broadcast, intelligent multi-platform adaptation, and automatic self-healing and reclamation of zombie streams, thereby improving the stability, security, and operability of multi-tenant concurrent live broadcast commentary.

[0009] This invention adopts the following technical solution: a multi-tenant live streaming commentary pipeline management method with unique stream address keys, running on a server divided into three planes: access plane, control plane, and media and artificial intelligence plane, including the following steps: S1. Tenant Code Allocation and Access Face Media Stream Authentication Steps: The upper-layer management platform assigns a unique stream address identifier (code) to each merchant or live channel. The tenant device's push stream address is configured as the media gateway base address / live / {code}. The access face media gateway receives the original audio and video streams pushed by the tenant into the live / {code} path, triggering the push stream authentication callback on_publish. It only verifies the code's validity and the push stream credentials; if verification passes, the media stream is allowed; otherwise, it is rejected. The on_publish authentication callback only executes the verification logic and does not start the AI ​​commentary pipeline. This step achieves accurate media stream authentication while completely decoupling the push stream action from the pipeline's start / stop, avoiding the uncontrollable problem of starting broadcast immediately upon push stream.

[0010] S2. Explicit Broadcast Scheduling and Pre-Verification Steps in the Control Plane: The upper-layer management platform calls the broadcast interface POST / api / live / {code} / start through the stateless REST control plane. The broadcast request carries the third-party platform's pushUrl, push key, platform identifier, language, timbre, gender, and opening remarks parameters. Pre-broadcast verification is performed by calling the media gateway to check if live / {code} has an active push stream. If no active push stream exists, broadcasting is rejected to avoid ghost output from a sourceless stream. After successful verification, the tenant's unique persona configuration is loaded from the persona table using code as an index. The timbre, gender, and opening remarks parameters not sent in the broadcast request are handled by the persona configuration. Pre-broadcast source stream verification eliminates ghost streams from a sourceless stream, while also supporting parameter customization and default configuration fallbacks to adapt to diverse tenant narration needs.

[0011] S3. Steps for creating an independent AI narration pipeline based on code six-dimensional physical isolation: Using code as the unique isolation key for the stream address, create an independent pipeline object for the current code, and simultaneously create a dedicated frame extraction subprocess, a mixing and pushing subprocess, and an application-layer priority audio queue. Execute the following six-dimensional physical isolation sub-steps in sequence: S31. Process Isolation: Each code has its own independent pipeline, frame extraction subprocess, mixed streaming push subprocess and independent worker thread. Each tenant's pipeline process is independent of each other, and process resources are completely isolated, so there will be no process preemption or resource interference issues.

[0012] S32. Stream source isolation: The stream input address is fixed and concatenated as the media gateway base address / live / {code}. Each code stream link is independent and does not interfere with each other, completely eliminating the problem of mixed and disordered media streams from different tenants.

[0013] S33. Knowledge Base Isolation: An independent vector set (kb) is created for each code within the vector database. Physical isolation is used instead of global database filtering. This supports creating, deleting, and rebuilding the knowledge base for a single code. Tenant knowledge base data is completely independent, ensuring extremely high security and independence.

[0014] S34. Persona Configuration Isolation: The persona table stores tenant-specific commentary style, title, catchphrase, prohibited words, and voice parameters with code as the primary key. Each tenant has an independent commentary persona system, with differentiated and customized commentary styles.

[0015] S35. Isolation of Bullet Comment Sources: Each code is bound to a dedicated live stream collection address. Bullet comment data is routed to the corresponding pipeline according to the code. Bullet comment data from different live streams do not cross stream, ensuring that the commentary and interaction are accurately matched to the corresponding tenant's live stream scenario.

[0016] S36. Output Target Authentication Isolation: The third-party pushUrl, streamKey and code issued at the start of broadcast are bound one by one. Each tenant's output push stream link is independently authenticated. The push stream targets and authentication credentials of different tenants are completely isolated to avoid cross-platform and cross-tenant push stream confusion.

[0017] S4. Intelligent output address splicing, broadcast initialization, and decoupled bullet screen distribution steps: The splicing adaptation logic is executed on pushUrl and streamKey. If pushUrl does not end with " / " and streamKey does not start with " / " or "?", a separator " / " is automatically added between them to generate a complete output push stream address that conforms to the third-party platform's specifications, compatible with the address format standards of different live streaming platforms. An opening speech is synthesized based on the code's unique persona, sent to the mixed streaming push subprocess, and occupies the playback timeline. The current code is written to the on-air registration table. The bullet screen live room address distribution and broadcast process are decoupled. After the pipeline starts, the upper-level management platform separately distributes the live room sharing text. The system automatically extracts the live room link from the sharing text and starts / stops the corresponding bullet screen collection task according to the code, adapting to the business scenario of generating the link after the live room is adapted.

[0018] S5, precise code shutdown and cache cleanup steps: After the upper-level management platform calls the stop shutdown interface on the control plane, it immediately terminates the frame extraction subprocess corresponding to the current code; synthesizes the shutdown voice, clears all backlogged historical narration audio cache in the mixing and pushing subprocess, and only retains the shutdown voice cache; after waiting for the shutdown voice to play completely and reserving a small amount of time margin, it forcibly destroys the mixing and pushing subprocess, removes the code from the on-play registration table, and simultaneously stops the bullet screen collection task corresponding to the code, completely eliminating the problem of residual cached audio being pushed after shutdown.

[0019] S6. Watchdog thread periodically inspects and automatically eliminates zombie streams: An independent watchdog thread is started, which cyclically inspects all playing codes at a fixed 15-second interval. Each inspection queries the media gateway for the corresponding code's push activity flag, `publish.active`. If the query result shows active push activity, or if the gateway query fails, the source stream disappearance timer for that code is reset to zero. If no active push activity is detected for a code consecutively, the source stream disappearance duration is continuously accumulated. When the continuous disappearance duration exceeds the 180-second jitter tolerance threshold, the shutdown process in step S5 is automatically and completely executed, reclaiming all resources from the frame extraction and mixing subprocesses, and cleaning up zombie streams. Through a dual fault tolerance mechanism, pipeline erroneous start / stop caused by network jitter and monitoring failures is avoided.

[0020] Furthermore, the control plane is a stateless, login-free, and session-free interface layer, relying on the X-Api-Key platform-level key in the request header to complete global authentication; all operations such as starting broadcast, stopping broadcast, querying live status, character editing, uploading vectorized knowledge base, configuring timbre, and setting bullet screen address all carry a code and verify the code's ownership permissions. The control plane as a whole supports horizontal scaling and can adapt to large-scale concurrent business scenarios.

[0021] Furthermore, the media streaming action is completely decoupled from the pipeline's broadcast trigger. Only the upper-level management platform actively initiates broadcast start and stop control through the control plane interface. The media gateway's on_publish callback is only used for authentication and will not automatically start or stop the AI ​​commentary pipeline, thus achieving complete controllability and programmability of the business.

[0022] Furthermore, in step S33, the vector library kb is an independent physical index set. The knowledge base data corresponding to different codes are isolated from each other, and the retrieval does not interfere with each other. Deleting a single code knowledge base will not affect the knowledge bases of other tenants, thus ensuring the security and independence of tenant data.

[0023] Furthermore, step S6 sets up a dual fault tolerance mechanism for the watchdog logic: if the gateway query fails, it will uniformly determine that the source stream is online normally and will not accumulate the disappearance time to prevent the monitoring component from mistakenly shutting down the normal live broadcast; the 180-second threshold is compatible with short-term network disconnection and reconnection to avoid pipeline destruction caused by network jitter.

[0024] This invention also provides a multi-tenant live streaming pipeline management system with a unique stream address key, used to execute the multi-tenant live streaming pipeline management method with a unique stream address key. The system is divided into an access plane module, a control plane module, and a media and artificial intelligence plane module. The stream address identifier code serves as the unique stream address key and is a unique tenant isolation key that runs through the entire link of the three modules.

[0025] The access plane module is a media gateway unit, used to receive the original audio and video streams pushed by each tenant to the live / {code} path, configure the on_publish push stream authentication callback logic, only verify the legality of the code and the push stream credentials, and the authentication logic does not trigger the start and stop of the artificial intelligence commentary pipeline.

[0026] The control plane module is a stateless REST interface unit with no login or session mechanism. It completes platform-level global authentication through the request header X-Api-Key and exposes interfaces for starting and stopping broadcasts, querying live status, configuring personas, uploading / vectorizing knowledge bases, configuring audio timbre, and configuring the address of the live chat room, which are bound to the code. These interfaces are then called and orchestrated by the upper-level management platform.

[0027] The media and AI module is a stateful pipeline management unit, comprising a pipeline manager, several independent pipeline instances, a watchdog thread unit, a vector storage unit, and a relational storage unit. The pipeline manager is configured to create and destroy independent pipeline objects using the code as the unique key index for the stream address. Each pipeline has a built-in dedicated frame extraction subprocess, a mixed streaming push subprocess, and an application-layer priority audio queue. Each pipeline is only bound to the dedicated pull address, third-party output push address, and bullet screen collection task for the corresponding code. The vector storage unit allocates an independent physical vector set (kb) for each code to achieve physical isolation of the multi-tenant knowledge base. The watchdog thread unit periodically traverses all playing codes every 15 seconds, queries the media gateway's publish.active push activity flag, and automatically triggers a shutdown process to reclaim all pipeline subprocess resources when a code has no push activity for more than 180 seconds, eliminating zombie streams that have been occupying resources for a long time.

[0028] Furthermore, the control plane module is a stateless service cluster that supports horizontal scaling; the media and artificial intelligence plane module is a stateful node service, with the pipeline corresponding to a single code fixed to a single node. The system has a built-in code-node registration table to support multi-node distributed scaling deployment.

[0029] Furthermore, the control plane module's broadcast interface has a built-in intelligent output address splicing component; the splicing component executes the following logic: when the input pushUrl does not end with a separator " / " and the first character of streamKey is not " / " or "?", it automatically adds " / " between the two to splice the complete push stream address, which is compatible with the address format specifications of two types of third-party live streaming platforms.

[0030] Furthermore, each pipeline has a built-in shutdown buffer cleanup component; when a shutdown command corresponding to the code is received, the component clears all historical narration audio cache in the mixing and pushing subprocess, retaining only the real-time generated shutdown voice; after the shutdown voice has been played completely and the preset margin duration has been added, the mixing and pushing subprocess is forcibly destroyed to prevent the cached audio from being continuously pushed out after shutdown.

[0031] Furthermore, the system is also configured with a bullet screen parsing and distribution subunit, which is bound to a code one by one; it supports delayed reception of the live room sharing text sent down from the upper layer after the pipeline is created, automatically extracts the live room access link from the sharing text, starts and stops the corresponding live room bullet screen collection task according to the code, and the bullet screen data is only routed to the corresponding code pipeline for artificial intelligence commentary and interaction.

[0032] The beneficial effects achieved by the present invention using the above solution are as follows: 1. This invention uses the stream address code as the unique isolation key to achieve physical isolation in six dimensions: process, stream source, knowledge base, character configuration, bullet screen source, and output authentication. The multi-tenant pipeline runs completely independently, with no data crosstalk or resource preemption. A single server can support concurrent live commentary from multiple tenants, significantly improving server resource utilization and reducing large-scale operation costs.

[0033] 2. The stateless control plane interface enables proactive and controllable operation of the entire process of starting and stopping broadcasting and configuring parameters. No front-end login session is required. It can be seamlessly connected to the upper-level management platform and is suitable for automated and batch live broadcast operation scenarios.

[0034] 3. Built-in intelligent concatenation logic for push streaming addresses can automatically adapt to the address formats of different third-party live streaming platforms without requiring manual configuration modifications, reducing the cost of multi-platform deployment and adaptation.

[0035] 4. By employing a 15-second watchdog cycle for inspection, a 180-second network jitter tolerance threshold, and a monitoring fault release mechanism, the system balances network fluctuation adaptability with the effectiveness of botnet cleanup, automatically reclaims idle resources, avoids long-term resource occupation, and ensures long-term stable operation of the system.

[0036] 5. Pre-broadcast source stream verification eliminates ghost commentary without source, and forced cache clearing after broadcasting avoids audio residue, solving abnormal output problems from the source and greatly improving the stability of live commentary picture and sound quality.

[0037] 6. The control plane is stateless and can be horizontally scaled. The media plane is deployed in a distributed manner according to code pin nodes, supporting multi-node cluster expansion, and can adapt to massive tenants and high-concurrency live broadcast commentary business scenarios. Attached Figure Description

[0038] Figure 1 This is a schematic diagram of the overall three-plane architecture of the present invention; Figure 2 This is a flowchart of the broadcast scheduling process for this invention; Figure 3 This is a schematic diagram of the automatic recycling process for the Watchdog zombie stream of the present invention. Detailed Implementation

[0039] The following detailed description, in conjunction with the accompanying drawings and embodiments, clearly and completely illustrates the complete technical solution, system architecture, core code isolation mechanism, three-plane layered logic, six-dimensional physical isolation system, decoupling and orchestration of broadcast start and stop, watchdog self-healing mechanism, decoupling configuration of bullet comments, independent isolation of the knowledge base, and multi-tenant concurrent implementation scenarios of the present invention. The following embodiments represent preferred implementation scenarios of the present invention. All equivalent embodiments and equivalent alternative architectures obtained by those skilled in the art without creative effort are within the protection scope of the present invention.

[0040] To facilitate a clear understanding of the overall technical solution of this invention, the core concept of the unique isolation key code, the three-plane overall architecture, the six-dimensional isolation standard, and the full-process orchestration logic of this invention will be explained in a unified manner first, and then verified one by one in detail with specific implementation examples.

[0041] like Figure 1-3 As shown, Definition of core technologies and overall solution description This invention uses a unique identifier string at the end of the streamer's push address as a unique isolation key, named "code". "Code" is the core tenant identifier that runs throughout the entire system chain. Specifically, it is a custom identifier character at the end of the pull address rtmp: / / media gateway / live / {code}. For example, in the push address rtmp: / / media gateway / live / cam01, "cam01" is the unique "code" identifier. During the business deployment phase, the upper-level management platform assigns a unique "code" to each merchant, each live channel, and each smart glasses live streaming device, and uniformly points the push addresses of the corresponding tenants and devices to the live / {code} path of the system's media gateway.

[0042] Based on this design, once a media stream is connected to the media gateway, the system can directly and accurately determine the tenant's affiliation through the stream address itself, without relying on additional binding conditions such as front-end login accounts, browser sessions, and user cookies, thus completely eliminating the session dependency problem of traditional multi-tenant architectures. In this invention, one code uniquely corresponds to one independent live streaming service, one independent tenant, and one dedicated AI commentary pipeline. All of the tenant's operating resources, configuration parameters, and data resources are all mounted on the corresponding code dimension, achieving a natural binding between resources and tenants.

[0043] Three-plane layered architecture This invention's server-side deployment is a three-layer decoupled architecture consisting of an access plane, a control plane, and a media and AI plane. Each plane has clearly defined responsibilities and completely decoupled functions, balancing access universality, business orchestration, media processing isolation, and system scalability. The overall structure corresponds to the attached... Figure 1 The standardization capabilities of each plane are as follows: Access plane (media gateway layer): The core is the media gateway unit, whose sole responsibility is media stream access and validity verification. It is responsible for receiving raw audio and video streams pushed into the live / {code} path by various broadcasters and smart glasses devices; configuring the on_publish push stream authentication callback mechanism, it only verifies the legality of the corresponding code and the validity of the push stream credentials. If the verification passes, the media stream is allowed; if the verification fails, it is directly blocked and rejected. This plane only handles traffic access and authentication, and does not trigger any AI commentary pipeline start / stop operations, achieving complete decoupling between push stream actions and business broadcasting.

[0044] Control plane (REST interface layer): This is a stateless, login-free, and session-free pure backend interface orchestration layer, supporting horizontal scaling. Global authentication is achieved using the X-Api-Key platform-level key in the request header. Standardized interfaces bound to unique codes are exposed externally, covering all business interfaces such as start / stop broadcasting, live stream status query, commentator setup, knowledge base upload and vectorization, audio parameter configuration, and live chat room configuration. All write operations verify code ownership permissions, ensuring accurate and controllable business operations. Seamless integration with upper-level management platforms enables automated, batch live stream orchestration.

[0045] Media and Artificial Intelligence Layer (Stateful Computing Layer): This layer serves as the core business computing and resource isolation layer. It is uniformly scheduled by the PipelineManager, creating and destroying independent pipeline objects using code as the unique index. Each pipeline corresponding to a code has its own dedicated frame extraction subprocess, mixed streaming subprocess, application-layer priority audio queue, character configuration handle, independent knowledge base handle, and dedicated bullet screen collection source. Furthermore, the pipeline is fixed to a single service node, eliminating node drift, resource corruption, and data inconsistency issues.

[0046] A code-based six-dimensional physical isolation system This invention differs from traditional database logic filtering and field differentiation-based soft isolation solutions. It achieves comprehensive, physical-level strong isolation, with all six isolation dimensions using the code as the unique binding key. This completely solves problems such as multi-tenant crosstalk, resource contention, data leakage, and misinterpretation. The specific isolation mechanism is as follows: Process isolation: Each code corresponds to an independent pipeline, an independent frame extraction subprocess, an independent mixing and pushing subprocess, and an independent worker thread. Process resources are independent of each other, there is no CPU or memory resource contention, and single-tenant failures do not spread or affect other tenants.

[0047] Isolation of streaming source: The input streaming address is uniformly concatenated as {media gateway base address} / live / {code}, and each code pulls the stream independently without interfering with each other. The media stream links are physically independent, completely eliminating cross-tenant streaming and stream mixing issues.

[0048] Knowledge base isolation: Each code in the vector database has an independent physical vector set (kb), with independent indexing and storage. It is not a global library filter and supports single-tenant knowledge base creation, deletion, and reconstruction. Tenant data is completely isolated and secure.

[0049] Persona / Configuration Isolation: The persona table uses code as the primary key and independently stores customized parameters such as commentary style, title, catchphrase, prohibited words, opening remarks, and default tone. Each tenant's commentary persona is customized differently, and the parameters do not overwrite or interfere with each other.

[0050] Bullet comment source isolation: Each code is bound to a unique live stream address, and bullet comment data is accurately routed to the corresponding pipeline according to the code. Bullet comments are accurately matched with the live stream scene, and there is no cross-scene bullet comment interference.

[0051] Output target / authentication isolation: Each code is bound to a unique third-party pushUrl and streamKey. Both interface operations and push stream authentication verify the code's ownership. The output push stream target is independent, and there are no cross-tenant or cross-platform push stream misalignment issues.

[0052] The isolation mechanism is shown in the table below: Standardized programming method for decoupling live broadcast and streaming One of the core innovative mechanisms of this invention is the complete decoupling of broadcast activation triggering and media streaming. It abandons the traditional passive triggering mode where streaming is equivalent to broadcast activation. The entire process is explicitly and actively driven by the upper-level management platform through the control plane interface, achieving full controllability and programmability of the business process. (Corresponding appendix...) Figure 2 The standardized steps for starting a live broadcast are as follows: Step 1: Interface Request Issuance: The upper-level management platform calls the standardized live broadcast interface POST / api / live / {code} / start. The request header carries the platform-level X-Api-Key key, and the request body sends custom parameters such as the third-party platform pushUrl, push stream key, platform identifier, language, timbre, gender, and opening remarks.

[0053] Step 2, Pre-broadcast Verification to Prevent Ghost Streams: The system actively queries the media gateway status to verify whether there is an active push stream in the current live / {code} path. If there is no valid source stream, the broadcast request is directly rejected, completely avoiding the ghost output problem of no stream running or no source narration.

[0054] Step 3: Loading the exclusive persona: The system reads the exclusive persona configuration library using the code as an index. If the live broadcast request does not include custom parameters such as timbre, gender, and opening remarks, the default configuration in the library will be automatically loaded as a fallback, taking into account both customization and universality.

[0055] Step 4: Independent Pipeline Initialization: Initialize tenant-specific data and audio queues using code as the unique key, clear historical cache and timeline data, create and start dedicated frame extraction subprocesses and mixed streaming push subprocesses, and bind dedicated pull stream addresses and output push stream parameters issued by the business side.

[0056] Step 5: Intelligent adaptive splicing of output address: The system has built-in adaptive splicing logic to adapt to the address specifications of two mainstream live streaming platforms: When pushUrl does not end with " / " and streamKey does not start with " / " or "?", the separator " / " is automatically added between the two to generate a compliant and complete push stream address without manual adaptation and modification.

[0057] Step 6, Initialization and Closure of Live Broadcast: Based on the exclusive persona, a customized opening voice is synthesized, sent to the mixed subprocess and occupied by the playback timeline to complete the initialization of the live broadcast. At the same time, the current code is entered into the system's live broadcast registration form to officially start the live commentary business.

[0058] Standardized shutdown procedure: After the upper layer calls the shutdown interface, the frame extraction subprocess corresponding to the current code is immediately terminated, and the acquisition of new images and generation of narration are stopped; a dedicated shutdown closing voice is synthesized in real time, and all backlogged historical audio caches in the mixing subprocess are forcibly cleared, retaining only the latest shutdown voice; after the shutdown voice has played completely and a safety margin of time has been reserved, the mixing push subprocess is forcibly destroyed, and the remaining cached data is completely discarded to prevent audio residue from being pushed after shutdown; finally, the code is removed from the on-air registration table, and the corresponding bullet screen acquisition task is stopped simultaneously to achieve complete resource reclamation.

[0059] Source disappearance self-healing watchdog anti-zombie mechanism The system has a resident, independent watchdog self-healing thread, with corresponding appendices. Figure 3 The self-healing process employs a fixed 15-second inspection cycle and a 180-second network jitter tolerance threshold to achieve automatic zombie stream cleanup and fault tolerance. The core logic is as follows: 1. Normal live streaming / monitoring anomaly scenarios: Periodically query the publish.active push stream activity flag of the corresponding code. If the push stream is active, or if the gateway interface query times out, reports an error, or returns no response, the system will determine that the source stream is online normally, clear the source stream disappearance timer for that code, and continue normal inspection to avoid monitoring failures from accidentally stopping normal live streaming.

[0060] 2. Stream interruption scenarios: If the code push stream status is continuously detected as inactive, the source stream disappearance time will be continuously accumulated; when the continuous disappearance time exceeds the threshold of 180 seconds, it will be judged as an invalid zombie pipeline, and the complete standardized shutdown process will be automatically triggered to reclaim all subprocess resources of frame extraction and mixing, and completely eliminate background idle and resource occupation issues.

[0061] The mechanism has dual fault tolerance: a 180-second threshold adapts to short-term disconnection and reconnection scenarios such as mobile networks and wireless live streaming, avoiding accidental shutdown of normal live streaming due to network jitter; and a monitoring and abnormal release mechanism prevents service components from being shut down due to malfunctions.

[0062] Decoupling and scheduling mechanism for bullet screen collection and broadcasting Addressing the real-world pain points of live stream sharing links being generated only after the broadcast begins and the inability to pre-configure bullet screen addresses during the broadcast phase, this invention features a unique decoupling mechanism between bullet screen configuration and the broadcast process. During the pipeline's broadcast startup phase, no bullet screen parameters need to be configured. Once the business side starts the live stream and generates the live stream sharing text, the upper-level management platform can delay sending the complete sharing text. The system has a built-in bullet screen parsing and distribution subunit that can intelligently extract the live stream access link from sharing texts of any format, accurately bind it according to the current code, and start and stop the corresponding live stream's bullet screen collection task as needed. Bullet screen data is routed entirely by code, serving only the AI ​​interactive commentary of the corresponding tenant's pipeline, highly adaptable to actual commercial business scenarios.

[0063] Preferred embodiment Example 1: Implementation of Multi-channel Concurrent Orchestration for Live Streaming by Smart Glasses Merchants This embodiment is the core commercial implementation of the present invention. It uses a real deployment environment with multiple smart glasses devices and mobile phone relay streaming to verify the full-process capabilities of multi-code multi-tenant concurrent isolation, interface orchestration, intelligent address splicing, and independent business operation.

[0064] This embodiment configures two independent live streaming tenants, each assigned a unique code: cam01 and shop8899, corresponding to two smart glasses live streaming devices and two independent merchants. All devices' push streaming addresses uniformly point to the dedicated paths live / cam01 and live / shop8899 of the system's media gateway. The upper-layer management platform issues broadcast start commands separately through a standard REST interface, enabling the two live streams to operate completely independently.

[0065] The standard live streaming request example for code=cam01 is as follows: Request method: POST / api / live / cam01 / start Authentication request header: X-Api-Key: {Platform-level global key} Request body parameters: {"pushUrl":"rtmp: / / target platform / live","streamKey":"570xxx?txSecret=...","platform":"xhs","language":"zh-CN","voice":"female1","gender":"female","openingScript":"Welcome to the live stream, everyone!"} Based on the parameters issued this time, the system automatically executes the intelligent address concatenation logic: the pushUrl is "rtmp: / / target platform / live", without a trailing " / " separator, and the streamKey does not have a " / " or "?" prefix at the beginning, which meets the automatic completion conditions. The system automatically adds a separator " / " between the two, and finally generates a compliant and complete push stream address: rtmp: / / target platform / live / 570xxx?txSecret=..., which is compatible with the address format specifications of third-party platforms and does not require manual configuration modification.

[0066] During operation, the system creates independent pipelines for the two codes cam01 and shop8899: each live stream pulls its own media stream, loads its own custom narrator persona, enables independent frame extraction and mixing subprocesses, and pushes content to its corresponding third-party live streaming platform. Simultaneously, the two tenants are bound to independent knowledge bases kb_cam01 and kb_shop8899, ensuring complete physical data isolation. This allows for multi-channel concurrent operation without crosstalk, resource contention, or narration errors. A single server can stably support multiple commercial live streaming services, significantly reducing the cost of large-scale deployment.

[0067] Example 2: Implementation of Intelligent Splicing and Adaptation of Output Address Addressing the differences in address formats between two mainstream live streaming platforms, this system utilizes a built-in adaptive splicing component to achieve fully automatic compatibility and adaptation. This eliminates the need for manual configuration and error-prone adaptations, completely resolving the pain points of poor compatibility with traditional platforms and the ease of error during manual splicing. It covers all commercial RTMP push streaming address scenarios. The standard adaptation logic is as follows: After the broadcast interface receives the pushUrl and streamKey parameters, the control plane parameter adaptation component automatically identifies the format characteristics of the beginning and end of the fields. It automatically adapts to both mainstream formats without distinguishing platform types or requiring manual rule presets. When the pushUrl ends without a separator " / " and the first character of the streamKey is not " / " or "?", it automatically adds " / " to generate a complete and compliant push streaming address. The entire splicing process is bound to the code, and different tenant addresses are adapted and stored independently, preventing cross-tenant address errors and push streaming failures.

[0068] Example 3: Implementation of Watchdog Self-Healing Anti-Zombie Flow and Fault Tolerance Mechanism This embodiment employs a fixed 15-second inspection cycle and a 180-second network jitter tolerance threshold, covering four core scenarios: normal live streaming operation, instantaneous network jitter, long-term interruption of botnet streams, and gateway monitoring anomalies. This achieves precise self-healing, zero false positives, and zero resource residue. During normal live streaming, the system continuously resets the disappearance timer, ensuring stable pipeline operation. Brief interruptions are resolved within 180 seconds without triggering shutdown, preventing accidental pipeline destruction. Continuous interruptions exceeding 180 seconds are identified as botnet streams, automatically executing a complete shutdown and recycling process to thoroughly release resources. When the gateway monitoring interface malfunctions, times out, or reports errors, the source stream is assumed to be normal by default, and the timer is not accumulated, preventing accidental shutdown of normal live streaming due to monitoring failures. This dual fault-tolerance mechanism ensures system adaptability to unstable wireless networks while preventing business failures caused by monitoring anomalies, achieving stable operation 24 / 7.

[0069] Example 4: Dedicated Implementation of Physical Isolation for Multi-Tenant Knowledge Base This embodiment verifies the core capability of the physical-level knowledge base isolation of this invention. Unlike traditional soft isolation schemes that rely on global library tag filtering, the system allocates an independent set of physical vectors to each code, providing independent indexes and storage space. Taking two tenants, cam01 and shop8899, as examples, cam01's dedicated knowledge base stores smart glasses product parameters and product descriptions, while shop8899's dedicated knowledge base stores store store products and promotional after-sales scripts. Retrieval between the two is independent, and their data is completely separate. During the operation and maintenance phase, creation, deletion, and reconstruction operations can be performed on a single tenant's knowledge base without affecting other tenants' data. Data security and independence meet commercial compliance requirements.

[0070] Example 5: Implementation of Precise Broadcast Closure Cache Cleanup and End-of-Life Mechanism This embodiment implements a clean and thorough standardized shutdown process, resolving issues such as residual audio, lingering sound, and chaotic endings associated with traditional shutdown technologies. The specific process is as follows: After calling the shutdown interface, immediately terminate the frame extraction subprocess and stop generating new narration; synthesize tenant-specific shutdown audio; forcibly clear all historical audio caches from the mixing subprocess, retaining only the shutdown audio; wait for the shutdown audio to play completely while reserving a safety margin; destroy the mixing subprocess, stop bullet screen collection, and deregister from the live broadcast registration, fully releasing all CPU, memory, and bandwidth resources, completely eliminating audio residue and background idling issues after shutdown.

[0071] Example 6: Implementation of the mechanism for decoupling bullet comments and extracting dynamic links This embodiment is adapted to commercial scenarios generated after the live stream link is launched, decoupling the launch and bullet screen configuration. No bullet screen parameters need to be configured during the pipeline launch phase. After initialization, the upper layer can delay sending any format of live stream sharing text. The system's bullet screen parsing subunit intelligently extracts the live stream link and binds it to the current code, starting and stopping bullet screen collection tasks as needed. Bullet screen data is precisely routed according to the code throughout the process, serving only the corresponding tenant's AI interactive commentary, significantly improving the system's business adaptability and flexibility, and closely aligning with real-world operational processes.

[0072] Example 7: Distributed Cluster Expansion and High Availability Deployment Implementation This embodiment implements a layered scaling architecture, adaptable to large-scale, high-concurrency scenarios. The control plane adopts a stateless cluster architecture with no login or session dependencies, supporting unlimited horizontal scaling and capable of handling tens of thousands of concurrent requests. The media and AI planes employ a code node-based mechanism, using a code-node registry to fix the single-tenant pipeline to a single node, avoiding node drift and data corruption. The cluster's distributed load balancing can balance the hardware load of each node, supporting the platform's stable operation under normalized high concurrency and multi-tenant commercial live streaming services.

[0073] Example 8: Comprehensive Implementation of Six-Dimensional Full Physical Isolation for Dual-Tenant Concurrency This embodiment employs a dual-tenant long-term concurrent operation scenario to comprehensively verify the six-dimensional physical isolation capability. Each of the two tenants has an independent pipeline process, a dedicated streaming link, an independent vector knowledge base, a customized narrator persona, a dedicated source of bullet comments, and an independent push stream authentication link. The two tenants simultaneously perform operations such as frame-by-frame narration, speech synthesis, mixed streaming, knowledge base updates, persona modification, and pipeline start / stop, with no interference between their operations. A single tenant's failure, restart, or data maintenance will not affect the other live stream, truly achieving commercial-grade strong physical isolation and completely resolving industry pain points such as multi-tenant crosstalk, data leakage, and narration errors.

[0074] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to the embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

[0075] The present invention and its embodiments have been described above. This description is not restrictive, and the accompanying drawings are only one embodiment of the present invention; the actual structure is not limited thereto. In conclusion, if those skilled in the art are inspired by this description and design similar structures and embodiments without departing from the spirit of the invention, such designs should fall within the protection scope of the present invention.

Claims

1. A method for managing a multi-tenant live streaming commentary pipeline with a unique stream address key, operating on a server-side architecture divided into three planes: an access plane, a control plane, and a media and artificial intelligence plane. Its features include: Includes the following steps: S1. Tenant code allocation and access plane media stream authentication steps: The upper-layer management platform assigns a unique stream address identifier code to each merchant or live channel, and the tenant device push stream address is configured as the media gateway base address / live / {code}; The access plane media gateway receives the raw audio and video stream pushed by the tenant into the live / {code} path, triggers the push stream authentication callback on_publish, only verifies the legality of the code and the push stream credentials. If the verification passes, the media stream is allowed to pass; if the verification fails, it is directly rejected. The on_publish authentication callback only executes the verification logic and does not start the artificial intelligence commentary pipeline. S2. Explicit Broadcast Scheduling and Pre-Verification Steps in the Control Plane: The upper-layer management platform calls the broadcast interface POST / api / live / {code} / start through the stateless REST control plane. The broadcast request carries the third-party platform's pushUrl, push key, platform identifier, language, timbre, gender, and opening parameters. Pre-verification is performed, and the media gateway is called to query whether there is an active push stream in live / {code}. If there is no active push stream, broadcasting is rejected to avoid ghost output of sourceless streams. After the verification is passed, the tenant's exclusive persona configuration is loaded from the persona table using the code as the index. The timbre, gender, and opening remarks parameters that are not sent in the broadcast request are covered by the persona configuration. S3. Steps for creating an independent AI narration pipeline based on code six-dimensional physical isolation: Using code as the unique isolation key for the stream address, create an independent pipeline object for the current code, and simultaneously create a dedicated frame extraction subprocess, a mixing and pushing subprocess, and an application-layer priority audio queue. Execute the following six-dimensional physical isolation sub-steps in sequence: S31, Process Isolation: Each code has its own independent pipeline, frame extraction subprocess, mixed streaming subprocess, and independent worker thread; S32. Stream source isolation: The stream input address is fixed and concatenated as the media gateway base address / live / {code}, and each code stream link is independent of each other and does not interfere with each other; S33. Knowledge base isolation: An independent vector set kb_{code} is created for each code in the vector database. Physical isolation is used instead of global database filtering. It supports creating, deleting and rebuilding the knowledge base corresponding to a single code. S34. Persona Configuration Isolation: The persona table stores tenant-specific commentary style, title, catchphrases, banned words, and voice parameters with code as the primary key; S35, Isolation of Bullet Comment Sources: Each code is bound to a dedicated live stream collection address, and bullet comment data is routed to the corresponding pipeline according to the code; S36. Output target authentication isolation: The third-party pushUrl, streamKey and code issued when the broadcast starts are bound one by one, and each tenant's output push stream link is independently authenticated; S4. Intelligent output address splicing, broadcast initialization and barrage decoupling and distribution steps: Execute splicing adaptation logic on pushUrl and streamKey. If pushUrl does not end with " / " and streamKey does not start with " / " or "?", then automatically add the separator " / " between the two to generate a complete output push address that adapts to the third-party platform specification. The opening voice is synthesized based on the code's unique persona, sent to the mixed streaming subprocess and occupied by the playback timeline, and the current code is written into the on-play registration table; The distribution of the live stream address and the live stream start-up process are decoupled. After the pipeline is started, the upper management platform will distribute the live stream sharing text separately. The system will automatically extract the live stream link from the sharing text and start and stop the corresponding live stream collection task according to the code. S5, code precise shutdown and cache clearing steps: After the upper-level management platform calls the control plane stop shutdown interface, it immediately terminates the frame extraction subprocess corresponding to the current code; synthesizes shutdown voice, clears all backlogged historical narration audio cache in the mixing and streaming subprocess, and only retains the shutdown voice cache; after waiting for the shutdown voice to play completely and reserving a small amount of time margin, it forcibly destroys the mixing and streaming subprocess, removes the code from the on-play registration table, and simultaneously stops the bullet screen collection task corresponding to the code; S6. Watchdog thread periodically inspects and automatically eliminates zombie streams: Start an independent watchdog thread, which cyclically inspects all playing codes at a fixed 15-second interval. Each inspection queries the push activity flag publish.active of the corresponding code on the media gateway. If the query result indicates that there is active streaming, or if the gateway query fails, the source stream disappearance timer for this code will be reset to zero. If no active streaming is detected for code continuously, the duration of the source stream disappearance will be continuously accumulated. When the duration of continuous disappearance exceeds the jitter tolerance threshold of 180 seconds, the shutdown process of step S5 is automatically and completely executed, the resources of all sub-processes of frame extraction and mixing are recovered, and zombie streams are cleaned up.

2. The method for managing multi-tenant live streaming commentary pipelines using unique stream address keys according to claim 1, characterized in that, The control plane described in step S2 is a stateless, login-free, and session-free interface layer. It relies on the X-Api-Key platform-level key in the request header to complete global authentication. All operations, such as starting broadcast, stopping broadcast, querying live status, editing character settings, uploading vectorized knowledge base, configuring audio, and setting bullet screen addresses, carry a code and verify the code's ownership permissions. The control plane as a whole supports horizontal expansion.

3. The method for managing multi-tenant live streaming commentary pipelines using unique stream address keys according to claim 1, characterized in that, The media streaming action is completely decoupled from the pipeline's broadcast trigger. Only the upper-level management platform actively initiates broadcast start and stop control through the control plane interface. The media gateway's on_publish callback is only used for authentication and will not automatically start or stop the AI ​​narration pipeline.

4. The method for managing multi-tenant live streaming commentary pipelines using unique stream address keys according to claim 1, characterized in that, In step S33, the vector library kb_{code} is an independent physical index set. The knowledge base data corresponding to different codes are isolated from each other, and the retrieval does not interfere with each other. Deleting a single code knowledge base will not affect the knowledge bases of other tenants.

5. The method for managing multi-tenant live streaming commentary pipelines using unique stream address keys according to claim 1, characterized in that, Step S6: The watchdog logic is configured with a dual fault tolerance mechanism: if the gateway query fails, the source stream is determined to be online normally, and the disappearance time is not accumulated to prevent the monitoring component from malfunctioning and accidentally shutting down the normal live broadcast; the 180-second threshold is compatible with short-term network outages and reconnections to avoid pipeline destruction caused by network jitter.

6. A multi-tenant live streaming commentary pipeline management system with unique stream address keys, used to execute the multi-tenant live streaming commentary pipeline management method with unique stream address keys as described in any one of claims 1 to 5, characterized in that, The system is divided into three modules: access plane, control plane, and media and artificial intelligence plane. The stream address identifier code serves as the unique key for the stream address and is the unique tenant isolation key that runs through the entire link of the three modules. The access plane module is a media gateway unit, used to receive the original audio and video streams pushed by each tenant to the live / {code} path, configure the on_publish push stream authentication callback logic, only verify the legality of the code and the push stream credentials, and the authentication logic does not trigger the start and stop of the artificial intelligence commentary pipeline; The control plane module is a stateless REST interface unit with no login or session mechanism. It completes platform-level global authentication through the request header X-Api-Key and exposes interfaces for starting and stopping broadcasting, querying live broadcast status, configuring character settings, uploading / vectorizing knowledge base, configuring timbre, and configuring the address of the live broadcast room with the code to the outside world for the upper management platform to call and orchestrate. The media and artificial intelligence module is a stateful pipeline management unit, which includes a pipeline manager, several independent pipeline instances, a watchdog thread unit, a vector storage unit, and a relational storage unit. The pipeline manager is configured to create and destroy independent pipeline objects using code as the unique key index for the stream address. Each pipeline has a built-in dedicated frame extraction subprocess, a mixed streaming subprocess, and an application-layer priority audio queue. Each pipeline is only bound to the dedicated pull stream address, third-party output push stream address, and bullet screen collection task corresponding to the code. The vector storage unit allocates an independent physical vector set kb_{code} for each code, thereby achieving physical isolation of the multi-tenant knowledge base; The watchdog thread unit periodically traverses all playing codes every 15 seconds, queries the media gateway publish.active push activity flag, and automatically triggers the shutdown process to reclaim all pipeline subprocess resources and eliminate zombie streams that occupy resources for a long time when a code has no push duration exceeding the 180-second threshold.

7. The multi-tenant live streaming commentary pipeline management system with unique stream address keys according to claim 6, characterized in that, The control plane module is a stateless service cluster that supports horizontal scaling; the media and artificial intelligence plane module is a stateful node service, with the pipeline corresponding to a single code fixed to a single node. The system has a built-in code-node registration table to support distributed scaling deployment across multiple nodes.

8. The multi-tenant live streaming commentary pipeline management system with unique stream address keys according to claim 6, characterized in that, The control plane module's broadcast interface has a built-in intelligent output address splicing component. The splicing component executes the following logic: when the input pushUrl does not end with a separator " / " and the first character of streamKey is not " / " or "?", it automatically adds " / " between the two to splice the complete push stream address, which is compatible with the address format specifications of two types of third-party live streaming platforms.

9. The multi-tenant live streaming commentary pipeline management system with unique stream address keys according to claim 6, characterized in that, Each pipeline has a built-in shutdown buffer cleanup component; when a shutdown command corresponding to the code is received, the component clears all historical narration audio cache in the mixing and pushing subprocess, and only retains the real-time generated shutdown voice; after the shutdown voice has been played completely and the preset margin duration has been added, the mixing and pushing subprocess is forcibly destroyed to prevent the cached audio from being continuously pushed out after shutdown.

10. The multi-tenant live streaming commentary pipeline management system with unique stream address keys according to claim 6, characterized in that, The system is also configured with a bullet screen parsing and distribution subunit, which is bound to a code one by one; It supports delaying the reception of live room sharing text sent from the upper layer after the pipeline is created, automatically extracting the live room access link from the sharing text, starting and stopping the corresponding live room bullet screen collection task according to the code, and the bullet screen data is only routed to the corresponding code pipeline for artificial intelligence commentary and interaction.

Images