A distributed multi-tenant call recording archiving method and system based on dual-track channel separation and idempotent multi-stage processing pipeline

Abstract

The application discloses a distributed multi-tenant call recording archiving method and system based on double-track channel separation and idempotent multi-stage processing pipeline. The main and called parties are separated and recorded when the call is established, and a double-track original file is generated. After the call is ended, an archiving event is constructed and delivered to a message middleware. The archiving service performs idempotent check with a composite key of tenant code and call record segment identification. If not processed, it enters a multi-stage pipeline. The pipeline successively performs speaker activity detection, mixing, transcoding, waveform extraction, counter mode encryption, three-step uploading, metadata writing and temporary file cleaning, each stage being independently retried and idempotent. After success, consumption is confirmed and downstream is notified. Periodic scanning is performed according to the tenant retention period, and expired records are cascaded and soft deleted in batches through a distributed lock single instance. The application realizes distributed idempotent archiving, double-track speaker timeline accurate restoration, streaming encryption random access and multi-attribute user side hiding, and significantly improves archiving reliability and system scalability.

Description

Technical Field

[0001] This application relates to the fields of communication and computer software technology, specifically to the field of distributed multimedia processing technology in a cloud computing environment, and particularly to a distributed multi-tenant call recording archiving method and system based on dual-track channel separation and idempotent multi-stage processing pipeline. Background Technology

[0002] Call recording archiving is a fundamental requirement for call centers and enterprise communications. With the increasing prevalence of multi-tenant SaaS deployments, call recording archiving has evolved into a multi-stage, distributed problem encompassing collection, separation, transcoding, encryption, uploading, storage, retrieval, playback, and cleanup. Existing related technical solutions mainly fall into the following categories: Existing technology 1 is a traditional recording solution based on local disk recording on a media server. On a softswitch or media server (e.g., a FreeSWITCH-based server), a built-in recording module directly writes the encoded audio stream to the local disk after a call is established, resulting in a mono or mixed-channel audio file (e.g., WAV, MP3). The technical process is: call establishment → media server starts recording → media stream is written to the local file → file is closed after hanging up → file is moved to the archive directory via a scheduled task. The main advantages of this solution are its simple architecture, few dependencies, and the fact that the recorded files closely follow the media server's lifecycle. However, it has significant limitations: the local disk capacity of the media server is limited, making it difficult to handle long-term archiving by large-scale tenants; file moving and metadata maintenance are loosely coupled, easily leading to file remnants or inconsistencies between metadata and files; it lacks independent channel analysis capabilities, making it difficult to reconstruct the actual speaker's timeline; and it lacks an encryption mechanism or only uses static key full encryption, failing to support random access decryption.

[0003] Existing technology 2 is an offline recording post-processing solution based on full batch processing. It uses nighttime batch processing tasks to uniformly transcode, compress, store metadata, and upload to object storage for all recording files generated that day. The technical process is as follows: daily scheduled trigger → scan all files to be processed → perform transcoding / encryption / upload one by one → batch write metadata. Its main advantages are simplicity and centralized resource scheduling for batch processing. However, it has significant limitations: high processing latency, typically taking several hours from recording generation to becoming queryable and playable; complex recovery and potential for duplicate processing if the batch processing task is interrupted or fails; lack of fine-grained idempotency, making it difficult to handle the "partial success" state where some files fail to process and others succeed; and a lack of differentiated processing strategies configured by tenant.

[0004] Existing technology 3 is a recording and archiving scheme based on simple message-driven methods. After the call ends, the call detail record (CDR) module sends a message to the message middleware, and the archiving service receives the message and performs processing. The technical process is: CDR generation → message sending → archiving service consumption → single-function completion of download, transcoding, encryption, upload, and storage → message confirmation. The main feature is that it has a certain degree of asynchronicity, decoupling upstream business. However, the technical limitations are prominent: encapsulating the steps of downloading, transcoding, encryption, uploading, and storage in a single function means that failure in any step will cause the entire message to be redeployed, wasting computing power and amplifying the impact of failure; it lacks stage-level idempotency, and repeated processing may generate duplicate metadata or duplicate storage objects; there is no unified abstraction for context data transmission, making it difficult to extend to new processing stages; and it lacks the ability to automatically clean up intermediate products (temporary files), which are prone to accumulation and residue.

[0005] Existing technology 4 is a simple speaker recognition scheme based on mono-channel mixed recording. It records only the mixed mono audio on a media server, and then attempts to distinguish different speakers during post-processing using static acoustic features or clustering algorithms. The technical process is: record mono file → load the entire audio segment → cluster based on voiceprint features → output speaker segments. Its main advantages are small storage space usage and simple recording logic. However, it has serious limitations: the acoustic separation accuracy after mono-channel mixing is significantly lower than the original dual-track recording; it has poor robustness in dialect, low signal-to-noise ratio, and background noise scenarios; it is difficult to meet real-time and consistency requirements in large-scale calls; and it cannot be reliably associated with the calling and called numbers, making it unsuitable for direct use as compliance evidence.

[0006] Existing technology 5 is a recording storage scheme based on static full encryption. It encrypts the entire recording file using block ciphers (such as AES-CBC) or other block patterns. During playback, the entire file must be decrypted and cached locally before playback. The technical process is: recording ends → random key generation → block encryption of the entire file → ciphertext upload → downloading the ciphertext during playback → full decryption → local playback. Its main features are high encryption strength and relatively standard implementation. However, it has significant limitations: it does not support decryption at random positions, making it impossible to use HTTP range requests for simultaneous downloading and playback, and audio waveform dragging; the first playback requires a complete download and decryption, resulting in a poor user experience and wasted bandwidth; there is no unified key version management, making key rotation difficult and affecting long-term compliance; and it lacks the ability to manage keys in conjunction with multi-tenant configurations.

[0007] Existing technology 6 is a recording access control scheme based on a single attribution relationship. Each recording is bound to only one primary attribution user, and access is verified based on user identity. Deletion directly affects the recording itself. The technical process is: recording archiving → binding to primary attribution user → user query / deletion → affecting the entity record. The main feature is a simple data model. However, the technology has obvious limitations: it cannot adapt to the attribution requirements of a call center where a single call involves multiple business participants (such as the main agent, coordinating agents, supervisors, etc.); user-side deletion will affect management's evidence collection and quality inspection, resulting in high compliance risks; the visibility isolation granularity between tenants and users is too coarse, resulting in insufficient privacy protection; and it is difficult to support the business requirement that "user-side logical deletion does not affect management's auditing."

[0008] Based on the analysis of the existing technologies mentioned above, the following common pain points of existing call recording archiving technologies can be summarized: First, insufficient distributed idempotency makes it difficult to guarantee result consistency and data uniqueness in real distributed environments such as message duplication, process anomalies, and node crashes; Second, the dual-channel information is not fully utilized, limiting the accuracy of speaker timeline recognition and making it difficult to reliably reconstruct "who spoke and when" during the call, affecting quality inspection and customer insights; Third, most encryption mechanisms do not support random access and streaming while downloading, resulting in poor playback experience and bandwidth utilization, and lack of key versioning. Fourth, in multi-tenant and multi-user scenarios, the granularity of visibility control between tenants and users is too coarse, failing to meet the compliance requirements of "user-side self-hiding" and "long-term retention on the management side"; Fifth, lifecycle management is lacking or crude, with imprecise retention period control, automatic cleanup lacking distributed mutual exclusion, and lacking cascading consistency guarantees, leading to data bloat and multi-instance concurrency conflicts; Sixth, the processing chain is not scalable, adding new stages (such as automatic speech recognition, emotion recognition, quality inspection scoring, etc.) requires large-scale code modification, and there is a lack of stage-level retry and intermediate product lifecycle management capabilities.

[0009] Therefore, how to achieve idempotent and reliable recording archiving, accurate speaker timelines, encrypted storage supporting random access, fine-grained isolation across multiple affiliations, and scalable processing links in a distributed environment are technical problems that those skilled in the art need to solve. Summary of the Invention

[0010] This application provides a distributed multi-tenant call recording archiving method and system based on dual-track channel separation and idempotent multi-stage processing pipeline, which can solve the problems of insufficient distributed idempotency, underutilization of dual-track information, lack of support for random access in encrypted recordings, coarse multi-tenant ownership isolation, and lack of distributed consistency in lifecycle management in existing call recording archiving technologies.

[0011] The first aspect of this application provides a distributed multi-tenant call recording archiving method based on dual-track channel separation and idempotent multi-stage processing pipeline, including: When a call is established, the media server starts multi-channel recording, writes the caller's media stream to the left channel file and the called party's media stream to the right channel file, and stores the original dual-track recording file to shared storage after the call ends. After the call detail record (CDR) processing module identifies a call segment with recording, it constructs a recording archive event and delivers it to the message middleware. The archive event includes at least the tenant code, CDR segment identifier, original recording file path, calling and called numbers, call start time, and user affiliation list. The archive service consumes archived events, deserializes them into processing pipeline tasks, and uses a composite key consisting of the tenant code and call detail record segment identifier to query the metadata database for judgment. If the corresponding recording master record already exists, the consumption is confirmed and the current processing ends; otherwise, it enters the idempotent multi-stage processing pipeline. The idempotent multi-stage processing pipeline executes multiple processing stages in a fixed order. Each stage passes intermediate data through the pipeline context. Each stage has an independent retry mechanism and idempotency guarantee. The multiple processing stages include speaker activity detection, mixing, transcoding, waveform extraction, encryption, uploading, metadata writing, and temporary file cleanup. When any processing stage fails, determine whether it can be retried based on the error type. If it can be retried, repeat the stage according to the exponential backoff strategy; otherwise, terminate the processing pipeline and report the failure. After all stages are executed successfully, the consumption archiving event is confirmed, and an archiving completion notification is sent to the downstream modules. Based on the retention period configured by the tenant, calculate the expiration time of the retention period for each recording, periodically scan the expired records, and ensure that a single instance in the cluster performs cleanup through a distributed scheduling lock, and perform batch cascading soft deletion of the main recording record, the attribution record, and the speaker segment record.

[0012] Optionally, the idempotent multi-stage processing pipeline executes multiple processing stages sequentially in a fixed order, including: During the speaker activity detection phase, silence detection is performed on the left and right channel files respectively to obtain the active time sequence of the caller and the called party, and the detection results are stored in the pipeline context. The mixing stage mixes the left and right channels into a mono audio file and detects the audio duration; During the transcoding stage, the audio is transcoded according to the output format configured by the tenant; The waveform extraction stage extracts low-density waveform sampling data from the transcoded audio. During the encryption phase, a counter-mode block cipher is used to encrypt the audio file, and an initialization vector and key version number are written into it. The upload phase employs a three-step process: requesting a pre-signed certificate, PUT upload, and confirmation of completion to upload the encrypted file to object storage and obtain a resource identifier. During the metadata writing phase, the master recording is inserted with the composite key as the unique constraint, the attribution relationship record is written in batches, and the speaker segment record containing the speaker's time period and waveform data is written. The temporary file cleanup phase deletes all temporary files registered in the pipeline context.

[0013] Optionally, the speaker activity detection phase includes: The audio silence detection tool is invoked, the noise threshold and minimum silence duration are configured, the silence start time and duration output by the tool are parsed, the active time period is calculated in reverse, the active segment sequence of the left channel and the active segment sequence of the right channel are obtained, the left channel sequence is marked as the calling party and the right channel sequence is marked as the called party, and the marked active segment sequence is stored in the pipeline context.

[0014] Optionally, the mixing stage includes: When the right channel file is missing or its size is below a set threshold, it is determined to be a mono original recording, the mixing operation is skipped, and the left channel file is reused as the mixed output.

[0015] Optionally, the encryption phase employs a counter-mode block cipher to encrypt the audio file, including: A random initialization vector is generated and written to the header of the ciphertext file. Audio data is encrypted block by block, and the version number of the currently active key is recorded. During decryption, the counter adjustment value is calculated based on the starting byte offset of the request, and a key stream is generated at the corresponding position to support independent decryption of any byte range.

[0016] Optionally, the upload phase employs a three-step process: requesting a pre-signed certificate, PUT upload, and confirmation of completion to upload the encrypted file to object storage and obtain a resource identifier, including: Request a pre-signed upload certificate from the resource service to obtain the pre-signed URL and temporary resource identifier; Upload the encrypted file to the pre-signed URL using the HTTP PUT method; The confirmation interface of the resource service is called to convert the temporary resource identifier into a formal resource identifier and return it, and then the formal resource identifier is stored in the pipeline context.

[0017] Optionally, batch writing of attribution records includes: Batch write ownership relationship records in a one-to-many or many-to-many manner. Each ownership relationship record includes a tenant code, recording master record identifier, user identifier, and user-side visibility field. Establish a unique index based on the combination of tenant code, master record identifier, and user identifier; When a user requests to hide a specific recording, the user-side visibility field in the corresponding attribution record is set to an invisible value. When the user queries the recording, only the recording records corresponding to the attribution relationship with a valid user-side visibility field are returned.

[0018] The second aspect of this application provides a distributed multi-tenant call recording archiving system based on dual-track channel separation and idempotent multi-stage processing pipeline, including: The recording unit is used to start multi-channel recording by the media server when the call is established, write the calling party's media stream to the left channel file and the called party's media stream to the right channel file, and store the original dual-track recording file to shared storage after the call ends. The production unit is used to construct a recording archive event and deliver it to the message middleware after the call detail record processing module identifies a call segment with recording. The archive event includes at least the tenant code, call detail record segment identifier, original recording file path, calling and called numbers, call start time and user affiliation list. The verification unit is used to consume archived events from the archiving service, deserialize them into processing pipeline tasks, and use a composite key consisting of the tenant code and the call detail record segment identifier to query the metadata database for judgment. If the corresponding recording master record already exists, the consumption is confirmed and the current processing ends; otherwise, the process enters the idempotent multi-stage processing pipeline. The execution unit is used to execute multiple processing stages in a fixed order through an idempotent multi-stage processing pipeline. Each stage passes intermediate data through the pipeline context. Each stage has an independent retry mechanism and idempotency guarantee. The multiple processing stages include speaker activity detection stage, audio mixing stage, transcoding stage, waveform extraction stage, encryption stage, upload stage, metadata writing stage, and temporary file cleanup stage. The retry unit is used to determine whether a retry is possible when any processing stage fails. If a retry is possible, the stage is repeated according to the exponential backoff strategy; otherwise, the processing pipeline is terminated and a failure is reported. The notification unit is used to confirm the consumption archiving event after all stages have been successfully executed, and to send an archiving completion notification to the downstream modules. The cleanup unit is used to calculate the retention period expiration time of each recording based on the retention period configured by the tenant, periodically scan the expired records, and ensure that a single instance in the cluster performs cleanup through a distributed scheduling lock, and softly delete the main recording record, the attribution record, and the speaker segment record in batches.

[0019] A third aspect of this application provides a distributed multi-tenant call recording and archiving device based on dual-track channel separation and idempotent multi-stage processing pipeline, comprising: One or more processors; A memory on which one or more programs are stored; When the one or more programs are executed by the one or more processors, the one or more processors implement the distributed multi-tenant call recording archiving method based on dual-track channel separation and idempotent multi-stage processing pipeline as described in any of the preceding claims.

[0020] The fourth aspect of this application provides a computer storage medium for storing a program, which, when executed, implements the distributed multi-tenant call recording archiving method based on dual-track channel separation and idempotent multi-stage processing pipeline as described in any of the preceding claims.

[0021] Compared with the prior art, this application has the following beneficial effects: The invention significantly improves idempotency and fault recovery capabilities by eliminating duplicate message delivery and duplicate side effects caused by node failures through stage-level independent retries and database unique constraints. Some failures do not require redoing the already successful stages.

[0022] The reliability of the speaker's timeline is greatly improved. This invention preserves the dual-track audio channel from the source, and only requires silence detection to obtain the precise active time period at the millisecond level, which is directly associated with the calling and called numbers, without the need for complex voiceprint clustering.

[0023] The streaming playback experience is optimized. This invention uses AES-256-CTR counter mode encryption, supports random access for HTTP range requests, and the server only decrypts the request range, resulting in low latency for the first byte and high bandwidth utilization.

[0024] By decoupling compliance auditing from user privacy, this invention hides user-side logic through the visibility field of the attribution relationship table, while ensuring that the actual retention on the management side remains unaffected, thus balancing user self-control with compliant retention.

[0025] Distributed lifecycle management is secure and efficient. This invention uses distributed locks to ensure that cleanup tasks are executed on a single instance basis, and batch cascading soft deletions avoid large transactions impacting the database, ensuring eventual consistency across the three tables.

[0026] The processing link is scalable. The pipeline design of this invention facilitates the insertion of new stages (such as ASR and sentiment analysis), and temporary artifacts are automatically cleaned up, reducing the operational burden. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 A flowchart illustrating a distributed multi-tenant call recording archiving method based on dual-track channel separation and idempotent multi-stage processing pipeline provided in this application embodiment; Figure 2 A flowchart of an idempotent multi-stage processing pipeline is provided for embodiments of this application; Figure 3 A flowchart of AES-256-CTR random access decryption provided for embodiments of this application; Figure 4 An overall framework diagram of a distributed multi-tenant call recording and archiving system based on dual-track audio channel separation and idempotent multi-stage processing pipeline provided in this application embodiment; Figure 5 The module interaction timing diagram provided for the embodiments of this application; Figure 6 A schematic diagram of a distributed multi-tenant call recording archiving system based on dual-track audio channel separation and idempotent multi-stage processing pipeline provided in an embodiment of this application; Figure 7 This is a schematic diagram of the structure of a device provided in an embodiment of this application. Detailed Implementation

[0029] This application provides a distributed multi-tenant call recording archiving method and system based on dual-track channel separation and an idempotent multi-stage processing pipeline. It addresses issues in existing recording archiving schemes such as insufficient idempotency, low speaker timeline reliability, lack of support for random access playback of encrypted recordings, conflicts between compliance auditing and user privacy in multi-user ownership scenarios, and lack of distributed consistency in lifecycle management. The system serially executes speaker activity detection, mixing, transcoding, waveform generation, encryption, uploading, and metadata processing through an idempotent multi-stage processing pipeline. Idempotency is achieved based on database unique constraints on the composite key composed of tenant encoding and call detail record segment identifiers. AES-256-CTR streaming encryption is used, and random access decryption is achieved by adjusting the counter by byte offset. User-side visibility fields in a many-to-many ownership table support parallel compliance retention and user privacy. Lifecycle management is achieved through batch cascading soft deletion based on distributed scheduling locks. Compared to traditional solutions, archiving reliability, speaker timeline reliability, streaming playback experience, and tenant isolation granularity are significantly improved. It is suitable for multi-tenant call recording archiving scenarios such as cloud contact centers, online education, and financial compliance.

[0030] This invention is applicable to, but not limited to, the following scenarios: call recording archiving in SaaS-based cloud contact centers (CCaaS); compliance archiving and quality inspection backtracking in call centers; multi-tenant call logging in enterprise communication platforms (Unified Communications as a Service, UCaaS); two-way call evidence preservation in industries with strong compliance requirements such as finance, healthcare, and government; and multi-party call archiving in scenarios such as online education, telemedicine, and remote customer service. This invention can work in conjunction with Session Initiation Protocol (SIP) softswitch platforms (such as FreeSWITCH-based media servers), object storage systems, message middleware, and relational database systems, providing end-to-end capabilities for recording archiving, encrypted storage, on-demand decryption and playback, and lifecycle management for large-scale tenant and high-concurrency call scenarios.

[0031] See Figure 1 This figure is a flowchart illustrating a distributed multi-tenant call recording archiving method based on dual-track channel separation and idempotent multi-stage processing pipeline provided in an embodiment of this application. The distributed multi-tenant call recording archiving method based on dual-track channel separation and idempotent multi-stage processing pipeline provided in this embodiment can be implemented, for example, through the following steps S101-107.

[0032] S101: When a call is established, the media server starts multi-channel recording, writes the calling party's media stream to the left channel file, writes the called party's media stream to the right channel file, and stores the original dual-track recording file to shared storage after the call ends.

[0033] In this embodiment, the SIP-based softswitch media server initiates multi-channel recording upon call establishment, writing the calling party's media stream to the left channel file (`_left.wav`) and the called party's media stream to the right channel file (`_right.wav`), outputting two original recording files on shared storage. This provides independent channel material for subsequent dual-track analysis, avoiding uncertainties in acoustic clustering.

[0034] S102: After the call detail record (CDR) processing module identifies a call segment with recording, it constructs a recording archive event and delivers it to the message middleware.

[0035] In this application embodiment, the archived event includes at least the tenant code, call detail record segment identifier, original recording file path, calling and called numbers, call start time, and user affiliation list.

[0036] Specifically, the input call detail record (CDR) Leg entry event is processed. The CDR processing module identifies a Leg with recordings, constructs a recording archive event (RecordingArchiveEvent), including fields such as tenant code, CDR master identifier, CDR Leg identifier, call unique identifier, original recording path and filename, calling and called numbers, call type, scenario, user affiliation list, and start time, and sends it to the recording archive topic in the message middleware. The archive event in the message middleware is then output. This serves as a trigger signal to drive asynchronous consumption of the archiving service, decoupling the real-time call link from the archiving processing link.

[0037] S103: The archive service consumes archived events, deserializes them into processing pipeline tasks, and uses a composite key consisting of the tenant code and call detail record segment identifier to query the metadata database for judgment. If the corresponding recording master record already exists, the consumption is confirmed and the current processing ends; otherwise, it enters the idempotent multi-stage processing pipeline.

[0038] In this embodiment, the input message middleware delivers an archived event. The archived service consumes the event, deserializes it into a task structure, and injects it with tenant context and tracing context. It then queries the metadata database using (tenant_code, cdr_leg_id). If a valid record already exists, the message is directly acknowledged, and the processing pipeline task that passes the pre-check is output. This avoids duplicate consumption and redundant processing, providing a clear "new task" semantic for the processing pipeline.

[0039] S104: The idempotent multi-stage processing pipeline executes multiple processing stages in a fixed order. Each stage passes intermediate data through the pipeline context, and each stage has an independent retry mechanism and idempotency guarantee.

[0040] In this embodiment, multiple processing stages include a speaker activity detection stage, a mixing stage, a transcoding stage, a waveform extraction stage, an encryption stage, an upload stage, a metadata writing stage, and a temporary file cleanup stage. The speaker activity detection stage performs silence detection on the left and right channel files respectively, obtaining the active time period sequences of the caller and the called party, and stores the detection results in the pipeline context. The mixing stage mixes the left and right channels into a mono audio file and detects the audio duration. The transcoding stage transcodes the audio according to the output format configured by the tenant. The waveform extraction stage extracts low-density waveform sampling data from the transcoded audio. The encryption stage uses a counter-mode block cipher to encrypt the audio file, writing the initialization vector and key version number. The upload stage uses a three-step process of requesting a pre-signed certificate, PUT upload, and confirmation to upload the encrypted file to object storage and obtain a resource identifier. The metadata writing stage inserts the recording master record with a composite key as the unique constraint, batch writes the attribution relationship record, and writes speaker segment records containing speaker time periods and waveform data. The temporary file cleanup stage deletes all temporary files registered in the pipeline context.

[0041] The speaker activity detection phase includes calling the audio silence detection tool, configuring the noise threshold and minimum silence duration, parsing the silence start time and duration output by the tool, reverse-calculating the active time period, obtaining the left channel active segment sequence and the right channel active segment sequence, marking the left channel sequence as the calling party and the right channel sequence as the called party, and storing the marked active segment sequence into the pipeline context.

[0042] The mixing stage includes determining that if the right channel file does not exist or its size is below a set threshold, it is a mono original recording, the mixing operation is skipped, and the left channel file is reused as the mixed output.

[0043] The encryption phase employs a counter-mode block cipher to encrypt the audio file, including generating a random initialization vector and writing it into the header of the ciphertext file, encrypting the audio data block by block, and recording the currently active key version number. During decryption, the counter adjustment value is calculated based on the requested starting byte offset, and a key stream is generated at the corresponding position to support independent decryption of any byte range.

[0044] The upload phase employs a three-step process: requesting a pre-signed upload credential, PUT upload, and confirmation of completion to upload the encrypted file to object storage and obtain a resource identifier. This includes: requesting a pre-signed upload credential from the resource service to obtain a pre-signed URL and a temporary resource identifier; uploading the encrypted file to the pre-signed URL via the HTTP PUT method; and calling the resource service's confirmation interface to convert the temporary resource identifier into a formal resource identifier and return it, while storing the formal resource identifier in the pipeline context.

[0045] Batch writing of ownership relationship records includes: batch writing ownership relationship records in a one-to-many or many-to-many manner, with each ownership relationship record containing a tenant code, recording master record identifier, user identifier, and user-side visibility field; establishing a unique index on the combination of tenant code, recording master record identifier, and user identifier; when a user receives a request to hide a specified recording, setting the user-side visibility field in the corresponding ownership relationship record to an invisible value, and when the user queries the recording, only the recording records corresponding to the ownership relationship with a valid user-side visibility field value are returned.

[0046] Specifically, the implementation of pipeline execution is as follows: The input processing pipeline task is executed sequentially by the stage orchestrator, which executes the VAD stage, mixing stage, transcoding stage, waveform stage, encryption stage, upload stage, and metadata stage. Each stage is retried independently based on retryability. Finally, the encrypted ciphertext is output as the resource identifier in the object storage and the three-table records in the metadata database, providing addressable archived data for query and playback functions.

[0047] This application's embodiments organize post-recording processing into several pluggable stages executed sequentially in a preset order. Each stage is independently retryable, idempotent, and skippable. A pipeline context serves as the cross-stage data carrier. During failure retries, stage-level idempotency logic and database uniqueness constraints jointly ensure result uniqueness. Retryable errors employ exponential backoff. The reason for adopting this feature is that by dividing the entire process into appropriately granular stages and combining them with multi-layered idempotency, duplicate side effects can be avoided in distributed anomaly scenarios such as duplicate message delivery and partial stage failures, enabling rapid recovery from partial failures and significantly improving the system's engineering reliability and scalability.

[0048] The VAD stage and the mixing stage are implemented in a coordinated manner as follows: Input the path to the original dual-track (or single-track) recording file, and call the speaker activity detection tool to obtain the active time periods for the left and right channels respectively; if the right channel is missing, mark it as mono mode; call the mixing tool to generate mono audio for the dual track; detect the entire duration, and output the speaker active time period sequence, the path to the mixed mono audio file, and the duration, providing a unified input audio for the transcoding stage; and providing necessary data for the waveform and speaker segment stages.

[0049] This application embodiment performs speaker activity detection on both the calling and called channels to obtain active time periods, and then generates a mono playback file through a mixer; when only the original mono file exists, the mixing is adaptively skipped; the detection results are labeled as calling or called by channel and directly aligned with the number field to form a verifiable speaker timeline. The reason for adopting this feature is that, compared with post-event acoustic separation from mono mixed recordings, channel-separated VAD has significant advantages in accuracy, robustness, and implementation complexity, especially in low signal-to-noise ratio and multi-party interruption scenarios, and can be directly associated with the number field required for compliant evidence collection.

[0050] The implementation of the coordination between the transcoding stage and the waveform stage is as follows: Input the mixed mono audio file and the tenant's output format configuration. The output format (WAV or MP3) is determined according to the tenant configuration. Variable bit rate encoding (e.g., libmp3lame, qscale=2) is used for MP3 scenes. Waveform sampling is performed on the transcoding or mixing results to obtain a low-density amplitude sequence. The final playback file path, final format marker, and waveform samples are output to provide the final distributable file for the encryption stage and fragment data for the metadata stage.

[0051] This application embodiment stores both speaker time segments (JSONB) and waveform samples (BYTEA) in the speaker segment table, mapping them one-to-one to the main recording. The front end can complete the timeline, waveform, and speaker highlight rendering with a single retrieval. The reason for adopting this feature is to reduce the round-trip overhead of front-end rendering and avoid the latency and computational waste caused by performing the waveform extraction process in real time during playback.

[0052] The encryption phase is implemented as follows: Input the final playback file and the current activation key version, generate a 16-byte random IV and write it to the file header, encrypt it block by block in AES-256-CTR mode; record the key version number, and output the encrypted ciphertext file and key version number to provide the ciphertext file for the upload stage.

[0053] This application uses AES-256-CTR mode to encrypt recording files, with a 16-byte random IV written to the file header. During decryption, an adjusted counter is calculated based on byte offsets to enable decryption at any position. The encryption_key_version metadata field supports the coexistence of multiple key versions and key rotation. The reason for adopting this feature is that CTR mode inherently supports random access, which can be used in conjunction with HTTP range requests to achieve streaming while downloading; key versioning provides a key rotation channel for long-term compliant operation without interrupting the readability of historical recordings.

[0054] The upload phase is implemented as follows: Input the encrypted ciphertext file and task metadata, request a pre-signed PUT credential from the resource service, upload it using the HTTP PUT method, call the confirmation interface, and output the resource identifier (ResourceID) to provide resource addressing for the metadata stage.

[0055] This application embodiment divides the upload process into three steps: "application certificate → PUT upload → confirmation completion," decoupling it from the resource service and decentralizing the storage policy to the resource service. Any failure at any step can be retried without introducing duplicate data visible to the user. The reason for adopting this feature is to: reduce the coupling of the archiving service to the underlying object storage implementation, facilitating support for multiple storage backends (cloud object storage, self-built object storage, distributed file system); and to facilitate centralized management of tenant quotas and storage policies.

[0056] The metadata phase is implemented as follows: Input all metadata in the pipeline context and insert it into the main table with a unique constraint of (tenant_code, cdr_leg_id); batch insert the attribution records with a unique constraint of (tenant_code, recording_file_id, user_id); insert speaker and waveform segment records, and output the metadata records of the three tables to provide searchable data for the cleanup phase and query module.

[0057] This application uses an ownership table (tb_recording_ownership) to express many-to-many user ownership relationships; the visible field is used to independently control user visibility, ensuring that user hiding behavior does not affect compliance evidence collection and quality inspection on the management side. The reason for adopting this feature is that a call in a call center often involves multiple participants, which a single ownership scheme cannot represent; at the same time, in compliance scenarios, the management side must retain complete audit information, while users expect a certain degree of autonomy over their personal recordings. By introducing a two-level isolation of "user-side logical visibility" and "management-side actual retention," both needs can be met simultaneously.

[0058] In this embodiment, the transcoding format and retention period for the processing pipeline can be obtained on demand from the tenant configuration service; within a stage, execution is based on the configured branch (e.g., skipping MP3 transcoding or adjusting VBR quality parameters). This feature is adopted because different tenants have significantly different requirements for compliance, storage costs, and audio quality. Differentiated processing is achieved through tenant-level configuration, avoiding a "one-size-fits-all" approach; it also naturally aligns with the SaaS-based multi-tenant operation model.

[0059] The cleanup and notification phases are implemented as follows: Input a list of temporary files in the pipeline context, delete all registered temporary files; push archiving completion notifications to downstream modules (e.g., for consumption by the quality inspection module or notification module), output a temporary artifact cleanup completion event to release disk resources; drive downstream business.

[0060] S105: When any processing stage fails, determine whether it can be retried based on the error type. If it can be retried, repeat the stage according to the exponential backoff strategy; otherwise, terminate the processing pipeline and report the failure.

[0061] S106: After all stages are executed successfully, confirm the consumption archiving event and send an archiving completion notification to the downstream module.

[0062] S107: Calculate the retention period expiration time for each recording based on the retention period configured by the tenant, periodically scan the expired records, and ensure that a single instance in the cluster performs cleanup through a distributed scheduling lock, and perform batch cascading soft deletion of the main recording record, the attribution record, and the speaker segment record.

[0063] In this embodiment, based on the master recording record in the metadata database, the distributed scheduling lock is preempted in real time; soft deletion is performed in batches according to the retention period; asynchronous deletion is initiated on the object storage side when necessary (handled by the resource service), and an expired data deletion completion event is output, thereby forming an end-to-end closed loop and keeping the storage cost within a controllable range in the long term.

[0064] This application's embodiments control cleanup with a retention period expiration time of microsecond precision (retention_expire_at=start_time+retention_days×86400000000); ensure single-instance execution in a multi-instance environment with a distributed cache scheduling lock; and protect database load through batch cascading soft deletion. The reason for adopting these features is that: multi-instance operation under high-availability deployment must avoid duplicate cleanup; large transactions can impact online queries; and cascading consistency can prevent dirty data residue.

[0065] In one implementation of this application, existing technical solutions (such as the single-function solution of prior art 3 and the CBC full encryption solution of prior art 5) and the technical solution of this invention are deployed on the same call sample set respectively. In scenarios such as simulated message duplicate delivery, node crash, and high-concurrency access, indicators such as duplicate data rate, first byte delay, and jitter cleanup are statistically analyzed. At the same time, independent reviewers compare the accuracy differences between dual-track VAD and monophonic voiceprint clustering schemes on the speaker timeline.

[0066] Now combined Figure 4 illustrate, Figure 4This is a general framework diagram of a distributed multi-tenant call recording and archiving system based on dual-track channel separation and idempotent multi-stage processing pipeline, provided in an embodiment of this application. The technical solution provided in this embodiment consists of two collaborative layers: hardware architecture and software architecture.

[0067] The hardware architecture consists of the following: Media service layer: This includes several softswitch media servers (such as FreeSWITCH-based media servers) based on Session Initiation Protocol (SIP), which are responsible for call connection, media stream processing, and generation of raw dual-track recording files; each media server mounts a shared storage volume (such as a distributed file system or network file system) to write the raw dual-track recording files to a predetermined path.

[0068] Shared storage layer: This includes network-mounted shared disks or distributed file systems used to transfer raw audio files between media servers and archiving servers.

[0069] The archiving layer consists of several archiving servers (deployed in a containerized manner), which run recording archiving services and cleanup scheduling services. Each server can independently consume messages and execute processing pipelines, and has horizontal scalability.

[0070] Message middleware layer: including message queue clusters (such as RocketMQ clusters), providing at least-once delivery semantics, and supporting consumer groups and dead-letter queues.

[0071] Caching and locking: This includes distributed caching clusters (such as Redis clusters) that provide distributed scheduling locks and state synchronization capabilities.

[0072] Object storage layer: includes object storage cluster or object storage gateway service, supports pre-signed upload credentials (presigned URL) and download by byte range.

[0073] Relational database layer: This includes a relational database cluster (such as a PostgreSQL cluster) that stores recording metadata, attribution relationships, speaker segments, and waveform data.

[0074] Telephone control layer: Includes telephone control services, providing the ability to initiate outbound calls and play specified audio files on designated extensions.

[0075] The software architecture is divided into the following modules according to functional responsibilities: Recording event production module: When the call detail record (CDR) processing module detects a call leg with recording, it generates a recording archive event and sends it to the specified topic in the message middleware.

[0076] Idempotent message consumption module: Receives recording and archiving events from the message middleware, performs idempotency pre-checks, and converts the events into processing pipeline tasks.

[0077] Idempotent multi-stage processing pipeline module: It consists of a stage orchestrator and several pluggable processing stages. Each stage is executed sequentially in a preset order. Each stage can be independently retried, idempotent, or skipped.

[0078] Dual-track audio analysis and mixing module: including speaker activity detection (VAD) unit, channel mixing unit, mono adaptive unit, and duration detection unit.

[0079] Waveform sampling and speaker segment storage module: Synchronously stores waveform sampling and speaker time segment data for front-end rendering.

[0080] Streaming encryption and key versioning module: It uses AES-256-CTR mode for encryption, writes the initialization vector (IV) into the file header, and records the key version number; during decryption, it looks up the key based on the version number and supports access to random byte positions.

[0081] The upload credential application and confirmation module uses a three-step process (application for credential → PUT upload → confirmation completion) to interact with object storage and obtain the resource identifier (ResourceID).

[0082] Metadata and attribution storage module: Stores the main recording record, attribution record, speaker and waveform segment record in a three-table structure with one-to-many, many-to-many, and one-to-one relationships.

[0083] The query and streaming playback module provides capabilities such as list query, detail query, streaming download (supports HTTP range requests), batch download (ZIP packaging), playback on multiple devices, and hiding user-side logic.

[0084] Lifecycle management module: Based on microsecond-precision retention period timestamps and distributed scheduling locks, it performs batch cascading soft deletion of expired recordings.

[0085] Tenant configuration module: Provides tenant-level configuration (such as output audio format, retention period days, etc.) for each stage of the processing pipeline to be executed according to tenant differences.

[0086] The following is combined with Figure 5 A detailed description of each core module of the present invention is provided. Figure 5 The module interaction timing diagram provided in the embodiments of this application illustrates the functional positioning, implementation mechanism and specific implementation of each module.

[0087] The idempotent message consumption module receives recording archive events from the message middleware, performs deserialization, tenant context injection, and tracing context injection, and then transforms the events into processing pipeline tasks. An idempotency pre-check is performed before entering the processing pipeline to avoid side effects from duplicate consumption. The input to this module is the recording archive event message delivered by the message middleware; the output is the processing pipeline task that has passed the pre-check.

[0088] The implementation mechanism is as follows: Subscribe to the recording archive topic (e.g., topic_recording_archive) of the message middleware and consume it concurrently using the consumer group pattern.

[0089] Perform two layers of retries: short retries at the message processing layer (e.g., up to 2 times, with a backoff of 500 milliseconds or 2 seconds) to cover momentary jitter; and stage-level retries within the processing pipeline to cover single-stage recoverable errors.

[0090] When deserialization fails, an ACK is sent directly to avoid an infinite loop, and an alarm log is recorded.

[0091] Idempotency pre-check: Query the metadata database using a composite key consisting of the tenant code and the call segment identifier (tenant_code + cdr_leg_id). If a valid record already exists, directly confirm the message and return success.

[0092] The specific implementation is as follows: Deserialization: The JSON-encoded event body is deserialized into a domain task structure (RecordingArchiveTask), which includes fields such as tenant code, call detail record (CDR) master identifier, CDR Leg identifier, call unique identifier, original recording path, file name, calling and called numbers, call scenario, start time, and user affiliation list.

[0093] Pre-check: Call the ExistsByCDRLeg method of the metadata repository to determine whether it already exists based on the database unique constraint and query; if it exists, skip the processing pipeline and record the idempotency hit index.

[0094] Idempotent multi-stage processing pipeline module: See Figure 2 , Figure 2This application provides an idempotent multi-stage processing pipeline flowchart. The idempotent multi-stage processing pipeline module executes several pluggable stages sequentially using a stage orchestrator to realize the entire post-recording processing flow: speaker activity detection, channel mixing, tenant-configured transcoding, waveform extraction, encryption, uploading, metadata writing, temporary file cleanup, and downstream notification. Each stage independently possesses retryability, idempotency, and skippability capabilities. The input of this module is the processing pipeline task that has passed the pre-check; the output is the archived recording metadata (master record, attribution, speaker segment) and the encrypted ciphertext file in object storage.

[0095] The implementation mechanism is as follows: The stage orchestrator uses a composite mode, holding a set of stages in a fixed order; each execution proceeds sequentially starting from the first stage.

[0096] Phased retry: For retryable error categories, retry a predetermined number of times (e.g., 3 times) with exponential backoff (e.g., 1 second, 2 seconds, 4 seconds); for non-retryable error categories (e.g., parameter error, idempotent hit), terminate directly.

[0097] The PipelineContext spans all stages and includes fields such as task parameters, intermediate output path list, channel active segment results, mixing output path, transcoding output path, waveform sampling results, encrypted output path and key version number, object storage resource identifier, and final file format.

[0098] Temporary artifact registration: Temporary files generated in each stage are registered in the pipeline context's temporary file list and deleted during the cleanup stage; Abnormal paths can also be scanned and cleared during the cleanup stage before retries.

[0099] The specific implementation is as follows: Phase sequence: VAD phase → Mixing phase → Transcoding phase → Waveform extraction phase → Encryption phase → Upload phase → Metadata phase → Cleanup phase → Notification phase.

[0100] Retry implementation: The stage orchestrator records the number of attempts before each stage call. If a failure occurs and the error is determined to be retryable, the stage will sleep for a backoff time and then be called again until success or the number of attempts is exhausted.

[0101] Idempotency implementation: The metadata phase relies on database unique constraints (e.g., uk_tenant_leg) to ensure write uniqueness; the resource identifier in the upload phase avoids duplicate occupation through the idempotent interface of the resource service; the temporary file name includes the task's unique identifier to prevent concurrent conflicts.

[0102] The dual-track audio analysis and mixing module performs speaker activity detection on the original audio from both the calling party (left channel) and the called party (right channel), obtaining a millisecond-level active time segment sequence. It then mixes the dual-track audio to generate a mono playback file; adaptively skips mixing when only the original mono recording exists, directly reusing the main channel; and provides the entire audio duration as a metadata field. The module's input is the path to the original dual-track (or single-track) recording file; the output includes two sets of speaker active time segments (main speaker and peer speaker), the path to the mixed mono audio file, and the entire audio duration.

[0103] The implementation mechanism is as follows: Speaker activity detection: Using audio processing tools (such as the FFmpeg silencedetect filter), configure the noise threshold (e.g., -30 dB) and the minimum silence duration (e.g., 0.3 seconds), and analyze the silence start time and silence duration in the output log of the parsing tool to infer the active time period.

[0104] Channel mixing: Using audio processing tools (such as the FFmpeg amix filter), the audio is aligned according to the longest duration (duration=longest) and output as a mono; in the case of original mono recording, the main channel file is directly used as the logical output of the mixing result.

[0105] Mono Adaptive: Determines whether dual tracks exist by detecting the existence of the right channel file and its size threshold; if not, the stage skips the mixing process but still maintains consistent context fields.

[0106] Duration detection: Using audio processing tools (such as FFprobe), the audio container metadata is parsed to obtain the duration in seconds, and then used in subsequent time calculations with microsecond precision.

[0107] The specific implementation is as follows: Audio channel file naming convention: In the original recording path, they are named '{basename}_left.{ext}' and '{basename}_right.{ext}' respectively; if they do not exist, they degenerate into '{basename}.{ext}'.

[0108] Active segment data structure: Represented by a two-dimensional integer array, each item is [start_ms, end_ms], representing an active time period, and sorted in ascending order by start time.

[0109] Speaker role labeling: The left channel detection result is labeled as "caller" and the right channel is labeled as "callee" to facilitate alignment with the number field and form a verifiable speaker timeline.

[0110] Waveform Sampling and Speaker Segment Storage Module: Extracts low-density waveform samples from the mixed mono audio for front-end timeline visualization; stores speaker active time periods and waveform samples in a unified manner, supporting one-time retrieval and rendering by the front-end. The input of this module is the mixed mono audio file; a set of speaker active time periods; the output is the speaker segment record (including waveform binary) of the corresponding master recording.

[0111] The implementation mechanism is as follows: Waveform sampling: Audio is sampled at a low resolution normalization density (e.g., 100 floating-point amplitude samples per second), and the resulting floating-point sequence is compressed and encoded and stored in the database in the form of a binary large object (BYTEA).

[0112] Joint storage: Speaker segment records and main recording records have a one-to-one relationship. Speaker time segments are stored as JSON binary objects (JSONB) fields, and waveform samples are stored as BYTEA fields. The two coexist to reduce round-trip delay.

[0113] The specific implementation is as follows: Waveform normalization: The amplitude is mapped to the range [-1.0, 1.0] and then encoded as a 32-bit floating-point sequence, and if necessary, represented in compact binary (e.g., 16-bit fixed-point or 8-bit quantization) to save storage.

[0114] Front-end rendering: The front-end can complete the rendering of timeline, waveform and speaker highlighting by pulling speaker fragment records from the server once.

[0115] Streaming encryption and key versioning module: See Figure 3 , Figure 3 This application provides an AES-256-CTR random access decryption flowchart, which encrypts mixed or transcoded audio files using a block cipher counter mode (AES-256-CTR) and outputs a ciphertext file with a [IV|ciphertext] structure. It supports key versioning management, recording the key version number after each encryption and searching for the key by version number during decryption. It also supports streaming decryption starting at any byte offset to facilitate random access playback in conjunction with HTTP range requests. The module's inputs are the plaintext audio file to be encrypted and the currently active key version number; the outputs are the encrypted ciphertext file path and the key version number; and during decryption, it outputs the decrypted data stream located at the specified offset.

[0116] The implementation mechanism is as follows: Encryption process: A 16-byte random initialization vector is generated and written to the header of the target file, and then plaintext is encrypted block by block in AES-256-CTR mode and written to the file body.

[0117] Key version: The system maintains a mapping table from version number to key; encryption retrieves the currently active version, and the metadata field records the version number.

[0118] Random access decryption: Given the starting byte offset, the counter field is offset to 'IV+block_index' by calculating 'block_index=offset / 16' and 'inner_offset=offset%16'. After discarding the 'inner_offset' byte, the plaintext stream is generated from the offset position.

[0119] The specific implementation is as follows: IV adjustment algorithm: Treat IV as a big-endian 128-bit integer and add it to block_index; handle overflow modulo 2^128 to maintain consistency with CTR counter semantics.

[0120] Decrypt the stream: Expose a readable data stream to the upper layer with 'io.Reader' semantics, and the upper layer can stop reading at any point; on the server side, write the data stream directly into the HTTP response body to achieve download-while-playing.

[0121] Key rotation: Adding a new key version does not affect the readability of historical ciphertext; before discarding an old version, an offline migration (decrypting line by line - re-encrypting - updating the version number) must be performed.

[0122] The three-step upload credential application and confirmation module interacts with object storage through a three-stage process: "Request credential → PUT upload → Confirm completion," reducing the coupling between the archiving service and object storage. It abstracts the resource addressing information in object storage as a resource identifier (ResourceID) and manages it through the resource service. The module's inputs are the encrypted ciphertext file path and task metadata; the output is the resource identifier (ResourceID).

[0123] The implementation mechanism is as follows: Step 1: Call the GenerateUploadCredential interface to the resource service to request a pre-signed PUT URL and corresponding credential information (including temporary resource identifier and optional HTTP header signature).

[0124] Step 2: The archiving service uses the HTTP PUT method to upload the encrypted file to the pre-signed URL; the transmission process supports checksum comparison (such as MD5 or SHA-256).

[0125] Step 3: Call the ConfirmUpload interface to the resource service to confirm successful upload; upon confirmation, the resource service will convert the temporary resource identifier into a formal resource identifier and return it.

[0126] The specific implementation is as follows: This three-step process allows the resource service to verify tenant quotas and storage policies (such as cross-region replication, archive level, and encrypted storage policies) in the first step; and to verify file integrity and tenant status in the third step; thereby pushing storage policies from the archive service to the resource service for unified governance.

[0127] Failure recovery: Any step can be retried if it fails; since the uploaded object and metadata are not written to the main database, retry will not generate duplicate data visible to the user.

[0128] Decoupling effect: The archiving service is completely unaware of whether the underlying object storage is built locally or in the cloud, or whether it is a single machine or a sharded cluster.

[0129] Metadata and Attribution Storage Module: This module stores the main recording record, attribution record, and speaker and waveform segment records in separate tables, achieving separation of concerns and query optimization; distributed idempotency is ensured through database unique constraints. The input to this module is the metadata in the processing pipeline context; the output is the records from the three tables.

[0130] The implementation mechanism is as follows: The main recording table (tb_recording_file) stores the immutable attributes of each recording (such as tenant code, call detail record identifier, call identifier, file name, format, duration, size, calling and called numbers, call type, scenario, start time, retention period expiration time, encryption algorithm, key version number, and resource identifier); a partially unique index is created on (tenant_code, cdr_leg_id) (for records that have not been deleted) to achieve idempotency.

[0131] The ownership table (tb_recording_ownership) expresses a scenario where a recording belongs to multiple users in a many-to-many structure; the fields include tenant code, recording master record identifier, user identifier, and user-side visibility; a unique index is created on (tenant_code, recording_file_id, user_id).

[0132] Speaker segment table (tb_recording_segment): One-to-one correspondence with the main recording record, storing speaker time segments (JSONB) and waveform samples (BYTEA).

[0133] The specific implementation is as follows: Soft deletion: The logical deletion status is expressed in all three tables using the deleted_at field; the query filters for deleted items by default; cascading soft deletions ensure synchronization of the three tables.

[0134] User-side logic hiding: The visible field of the attribution table allows users to hide recordings without affecting the management side.

[0135] Composite index optimization: Optimize list queries by creating an index on (tenant_code, start_time DESC), optimize clean scans by creating an index on (retention_expire_at), and optimize client "My Recordings" queries by creating an index on (user_id, visible).

[0136] The query and streaming playback module provides two sets of query interfaces: one for the management end and one for the client end, corresponding to compliance and evidence collection scenarios and personal query scenarios, respectively. It offers diverse playback capabilities such as streaming download, batch download, and playback on multiple devices. The module's input is a query request or playback request (including tenant context, user context, and byte range request headers); the output is a paginated result set, streaming audio data, or a playback receipt from the telephone side.

[0137] The implementation mechanism is as follows: Management query: Index scan by (tenant code, start time descending order), supports multi-dimensional filtering and whitelist sorting fields such as calling and called numbers, time range, call type, and scenario.

[0138] Client query: Perform an index scan on the attribution table by (user ID, visibility=1) and join the table to query the data in the main table.

[0139] Streaming download: Receives HTTP range requests, locates the request start offset of the encrypted file, and uses the key versioning decryption module to generate a decrypted data stream that is directly written to the response body.

[0140] Batch download: Packages files into a ZIP stream according to a batch limit (e.g., 100 files); failure of a single file will not block other files.

[0141] Extension playback: Decrypt the target recording to a temporary directory, call the telephone control service to initiate an outbound call on the specified extension and play the audio; temporary files are cleaned up by the temporary file cleanup subtask of the lifecycle management module according to the modification time.

[0142] The specific implementation is as follows: Range request compatibility: Supports 'Range: bytes=START-END' semantics. The server adjusts the IV counter by the START offset, writes the response body by END-START+1 bytes, and returns a 206 Partial Content response code.

[0143] Temporary file cleanup for extension playback: Instead of deleting immediately after returning from Originate due to a delay in reading by the telephone system, it is now performed in batches in the background based on file modification time exceeding a threshold (e.g., 10 minutes).

[0144] The lifecycle management module performs batch cascading soft deletion of expired recordings based on their retention period expiration time. It uses a distributed scheduling lock to ensure that cleanup is performed on a single instance when multiple instances are deployed, avoiding conflicts and duplication. The input to this module is the master recording record in the metadata database; the output is the soft deletion results of expired recordings along with their corresponding cascading attribution and segment records.

[0145] The implementation mechanism is as follows: Distributed scheduling lock: Implemented in a distributed cache with a fixed key (e.g., 'novo-commctl-report:recording:cleanup:scheduler') and a time-to-live (TTL) of 30 minutes; a universally unique identifier (UUID) is used as the lock holder identifier, and atomic verification is performed upon release.

[0146] Periodic scanning: Triggered at a fixed time every day (e.g., early morning off-peak period), skipping the current round if the lock-grabbing fails.

[0147] Batch cascading: Scan the main table by retention_expire_at < current time, and perform cascading soft deletion in each batch (e.g., 100 records); protect the database with a single round limit (e.g., 10,000 records).

[0148] Cascading order: First, soft delete the main table, then soft delete the belonging table and fragment table according to the main table identifier, ensuring that the three tables are eventually consistent.

[0149] The specific implementation is as follows: Retention period precision: The retention period is stored with microsecond precision (retention_expire_at is a microsecond-level timestamp), and uses the same precision as the start time (start_time) to facilitate accurate calculation of retention_expire_at = start_time + retention_days × 86400000000.

[0150] Tenant Differentiation: The retention period in days is obtained from the tenant configuration, supporting tenant-level differentiation strategies.

[0151] Failure tolerance: If a single batch fails, the transactions in that batch are rolled back, and the next batch continues; failure indicators are reported to the monitoring system.

[0152] Compared with the prior art, the present invention has the following beneficial effects: This invention significantly improves distributed idempotency and fault recovery capabilities. It employs an idempotent multi-stage processing pipeline, combined with database unique constraints (e.g., uk_tenant_leg) and idempotency pre-checks. Even if message middleware experiences duplicate delivery or node failures leading to stage-level retries, duplicate metadata and stored objects will not be generated. Compared to traditional solutions that encapsulate the entire process into a single function, this invention eliminates the need to redo successful stages in the event of a partial failure, saving processing resources and eliminating the side effects of duplicate writes. The "double insurance" of stage-level idempotency and database unique constraints moves the eventual consistency constraint forward to the write side in a distributed environment, aligning with BASE theory and distributed idempotent design principles.

[0153] The usability and reliability of the speaker timeline are significantly improved. This invention employs a dual-track separated VAD and directly aligns the detection results with the number field. The speaker time period no longer relies on post-event acoustic clustering, and the resulting timeline can be directly consumed by the business. Compared to the mono-channel hybrid recording + voiceprint clustering scheme, this invention has higher robustness to complex scenarios such as low signal-to-noise ratio, dialects, and multiple interruptions, and eliminates the complex voiceprint modeling process. Speaker information (left and right channels) is preserved from the physical acquisition layer, and an accurate timeline can be obtained by simply performing VAD in post-processing. Preserving information from the source is significantly better than post-event reconstruction.

[0154] The playback experience and bandwidth utilization are significantly improved. This invention employs AES-256-CTR random access encryption, allowing the server to decrypt only the byte range required for the response request, while the client can download and play simultaneously, and drag to locate specific bytes. Compared to the static AES-CBC full encryption scheme, this invention eliminates the need for a complete download, significantly reducing first-byte latency and improving bandwidth utilization. CTR mode is a stream cipher, where each byte's keystream is uniquely determined by the IV and a counter, allowing for independent keystream generation at any byte position to complete decryption.

[0155] The conflict between compliance evidence collection and user privacy control is resolved by employing multi-user attribution and user-side logical visibility control. User-side "hide / delete" actions only affect the visibility field and do not impact the actual retention and compliance auditing on the management side. Compared to single-attribution schemes, this invention simultaneously supports multi-user collaboration and hierarchical visibility control, ensuring that compliance auditing is not compromised by user hiding. A hierarchical visibility model is constructed by separating the three states of "physical deletion," "management-side logical deletion," and "user-side logical hiding."

[0156] The accuracy and security of lifecycle management are significantly improved. This invention employs distributed scheduling locks and batch cascading soft deletes, preventing duplicate cleanup in multi-instance deployments. Database load remains stable during the cleanup window, and the three tables remain consistently cascaded. Compared to single-instance scripts and one-time large transaction solutions, this invention can be seamlessly migrated to high-availability deployments and significantly reduces the impact on online query jitter. Distributed mutual exclusion is a necessary condition for high-availability deployment; batch transactions are a common means of database load management; and cascading soft deletes are a necessary mechanism for ensuring multi-table consistency.

[0157] The scalability of the processing link and the ability to manage storage costs are significantly enhanced. This invention adopts a three-step upload process, integrated storage of waveforms and speaker segments, and tenant-level differentiated configuration. The system can smoothly connect to different object storage backends, reduce front-end round trips, and optimize storage costs on a tenant-by-tenant basis. Compared with tightly coupled storage solutions and generalized configuration solutions, this invention demonstrates better portability, manageability, and cost control capabilities in actual SaaS operations. Decoupling is a universal principle of software engineering; integrated storage is an effective means of reducing end-to-end latency; and tenant-level configuration is a core capability of SaaS operations.

[0158] Based on the methods provided in the above embodiments, this application also provides a distributed multi-tenant call recording archiving system based on dual-track channel separation and idempotent multi-stage processing pipeline. The following describes the distributed multi-tenant call recording archiving system based on dual-track channel separation and idempotent multi-stage processing pipeline with reference to the accompanying drawings.

[0159] See Figure 6 The figure is a schematic diagram of the structure of a distributed multi-tenant call recording archiving system based on dual-track channel separation and idempotent multi-stage processing pipeline provided in an embodiment of this application.

[0160] The distributed multi-tenant call recording and archiving system 600 based on dual-track channel separation and idempotent multi-stage processing pipeline provided in this application includes: a recording unit 601, a production unit 602, a verification unit 603, an execution unit 604, a retry unit 605, a notification unit 606, and a cleanup unit 607.

[0161] The recording unit 601 is used to start multi-channel recording by the media server when the call is established, write the calling party's media stream to the left channel file and the called party's media stream to the right channel file, and store the original dual-track recording file to the shared storage after the call ends. Production unit 602 is used to construct a recording archive event and deliver it to the message middleware after the call detail record processing module identifies a call segment with recording. The archive event includes at least the tenant code, call detail record segment identifier, original recording file path, calling and called numbers, call start time and user affiliation list. The verification unit 603 is used to consume archived events from the archiving service, deserialize them into processing pipeline tasks, and use a composite key consisting of the tenant code and the call detail record segment identifier to query the metadata database for judgment. If the corresponding recording master record already exists, the consumption is confirmed and the current processing ends; otherwise, the process enters the idempotent multi-stage processing pipeline. The execution unit 604 is used to execute multiple processing stages in a fixed order through an idempotent multi-stage processing pipeline. Each stage passes intermediate data through the pipeline context. Each stage has an independent retry mechanism and idempotency guarantee. The multiple processing stages include speaker activity detection stage, mixing stage, transcoding stage, waveform extraction stage, encryption stage, upload stage, metadata writing stage, and temporary file cleanup stage. The retry unit 605 is used to determine whether a retry is possible based on the error type when any processing stage fails. If a retry is possible, the stage is executed repeatedly according to the exponential backoff strategy; otherwise, the processing pipeline is terminated and a failure is reported. Notification unit 606 is used to confirm the consumption archiving event after all stages have been successfully executed, and to send an archiving completion notification to the downstream module. The cleanup unit 607 is used to calculate the retention period expiration time of each recording according to the retention period configured by the tenant, periodically scan the expired records, and ensure that a single instance in the cluster performs cleanup through a distributed scheduling lock, and softly delete the main recording record, the attribution record, and the speaker segment record in batches.

[0162] In one possible implementation, execution unit 604 is specifically used for: During the speaker activity detection phase, silence detection is performed on the left and right channel files respectively to obtain the active time sequence of the caller and the called party, and the detection results are stored in the pipeline context. The mixing stage mixes the left and right channels into a mono audio file and detects the audio duration; During the transcoding stage, the audio is transcoded according to the output format configured by the tenant; The waveform extraction stage extracts low-density waveform sampling data from the transcoded audio. During the encryption phase, a counter-mode block cipher is used to encrypt the audio file, and an initialization vector and key version number are written into it. The upload phase employs a three-step process: requesting a pre-signed certificate, PUT upload, and confirmation of completion to upload the encrypted file to object storage and obtain a resource identifier. During the metadata writing phase, the master recording is inserted with the composite key as the unique constraint, the attribution relationship record is written in batches, and the speaker segment record containing the speaker's time period and waveform data is written. The temporary file cleanup phase deletes all temporary files registered in the pipeline context.

[0163] In one possible implementation, execution unit 604 is specifically used for: The audio silence detection tool is invoked, the noise threshold and minimum silence duration are configured, the silence start time and duration output by the tool are parsed, the active time period is calculated in reverse, the active segment sequence of the left channel and the active segment sequence of the right channel are obtained, the left channel sequence is marked as the calling party and the right channel sequence is marked as the called party, and the marked active segment sequence is stored in the pipeline context.

[0164] In one possible implementation, execution unit 604 is specifically used for: When the right channel file is missing or its size is below a set threshold, it is determined to be a mono original recording, the mixing operation is skipped, and the left channel file is reused as the mixed output.

[0165] In one possible implementation, execution unit 604 is specifically used for: A random initialization vector is generated and written to the header of the ciphertext file. Audio data is encrypted block by block, and the version number of the currently active key is recorded. During decryption, the counter adjustment value is calculated based on the starting byte offset of the request, and a key stream is generated at the corresponding position to support independent decryption of any byte range.

[0166] In one possible implementation, execution unit 604 is specifically used for: Request a pre-signed upload certificate from the resource service to obtain the pre-signed URL and temporary resource identifier; Upload the encrypted file to the pre-signed URL using the HTTP PUT method; The confirmation interface of the resource service is called to convert the temporary resource identifier into a formal resource identifier and return it, and then the formal resource identifier is stored in the pipeline context.

[0167] In one possible implementation, execution unit 604 is specifically used for: Batch write ownership relationship records in a one-to-many or many-to-many manner. Each ownership relationship record includes a tenant code, recording master record identifier, user identifier, and user-side visibility field. Establish a unique index based on the combination of tenant code, master record identifier, and user identifier; When a user requests to hide a specific recording, the user-side visibility field in the corresponding attribution record is set to an invisible value. When the user queries the recording, only the recording records corresponding to the attribution relationship with a valid user-side visibility field are returned.

[0168] Since the distributed multi-tenant call recording archiving system 600 based on dual-track channel separation and idempotent multi-stage processing pipeline is a system corresponding to the distributed multi-tenant call recording archiving method based on dual-track channel separation and idempotent multi-stage processing pipeline provided in the above method embodiments, the specific implementation of each unit of the distributed multi-tenant call recording archiving system 600 based on dual-track channel separation and idempotent multi-stage processing pipeline is based on the same concept as in the above method embodiments. Therefore, for the specific implementation of each unit of the distributed multi-tenant call recording archiving system 600 based on dual-track channel separation and idempotent multi-stage processing pipeline, please refer to the description of the distributed multi-tenant call recording archiving method based on dual-track channel separation and idempotent multi-stage processing pipeline in the above method embodiments, and will not be repeated here.

[0169] This application embodiment also provides a distributed multi-tenant call recording and archiving device based on dual-track channel separation and idempotent multi-stage processing pipeline, the device including: a processor and a memory; The memory is used to store instructions; The processor is used to execute the instructions in the memory to perform the distributed multi-tenant call recording archiving method based on dual-track channel separation and idempotent multi-stage processing pipeline mentioned in the above embodiments.

[0170] It should be noted that the distributed multi-tenant call recording and archiving device based on dual-track channel separation and idempotent multi-stage processing pipeline provided in this application embodiment can all have the following hardware structure: Figure 7 The structure shown, Figure 7 This is a schematic diagram of the structure of a device provided in an embodiment of this application.

[0171] Please see Figure 7 As shown, device 700 includes: processor 710, communication interface 720, and memory 730. The number of processors 710 in device 700 can be one or more. Figure 7 Taking a processor as an example, in this embodiment, the processor 710, communication interface 720, and memory 730 can be connected via a bus system or other means. Figure 7 Taking the connection between China and Israel via the 740 bus system as an example.

[0172] Processor 710 may be a central processing unit (CPU), a network processor (NP), or a combination of a CPU and an NP. Processor 710 may further include hardware chips. These hardware chips may be application-specific integrated circuits (ASICs), programmable logic devices (PLDs), or combinations thereof. The PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), generic array logic (GAL), or any combination thereof.

[0173] The memory 730 may include volatile memory, such as random-access memory (RAM); the memory 730 may also include non-volatile memory, such as flash memory, hard disk drive (HDD) or solid-state drive (SSD); the memory 730 may also include a combination of the above types of memory.

[0174] Optionally, the memory 730 stores an operating system and programs, executable modules, or data structures, or subsets thereof, or extended sets thereof. The programs may include various operation instructions for implementing various operations. The operating system may include various system programs for implementing various basic services and handling hardware-based tasks. The processor 710 can read the programs in the memory 730 to implement the distributed multi-tenant call recording archiving method based on dual-track channel separation and idempotent multi-stage processing pipeline provided in this embodiment of the application.

[0175] The bus system 740 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus system 740 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 7The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0176] This application also provides a computer-readable storage medium including instructions that, when executed on a computer, cause the computer to perform the distributed multi-tenant call recording archiving method based on dual-track channel separation and idempotent multi-stage processing pipeline mentioned in the above embodiments.

[0177] This application also provides a computer program product containing instructions that, when run on a computer, cause the computer to execute the distributed multi-tenant call recording archiving method based on dual-track channel separation and idempotent multi-stage processing pipeline mentioned in the above embodiments.

[0178] Although the invention has been specifically shown and described in conjunction with preferred embodiments, those skilled in the art should understand that various changes in form and detail may be made to the invention without departing from the spirit and scope of the invention as defined in the appended claims, all of which shall be within the scope of protection of the invention.

Claims

1. A distributed multi-tenant call recording archiving method based on dual-track channel separation and idempotent multi-stage processing pipeline, characterized in that, include: When a call is established, the media server starts multi-channel recording, writes the caller's media stream to the left channel file and the called party's media stream to the right channel file, and stores the original dual-track recording file to shared storage after the call ends. After the call detail record (CDR) processing module identifies a call segment with recording, it constructs a recording archive event and delivers it to the message middleware. The archive event includes at least the tenant code, CDR segment identifier, original recording file path, calling and called numbers, call start time, and user affiliation list. The archived events are consumed by the archive service, deserialized into processing pipeline tasks, and the composite key consisting of the tenant code and the call detail record segment identifier is used to query the metadata database for judgment. If the corresponding recording master record already exists, the consumption is confirmed and the current processing ends; otherwise, the process enters the idempotent multi-stage processing pipeline. The idempotent multi-stage processing pipeline executes multiple processing stages in a fixed order. Each stage passes intermediate data through the pipeline context. Each stage has an independent retry mechanism and idempotency guarantee. The multiple processing stages include speaker activity detection stage, mixing stage, transcoding stage, waveform extraction stage, encryption stage, upload stage, metadata writing stage, and temporary file cleanup stage. When any processing stage fails, determine whether it can be retried based on the error type. If it can be retried, repeat the stage according to the exponential backoff strategy; otherwise, terminate the processing pipeline and report the failure. After all stages are executed successfully, the archive event is confirmed to be consumed, and an archive completion notification is sent to the downstream module. Based on the retention period configured by the tenant, calculate the expiration time of the retention period for each recording, periodically scan the expired records, and ensure that a single instance in the cluster performs cleanup through a distributed scheduling lock, and perform batch cascading soft deletion of the main recording record, the attribution record, and the speaker segment record.

2. The method according to claim 1, characterized in that, The idempotent multi-stage processing pipeline executes multiple processing stages sequentially in a fixed order, including: The speaker activity detection phase performs silence detection on the left channel file and the right channel file respectively to obtain the active time period sequence of the caller and the called party, and stores the detection results in the pipeline context. The mixing stage mixes the left and right channels into a mono audio file and detects the audio duration. The transcoding stage transcodes the audio according to the output format configured by the tenant; The waveform extraction stage extracts low-density waveform sampling data from the transcoded audio. The encryption phase uses a counter-mode block cipher to encrypt the audio file, and writes the initialization vector and key version number. The upload phase employs a three-step process: applying for a pre-signed certificate, PUT upload, and confirmation of completion to upload the encrypted file to object storage and obtain a resource identifier. The metadata writing stage inserts the main recording record with the composite key as the unique constraint, writes the attribution relationship record in batches, and writes the speaker segment record containing the speaker's time period and waveform data. The temporary file cleanup phase deletes all temporary files registered in the pipeline context.

3. The method according to claim 2, characterized in that, The speaker activity detection phase includes: The audio silence detection tool is invoked, the noise threshold and minimum silence duration are configured, the silence start time and duration output by the tool are parsed, the active time period is calculated in reverse, the active segment sequence of the left channel and the active segment sequence of the right channel are obtained, the left channel sequence is marked as the calling party and the right channel sequence is marked as the called party, and the marked active segment sequence is stored in the pipeline context.

4. The method according to claim 2, characterized in that, The mixing stage includes: When the right channel file is missing or its size is below a set threshold, it is determined to be a mono original recording, the mixing operation is skipped, and the left channel file is reused as the mixed output.

5. The method according to claim 2, characterized in that, The encryption phase employs a counter-mode block cipher to encrypt the audio file, including: A random initialization vector is generated and written to the header of the ciphertext file. Audio data is encrypted block by block, and the version number of the currently active key is recorded. During decryption, the counter adjustment value is calculated based on the starting byte offset of the request, and a key stream is generated at the corresponding position to support independent decryption of any byte range.

6. The method according to claim 1, characterized in that, The upload phase employs a three-step process: requesting a pre-signed certificate, PUT upload, and confirmation of completion to upload the encrypted file to object storage and obtain a resource identifier, including: Request a pre-signed upload certificate from the resource service to obtain the pre-signed URL and temporary resource identifier; The encrypted file is uploaded to the pre-signed URL using the HTTP PUT method; The confirmation interface of the resource service is invoked to convert the temporary resource identifier into a formal resource identifier and return it, and the formal resource identifier is stored in the pipeline context.

7. The method according to claim 2, characterized in that, The batch writing of ownership relationship records includes: Batch write ownership relationship records in a one-to-many or many-to-many manner. Each ownership relationship record includes a tenant code, recording master record identifier, user identifier, and user-side visibility field. A unique index is established on the combination of the tenant code, the recording master record identifier, and the user identifier; When a user requests to hide a specific recording, the user-side visibility field in the corresponding attribution record is set to an invisible value. When the user queries the recording, only the recording records corresponding to the attribution relationship where the user-side visibility field has a valid value are returned.

8. A distributed multi-tenant call recording archiving system based on dual-track channel separation and idempotent multi-stage processing pipeline, characterized in that, include: The recording unit is used to start multi-channel recording by the media server when the call is established, write the calling party's media stream to the left channel file and the called party's media stream to the right channel file, and store the original dual-track recording file to shared storage after the call ends. The production unit is used to construct a recording archive event and deliver it to the message middleware after the call detail record processing module identifies a call segment with recording. The archive event includes at least the tenant code, call detail record segment identifier, original recording file path, calling and called numbers, call start time and user affiliation list. The verification unit is used to consume the archived events by the archiving service, deserialize them into processing pipeline tasks, and query the metadata database using the composite key composed of the tenant code and the call detail record segment identifier. If a corresponding recording master record already exists, the consumption is confirmed and the current processing ends; otherwise, the process enters the idempotent multi-stage processing pipeline. The execution unit is used to execute multiple processing stages in a fixed order through the idempotent multi-stage processing pipeline. Each stage passes intermediate data through the pipeline context. Each stage has an independent retry mechanism and idempotency guarantee. The multiple processing stages include speaker activity detection stage, mixing stage, transcoding stage, waveform extraction stage, encryption stage, upload stage, metadata writing stage, and temporary file cleanup stage. The retry unit is used to determine whether a retry is possible when any processing stage fails. If a retry is possible, the stage is repeated according to the exponential backoff strategy; otherwise, the processing pipeline is terminated and a failure is reported. The notification unit is used to confirm the consumption of the archiving event after all stages have been successfully executed, and to send an archiving completion notification to the downstream module. The cleanup unit is used to calculate the retention period expiration time of each recording based on the retention period configured by the tenant, periodically scan the expired records, and ensure that a single instance in the cluster performs cleanup through a distributed scheduling lock, and softly delete the main recording record, the attribution record, and the speaker segment record in batches.

9. An electronic device, characterized in that, The device includes: a processor and a memory; The memory is used to store instructions; The processor is configured to execute the instructions in the memory to perform the method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, Including instructions that, when run on a computer, cause the computer to perform the method described in any one of claims 1-7 above.

Images