Digital badge driven business window multi-source recording data fusion intelligent analysis system

Abstract

The application discloses a digital badge driven business window multi-source recording data fusion intelligent analysis system and particularly relates to the technical field of voice processing. The system is based on asynchronous audio streams collected by a wearable digital badge, an array microphone and an environmental pickup device. Anchor point detection, event graph construction, cross-source matching, elastic time warping alignment and other steps are adopted to realize the structured alignment of multi-source recording data and the construction of a unified time reference. The system introduces multiple anchor point detectors (semantic keywords, acoustic mutations, prompt sounds) to complete cross-source event labeling. In the anchor point pairing process, a unified clipping and cross sliding matching strategy is combined to construct an event alignment graph and perform confidence classification. Then, based on a hierarchical elastic time warping method, the audio stream is subjected to multi-scale alignment and resampling, and finally, frame-level synchronized voice data streams are output, thereby providing unified time support for downstream speaker separation, behavior audit and semantic mapping.

Description

Technical Field

[0001] This invention relates to the field of voice processing technology, and more specifically, to a digital employee badge-driven intelligent analysis system for fusing and analyzing multi-source recording data from business windows. Background Technology

[0002] With the increasing demands for data traceability and service quality analysis in scenarios such as smart government affairs, financial services, and customer operation windows, multi-source audio acquisition and fusion processing have become crucial components of intelligent business systems. In actual deployments, systems often simultaneously access multiple heterogeneous audio sources, including wearable devices (such as digital badge microphones), fixed pickup devices (such as counter microphones), and environmental monitoring devices, to record voice information, service prompts, and user responses during business processes. However, due to factors such as different locations of acquisition devices, asynchronous sampling frequencies, and device clock drift, significant time discrepancies and structural misalignments often exist between different audio streams, severely hindering the processing effectiveness of subsequent modules such as speech recognition, speaker separation, and behavior auditing.

[0003] In existing technologies, most multi-channel recording systems only perform coarse alignment based on timestamps or static templates, lacking structured registration methods at the level of actual interactive events. Furthermore, they have poor tolerance for asynchrony between audio sources and cannot effectively cope with complex factors such as noisy environments, clock inconsistencies, and speech rate variations. At the same time, alignment quality cannot be quantitatively evaluated, and there is a lack of dynamic adjustment mechanisms based on confidence feedback, making it difficult to guarantee system stability and the accuracy of dialogue reconstruction.

[0004] Therefore, there is an urgent need for a multi-source recording fusion analysis system with anchor point recognition capabilities, structured pairing mechanisms, and multi-scale flexible alignment strategies, which can achieve highly robust and accurate asynchronous audio stream alignment and time-unified processing in complex business environments to meet the needs of intelligent analysis and compliance auditing. Summary of the Invention

[0005] In order to overcome the above-mentioned defects of the prior art, embodiments of the present invention provide a digital employee badge-driven intelligent analysis system for multi-source recording data fusion at business windows to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] The digital employee ID card-driven intelligent analysis system for multi-source audio recording data fusion at business windows includes the following modules:

[0008] The multi-source anchor detection and quadruple generation module is used to perform anchor detection in parallel in each audio stream and generate anchor structure quadruples. The anchor structure quadruple includes the timestamp of the anchor in the audio source, the anchor type label, the anchor detection confidence score, and the time scale tolerance range.

[0009] The event alignment graph construction and anchor pairing strategy module is used to select either a cross-sliding matching strategy or a unified time axis pruning strategy based on the anchor time axis pruning strategy selection mechanism to construct a cross-source event alignment graph and search for the anchor pair with the lowest matching cost.

[0010] The multi-scale elastic time warp and synchronous resampling module constructs a time mapping function based on anchor point-to-confidence level hierarchies. It achieves multi-scale alignment through affine mapping, constrained elastic time warp, and frame-level fine-tuning, and outputs a frame-level synchronized speech data stream.

[0011] In a preferred embodiment, anchor point detection includes three types of detectors: a semantic anchor detector based on a lightweight keyword recognition model, which detects preset business keywords and generates semantic anchor points; an acoustic anchor detector that identifies acoustic mutation events through short-time energy and spectral flux analysis; and a prompt tone anchor detector that uses a template matching filter algorithm to identify fixed-frequency prompt tones.

[0012] In a preferred embodiment, the anchor timeline pruning strategy selection mechanism is dynamically triggered through the following steps: using the digital ID card audio stream as the main reference source, a sliding time window is defined; if the number of anchor points for any audio source is zero after the unified timeline pruning strategy, the cross-sliding matching strategy is enabled; otherwise, the short-time RMS energy fluctuation rate of the main reference source and the cross-source anchor point synchronization rate are calculated, and a comprehensive score is calculated based on the weighted summation; when the comprehensive score is greater than a preset switching threshold, the cross-sliding matching strategy is switched; otherwise, the unified timeline pruning strategy continues to be used.

[0013] In a preferred embodiment, the cross-source anchor synchronization rate is defined as follows: within the current sliding time window, traverse the anchor structure quadruple of all audio sources, and determine whether each anchor in a certain audio source has an anchor of the same type and close in time position in any other audio source. If the above condition is met, the anchor pair is regarded as a valid cross-source anchor synchronization event. Further, count the number of anchor pairs constituting cross-source synchronization events among all anchors in the current window, and calculate the cross-source anchor synchronization rate accordingly.

[0014] The cross-source anchor synchronization rate is defined as follows: within the current sliding time window, traverse the anchor structure quadruples of all audio sources, and determine whether each anchor in a certain audio source has an anchor of the same type and close in time position in any other audio source. If the above conditions are met, the anchor pair is considered a valid cross-source anchor synchronization event. Further, count the number of anchor pairs constituting cross-source synchronization events among all anchors in the current window, and calculate the cross-source anchor synchronization rate accordingly.

[0015] In a preferred embodiment, the cross-sliding matching strategy proceeds as follows: retain the original set of anchor points for all audio sources; for each anchor point in source m, dynamically calculate the maximum matching sliding window width on source n; the maximum matching sliding window width is proportional to the weighted sum of the anchor point time divergence index and the anchor point density variance; search for anchor points of the same type and with a time difference less than the tolerance range within the window to construct candidate edges; sort by matching cost and truncate redundant candidate edges.

[0016] In a preferred embodiment, the anchor point time divergence index is quantified by the mean of the maximum time difference of the same type of anchor points in different sources; the anchor point density variance is quantified by the degree of dispersion of the number distribution of anchor points in each source.

[0017] In a preferred embodiment, the joint confidence of anchor points is used for classification. Points above the strong constraint threshold are marked as strong alignment control anchor points, serving as key control points for piecewise time mapping; points between the strong and soft constraint thresholds are marked as soft alignment guiding anchor points, providing weak guiding support; and points below the soft constraint threshold are determined to be abnormal matches and removed.

[0018] In a preferred embodiment, multi-scale elastic time-warping is performed in a layered manner:

[0019] Coarse-scale: Using the strong alignment control anchor point as a reference point, a piecewise affine time mapping function is constructed to correct global delay and linear drift;

[0020] Mesoscale: Within adjacent strong anchor point intervals, combine soft alignment to guide anchor points and perform constrained DTW;

[0021] Fine-scale: In overlapping segments of speech activity, frame-level fine-tuning is performed based on Mel spectral distance or coherent phase difference.

[0022] In a preferred embodiment, the audio stream source includes: a wearable digital name tag microphone; a counter array microphone; and a business window environmental sound pickup device.

[0023] The technical effects and advantages of this invention are as follows:

[0024] This invention utilizes parallel detection of multiple event markers, including semantic anchors, acoustic anchors, and cue tone anchors. Combined with a multi-scale elastic time warp (ETW) and anchor confidence grading mechanism, the system can compress cross-source time deviations to the millisecond level under conditions of sampling rate differences, clock skew, or network latency, achieving frame-level synchronization and providing a unified and reliable time reference for subsequent processing such as speaker separation and speech recognition.

[0025] The system dynamically switches between "unified timeline pruning" and "intra-source retention + cross-sliding matching": it evaluates the alignment environment in real time using a scoring function composed of short-time RMS energy fluctuation rate and cross-source anchor synchronization rate, and selects the optimal strategy. This design effectively resists device clock out-of-sync, speech rate variations, and noise interference, ensuring stable alignment accuracy in multiple scenarios.

[0026] Event alignment graph modeling is adopted to calculate the matching cost and confidence for each candidate edge. Strong alignment control anchors, soft alignment guiding anchors and anomaly removal mechanisms are introduced to quantify the overall alignment quality in real time and automatically filter out false matches, which significantly reduces the error rate of downstream speech analysis. Attached Figure Description

[0027] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings;

[0028] Figure 1 This is a schematic diagram of the structure of the intelligent analysis system for multi-source audio recording data fusion at a business window driven by a digital employee badge, as described in this invention.

[0029] Figure 2 This is a schematic diagram of the anchor point pairing process under the cross-sliding matching strategy of the present invention. Detailed Implementation

[0030] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] Example

[0032] This invention relates to a digital employee badge-driven intelligent analysis system for multi-source audio recording data fusion at business windows, such as... Figure 1 As shown, it includes the following modules:

[0033] The multi-source anchor point detection and quadruplet generation module is used to achieve frame-level alignment and structured fusion processing between multi-source audio streams. First, anchor point detection is performed in parallel in each input audio stream to extract structural events that can be used for alignment reference. The implementation method of this step includes the following specific processes:

[0034] First, the system initializes a sliding window buffer for anchor point caching and loads a preset anchor point detector for each audio stream. These audio streams include, but are not limited to, audio streams from digital name tags worn by employees, audio streams from array microphones installed inside and outside the counter, and audio streams from environmental microphones deployed around the sales window. The anchor point detectors operate independently on each audio stream and are categorized into three types based on anchor point type:

[0035] Firstly, the semantic anchor detector. This module is based on a pre-trained lightweight keyword recognition model, implemented using a CTC structure or a Time Delayed Neural Network (TDNN) structure, and performs real-time semantic trigger detection on the device side. The system constructs a keyword vocabulary from common business phrases used in business windows (such as "Hello, what business do you need?", "Please show your ID card", "Please sign"), and the model input is the MFCC or Log-Mel features of the audio frame, and the output is the confidence time series of the corresponding semantic word. If the cumulative confidence of a keyword exceeds a set threshold within the sliding time window, and there is a decoding interval with clear start and end boundaries, it is determined to be a valid semantic anchor. The system records its start frame time, end frame time, keyword category label, and recognition confidence.

[0036] Secondly, the acoustic anchor detector. This module is used to identify non-verbal acoustic events, especially energy bursts and spectral shifts, commonly seen in instantaneous behaviors such as speech bursts, sudden questions, table taps, and falling objects. The system extracts the short-time energy and spectral flux of audio frames within a sliding window and sets sliding detection thresholds at multiple resolutions. If the energy increase or spectral flux increment of a frame exceeds a preset standard deviation multiple, and the energy difference between the preceding and following frames is not within the range of background noise estimation, it is considered an acoustic burst event. These anchors are mainly used to pinpoint the occurrence time of the start or interruption of a strong vocal burst, recording frame time, event intensity level, and rate of change.

[0037] Thirdly, the prompt tone anchor detector. This module targets common fixed-frequency prompt tones at service windows (such as the "ding" sound of number calling and employee number announcements), and uses a matching filter algorithm based on a preset template for identification. The system pre-collects standard prompt tone samples, extracts their frequency envelope features or Mel templates, and performs convolutional matching or DTW-based matching scoring on the real-time audio. If the matching score exceeds a threshold and the time interval between occurrences is consistent with or close to the timestamp of the prompt event in the back-end business system, it is recorded as a prompt tone anchor point.

[0038] For the above three types of anchor points, the system uniformly encapsulates them into an anchor point structure quadruple, in the form of: <t i ,τ i ,s i ,δ i >;where: t iThis represents the timestamp of the anchor point within the current audio source; τ i Anchor type tags (such as semantic keywords, acoustic abrupt changes, and cue tone identifiers); s i This represents the anchor point detection confidence score (range 0–1); δ i This indicates the time scale, i.e., the maximum acceptable range of time deviation for this anchor point in the alignment map construction.

[0039] All detected anchor point structure quadruples are written to the anchor point buffer queue of the current audio source in chronological order. This buffer queue serves as the input interface for subsequent steps, forming the structural basis for multi-source audio alignment processing.

[0040] The event alignment graph construction and anchor pairing strategy module is used after the system completes anchor detection of multi-source audio streams and generates standardized anchor structure quadruples, leading to the event alignment graph construction and anchor pairing stage. The goal of this step is to identify cross-source anchor pairs reflecting the same real-world event in multiple asynchronous recording data streams, providing a structured alignment constraint basis for subsequent time alignment and speech separation. To improve the system's adaptability to diverse scenarios, an "anchor timeline pruning strategy selection mechanism" is introduced before this step to dynamically determine whether to adopt a unified timeline pruning strategy or enable a cross-sliding matching strategy.

[0041] First, the system uses a preset primary reference source (usually the microphone stream from the digital name tag worn by employees) as the time alignment axis, defines the current sliding time window (e.g., 2 seconds) on it, and initially attempts to synchronously prune the anchor point structure quadruples of each audio source according to this time axis, retaining only the anchor points whose timestamps fall within the current window interval to form a set. If any audio source is found to have zero anchor points after cropping, indicating that the source lacks valid event markers in the current time period, the system determines that the unified cropping has failed, skips subsequent evaluation, and directly activates the cross-sliding matching strategy to retain the complete original anchor point set A for each source. m .

[0042] If all sources successfully cut out anchor points, the system further evaluates the alignment quality of the unified clipping strategy within the current window, involving two evaluation metrics: the short-time RMS (root mean square) energy fluctuation rate of the main reference source, reflecting the stability of the recording intensity and the activity of the sound source; and the cross-source anchor point synchronization rate, which refers to the proportion of similar anchor points that can be detected on two or more sources within the current window.

[0043] Calculation of short-time RMS energy fluctuation rate of the main reference source: The system selects a certain audio source m* as the main reference source (usually the device on the employee's side wearing a digital name tag), and divides its audio signal within the current sliding window into frames. Let there be K frames in this window, and each frame signal is... The RMS energy of the k-th frame is denoted as: L: Number of sampling points per frame; The l-th sampling point of the k-th frame as the main reference source; E k RMS energy of frame k.

[0044] The short-time RMS energy volatility of the primary reference source was then calculated as follows: in: Represents the average energy; R represents the standard deviation of energy. RMS The primary reference source is the short-time RMS energy volatility.

[0045] The system iterates through the set of four-tuple structures of all audio source anchor points within the current time window, and for each anchor point... If an anchor point exists in any other source n≠m Satisfying τ i =τ j ;|t i -t j |≤min(δ i ,δ j ); then it is believed and This constitutes a cross-source anchor point synchronization event.

[0046] Let the total number of anchor points be N, and the number of anchor point pairs satisfying the above formula be N. sync Then the cross-source anchor synchronization rate R sync for:

[0047] The anchor pairing process is determined by combining the short-time RMS energy volatility of the main reference source and the synchronization rate of the cross-source anchor points.

[0048] First, the short-time RMS energy volatility of the main reference source and the cross-source anchor synchronization rate are normalized. Then, the comprehensive score is calculated based on a weighted summation, as shown in the following formula: Where α and β are the weighting coefficients of the two indicators, satisfying α+β=1, for example α=0.4, β=0.6; S is the comprehensive score result, the higher the score, the worse the alignment quality. The normalized value of the short-time RMS energy volatility of the primary reference source; This is the normalized value of the cross-source anchor synchronization rate.

[0049] If the overall score is greater than the preset switching threshold (e.g., 0.8), switch to the cross-sliding matching strategy; otherwise, continue using the unified timeline pruning strategy.

[0050] After entering the candidate edge construction phase, the system executes different anchor point pairing processes according to the currently adopted matching strategy. The implementation method is as follows:

[0051] First, the system obtains the set of anchor points for each audio stream within the current window from the previous stage, represented as:

[0052] The anchor point pairing process under the unified timeline clipping strategy is as follows:

[0053] This strategy applies when there is a clear synchronization reference between multiple recording devices (such as the clock of a certain ID badge wearing device as the main reference clock). The system uses a unified time window to trim anchor points in all audio sources, ensuring that pairing is performed only on anchor points within the current window, and the pairing range is strictly limited to the boundaries of this window.

[0054] The implementation steps are as follows:

[0055] The system first sets the sliding time window length (e.g., 2 seconds) and constructs the current active window interval [T0, T0+Tw] based on a unified reference time axis, where T0 is the start time of the current window and Tw is the window length.

[0056] For each audio source m, the system iterates through its set of anchor points A. m Only anchor points satisfying ti∈[T0,T0+Tw] are retained, denoted as

[0057] The system performs pairwise source-to-source combination: for any two sources m ≠ n, the set of anchor points after clipping is determined. Perform a pairing attempt.

[0058] For any pair of anchor points, if the anchor point types are consistent: τ i =τ j The time difference is within the scale tolerance range: |t i -t j |≤min(δ i ,δ j Then construct candidate edges.

[0059] For each candidate edge, calculate the matching cost. The formula is as follows: in: This represents the difference in timestamps between two anchor points on their respective sources, reflecting the degree of time alignment. This is an indicator function for inconsistent anchor types; it returns 1 if the types are different and 0 if the types are the same, ensuring that only anchors of the same type are matched; min(s) i ,s j The lower confidence value among the two anchor points represents the pairing confidence; the pairing confidence is affected by the weakest confidence value. If either anchor point is unreliable, the overall pairing quality decreases; 1-min(s)i ,s j As a cost term, a higher value indicates a lower confidence level; λ t The weight for the time difference term can be set, for example, to λ. t =0.5; λ τ For type inconsistency penalty terms, for example, it can be set to λ. τ =1.0; λ s The minimum confidence penalty term can be set to λ, for example. s =0.5.

[0060] The final candidate edges are constructed as edge elements of a graph structure, with nodes being all anchor points retained within the source, forming a cross-source anchor event graph G = (V, E); node V represents all anchor points within the current window; edge E represents all constructed candidate anchor point pairs, with matching costs attached.

[0061] The anchor pairing process under the cross-sliding matching strategy is as follows: Figure 2 As shown, the specific steps are as follows:

[0062] This strategy applies to situations where there is a slight clock inconsistency or a faulty reference clock between recording devices. The system does not uniformly trim the anchor points of each source, but retains the original anchor points within each source and performs forgiving matching between different sources through a cross-sliding window.

[0063] For any source m, the system retains its set of anchor points A. m No uniform time clipping is performed.

[0064] For any source m ≠ n, perform the following cross-sliding matching procedure:

[0065] For each anchor point in source m The system defines its matching window interval on source n as [t i -δ match ,t i +δ match ], where δ match To match the maximum width of the sliding window.

[0066] The maximum matching sliding window width is calculated using the following steps:

[0067] Anchor point time divergence index D anchor This value quantifies the degree of temporal aggregation of the same type of event among different source anchors within the current sliding window. A larger value indicates a greater time difference between different sources when capturing the same anchor, suggesting clock skew or inconsistent speech rates, and the matching should be more forgiving. K is the total number of cross-source anchor groups marked as the same type within the current time window; The timestamp of the anchor point was detected by the m-th audio source in the k-th group; max-min: represents the maximum time difference of the anchor point group in different sources.

[0068] Anchor point density variance This measures the uniformity of anchor point appearance across different audio sources within the current window. Uneven density often indicates significant differences in response between sources, sound field differences, or inconsistent pickup patterns, increasing matching risk and necessitating a higher window tolerance. M: Total number of audio sources; N m : The number of anchor points detected in the current window for the m-th audio source; T: Window length; The average density of all source anchor points; the more uneven the density, the better. The higher.

[0069] The system inputs the above two indicators into the matching window adjustment function: Δt0 baseline matching window width (e.g., 300 milliseconds); D0, λ1 and λ2 are the baseline values ​​of anchor point divergence and density variance recorded in system training or historical statistics, used for normalization; λ1 and λ2 represent adjustment coefficients, controlling the influence of the two factors on window widening. The recommended value range for λ1 is 0.2–0.6. When the system is running in scenarios with multiple speech rate differences or large device clock errors, this coefficient should be appropriately increased. The recommended value range for λ2 is 0.1–0.4. If the system is sensitive to changes in anchor point density (e.g., in dense speaker dialogue scenarios), this weight can be increased. λ1 + λ2 ≤ 1 to ensure that the widening is within a reasonable range and to avoid infinite window expansion.

[0070] In the interval [t] i -δ match ,t i +δ match Within ], search for all anchor points in source n. If it satisfies: anchor point type is consistent: τ i =τ j ;|t i -t j |≤min(δ i ,δ j If the match is 0, it is considered a candidate match.

[0071] For each pair of candidate anchor points, construct candidate edges and calculate their cost function, consistent with the above.

[0072] To avoid excessive node redundancy, the system sets a constraint threshold on the maximum number of anchor pairs in the sliding matching window. For example, each anchor can try to pair with a maximum of K candidate pairs on other sources (e.g., K=5), and is truncated according to a cost priority queue.

[0073] The final candidate edges are constructed as edge elements of a graph structure, with nodes being all anchor points retained within the source, forming a cross-source anchor event graph G = (V, E). Node V represents all anchor points within the current window; edge E represents all constructed candidate anchor point pairs, with matching costs attached.

[0074] After constructing the nodes of the cross-source event alignment graph, the system enters the graph search phase for anchor pairing to identify anchor pairs reflecting the same real-world event from different audio sources. This phase is implemented using the following steps:

[0075] First, the system uses a sliding window strategy, performing local graph structure updates only on anchor nodes and their adjacent edges that newly enter the current window, thus avoiding repeated calculations of cached historical matching relationships and reducing system load. To prevent ambiguity and pairing conflicts caused by multiple matching, the system imposes a unique matching constraint on each anchor node, limiting each anchor node to participate in only one valid pairing within any window.

[0076] Subsequently, the system performs a pairing path search operation on the constructed event graph. To balance matching accuracy and operational efficiency, the system supports two matching algorithms: one is a heuristic greedy algorithm based on ascending order of matching cost, which traverses the candidate edge set one by one and includes unoccupied anchor pairs in the pairing results; the other is a variant of the Hungarian algorithm suitable for scenarios with a small number of anchors, which can search for the set of anchor pairs with the minimum matching cost within the local graph structure to obtain a better global pairing solution. The two strategies can be automatically switched according to system configuration or selected according to computing resources.

[0077] For each matching anchor pair obtained from the search, the system further calculates its joint confidence value. The joint confidence value is jointly determined by the recognition confidence of each anchor point (such as semantic keyword recognition score, acoustic mutation score, etc.) and the matching cost corresponding to its pairing edge, and is used to measure the consistency of the anchor pair in multiple dimensions such as time alignment, event semantics, and acoustic features. The system presets two confidence level thresholds, which are used to classify the validity level of the matching results.

[0078] Specifically, when the joint confidence of an anchor pair is higher than the strong constraint anchor threshold (e.g., 0.8), the anchor pair is identified as a "strong alignment control anchor" and will serve as a key control point in the piecewise time mapping fitting process; when the joint confidence is between the strong constraint and soft constraint thresholds (e.g., 0.6-0.8), the anchor pair is marked as a "soft alignment guiding anchor" and will provide weak guiding support in the subsequent elastic alignment stage; other pairings with insufficient joint confidence will be regarded as abnormal matches and eliminated.

[0079] The multi-scale elastic time warp and synchronous resampling module is used after the system completes the screening of cross-source anchor pair matching and joint confidence level classification. In this phase, the system constructs a coarse-to-fine time mapping function based on the sorted anchor pair sequence, achieving frame-level alignment and unified resampling of the asynchronously sampled audio stream. The implementation method of this phase includes the following steps:

[0080] First, the system constructs a hierarchical pyramid structure for time mapping based on the confidence levels in the anchor pairing results. At the coarsest level, the system uses only pairs marked as strongly aligned control anchors as reference control points to construct a piecewise affine time mapping function. This function achieves precise alignment at the time positions of the strongly constrained anchor pairs and extrapolates the time offset linearly within their intervals to comprehensively correct the global delay and linear drift problems of non-reference audio sources. This processing is applicable to long-term cumulative offset phenomena caused by clock frequency deviations, sampling delays, or asynchronous network transmissions.

[0081] Secondly, at the intermediate scale, the system employs a locally constrained elastic time warp (DTW) search process between any pair of adjacent strong alignment control anchors to achieve finer nonlinear alignment. Specifically, the system constructs a local dynamic programming path within the aforementioned segment and introduces a "Sakoe-Chiba constraint band" to limit the search path's boundaries, ensuring that the path only deviates within a specified time tolerance range and preventing erroneous cross-interval pairings. During this process, the system uses the upper-level affine mapping result as the initial trajectory and combines it with soft alignment guide anchors located within soft confidence intervals to provide soft guidance on the local offset direction, thereby improving the accuracy and stability of the alignment path.

[0082] Secondly, at the finest scale, the system further performs frame-level fine-grained sub-band time fine-tuning for audio frame segments where significant overlap in speech activity is detected within strong anchor point pairs. This stage uses frame-level time-frequency distance metrics such as Mel-spectral distance and coherent phase difference to construct an alignment cost function to perform sub-frame-level alignment path correction. This step primarily eliminates fine-grained misalignments caused by differences in speech rate, pronunciation inconsistencies, blurred speech burst boundaries, or slight drift in the sampling clock, ensuring that alignment errors are controlled within milliseconds in scenarios involving critical semantic events (such as customer name confirmation and amount announcements). To reduce the computational overhead of this stage, the system activates fine-scale alignment only within overlapping speech segments and employs a sparse sampling strategy to limit the dynamic programming bandwidth.

[0083] Finally, the system integrates the three-layer time alignment results in a top-down manner, outputs a complete time mapping function, and transforms it into a differentiable time resampling operator. This operator is then applied to the resampling processing of all non-reference audio sources to achieve frame-level synchronization. During the resampling process, the system synchronously records the error residuals and deviation trends of each mapping segment, and uses these data as the basis for subsequent alignment quality evaluation, abnormal pairing detection, and alignment rescheduling triggering logic.

[0084] The aforementioned multi-scale elastic time warp mechanism employs a layered strategy of "anchor-driven—affine-first—constrained fine-tuning," which not only ensures the system's alignment robustness in multi-source asynchronous scenarios but also effectively controls algorithm complexity, making it suitable for processing real-time multi-channel voice data streams in business window scenarios. The output unified time alignment benchmark provides structured time support for subsequent modules such as speaker separation, role recognition, semantic mapping, and behavior auditing.

[0085] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0086] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0087] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0088] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0089] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A digital employee badge-driven intelligent analysis system for multi-source audio recording data fusion at business windows, characterized in that: It includes a multi-source anchor point detection and quadruplet generation module, an event alignment graph construction and anchor point pairing strategy module, and a multi-scale elastic time warp and synchronous resampling module. The multi-source anchor detection and quadruple generation module is used to perform anchor detection in parallel in each audio stream and generate anchor structure quadruples. The anchor structure quadruple includes the timestamp of the anchor in the current audio source, the anchor type label, the anchor detection confidence score, and the time scale tolerance range. The event alignment graph construction and anchor point pairing strategy module is used to define the current sliding time window with the digital ID card audio stream as the main reference source, and first to synchronously prune the anchor point structure quadruples of each audio source according to the unified time axis pruning strategy; when the number of anchor points of any audio source is zero after unified time axis pruning, the cross sliding matching strategy is directly enabled and the original anchor point set of all audio sources is retained; when each audio source retains anchor points after unified time axis pruning, the short-time RMS energy fluctuation rate and cross-source anchor point synchronization rate of the main reference source are calculated, and the unified time axis pruning strategy or the cross sliding matching strategy is selected to continue using the unified time axis pruning strategy or switch to the cross sliding matching strategy based on the comparison result between the weighted sum of the two comprehensive scores and the preset switching threshold. In the cross-sliding matching strategy, for each anchor point in source m, the maximum matching sliding window width is dynamically calculated on source n. The maximum matching sliding window width is proportional to the weighted sum of the anchor point time divergence index and the anchor point density variance. Within the maximum matching sliding window width, anchor points of the same type and with a time difference less than the tolerance range of the corresponding time scale are searched to construct candidate edges. The candidate edges are sorted according to the matching cost and redundant candidate edges are truncated to obtain candidate anchor point pairs in the cross-source event alignment graph. The multi-scale elastic time warp and synchronous resampling module is used to classify the joint confidence of candidate anchor pairs. Candidate anchor pairs with a joint confidence level higher than the strong constraint threshold are marked as strong alignment control anchors, serving as key control points for piecewise time mapping. Candidate anchor pairs with a joint confidence level between the strong constraint threshold and the soft constraint threshold are marked as soft alignment guiding anchors, providing weak guiding support. Candidate anchor pairs with a joint confidence level lower than the soft constraint threshold are judged as abnormal matches and are eliminated. Based on the classification results, a time mapping function is constructed hierarchically. The coarse scale constructs a piecewise affine time mapping function with the strong alignment control anchor as the reference point to correct global delay and linear drift. The medium scale performs constrained elastic time warp in the interval between adjacent strong alignment control anchors in combination with soft alignment guiding anchors. The fine scale performs frame-level fine-tuning based on Mel spectrum distance or coherent phase difference in the overlapping segment of speech activity, and outputs a frame-level synchronized speech data stream.

2. The intelligent analysis system for multi-source audio recording data fusion at a business window driven by a digital employee badge as described in claim 1, characterized in that: The anchor point detection includes three types of detectors: semantic anchor detectors, which detect preset business keywords and generate semantic anchor points based on a lightweight keyword recognition model; acoustic anchor detectors, which identify acoustic mutation events through short-time energy and spectral flux analysis; and prompt tone anchor detectors, which use template matching filtering algorithms to identify fixed-frequency prompt tones.

3. The intelligent analysis system for multi-source audio recording data fusion at a business window driven by a digital employee badge as described in claim 1, characterized in that: The cross-source anchor synchronization rate is defined as follows: within the current sliding time window, traverse the anchor structure quadruples of all audio sources, and determine whether each anchor in a certain audio source has an anchor of the same type and close in time position in any other audio source. If the above condition is met, the anchor pair is regarded as a valid cross-source anchor synchronization event. Further, count the number of anchor pairs constituting cross-source synchronization events among all anchors in the current window, and calculate the cross-source anchor synchronization rate accordingly.

4. The intelligent analysis system for multi-source audio recording data fusion at a business window driven by a digital employee badge as described in claim 1, characterized in that: The anchor point time divergence index is quantified by the mean of the maximum time difference of the same type of anchor points in different sources, and the anchor point density variance is quantified by the degree of dispersion of the number distribution of anchor points in each source.

5. The intelligent analysis system for multi-source audio recording data fusion at a business window driven by a digital employee badge as described in claim 1, characterized in that: Audio stream sources include: wearable digital name tag microphones; counter array microphones; and ambient sound pickup devices at the sales window.

Images