Multichannel output state real-time monitoring method and system based on voiceprint recognition

Abstract

The invention discloses a multi-channel output state real-time monitoring method and system based on voiceprint recognition, and the method comprises the following steps: synchronously collecting real-time sound signals outputted by each channel, and sequentially carrying out the enhancement, framing, feature extraction and feature fusion, thereby forming a multi-dimensional voiceprint feature vector of each channel; collecting voiceprint feature vectors of standard test sounds output by each sound channel, and training through a support vector machine to obtain a standard state feature space of each sound channel; obtaining a comprehensive abnormal index through the multi-dimensional voiceprint feature vector and the standard state feature space, and recognizing an abnormal type; and a regulation and control instruction is generated according to the abnormality type, and the abnormality type, the occurrence time and the sound channel position are transmitted to a monitoring terminal in real time through a bus communication unit. The method is used for solving the technical problems of insufficient synchronous acquisition precision, single feature extraction dimension, incomplete standard state space modeling, poor anomaly detection robustness and incapability of accurately classifying anomaly types in the real-time monitoring of the existing multichannel output system.

Description

Technical Field

[0001] The present invention relates to the technical field of voiceprint recognition, and in particular to a method and system for real-time monitoring of multi-channel output status based on voiceprint recognition. Background Art

[0002] In the field of real-time monitoring of multi-channel output systems, accurately capturing the characteristics of the acoustic signals of each channel and performing effective analysis is the key to achieving status monitoring. However, existing technologies face the following challenges: First, there are deficiencies in the synchronous acquisition and feature alignment of multi-channel acoustic signals. In traditional methods, the sampling clocks of the sound pickup units in each channel are not strictly synchronized with the clocks of the digital-to-analog conversion modules. This results in misaligned frame starting points and misaligned temporal features of each channel, affecting the consistency of multi-channel collaborative analysis.

[0003] Secondly, feature extraction is limited in dimension and has poor adaptability. Existing technologies rely heavily on time-frequency domain features (such as Mel-frequency cepstral coefficients (MFCCs)) but fail to incorporate temporal correlation features (such as first- and second-order differences of MFCCs), making it difficult to fully reflect the dynamic changes in the vocal tract state. Furthermore, feature extraction parameters (such as frame length and window function) are not dynamically adjusted based on the signal type (speech, white noise, or swept frequency signals), resulting in insufficient ability to capture features from different signal types.

[0004] Furthermore, the construction of the standard state feature space lacks multimodal adaptation. Existing methods typically train the standard feature space based on a single signal type (e.g., speech alone), failing to incorporate the features of multiple test tones, such as white noise and swept frequency signals. Furthermore, they fail to consider the spatial consistency constraints of adjacent channels. This results in weak model generalization and makes it difficult to accurately characterize the normal state of multi-channel collaboration.

[0005] Furthermore, anomaly detection lacks robustness and accuracy. Traditional methods often use the Euclidean distance of a single channel to determine anomalies, without incorporating the spatial weight of the channel's physical location or the rate of change within the time window. This can easily lead to misjudgments due to accidental interference (such as transient noise) or hardware fluctuations. Furthermore, these methods can only detect the presence of anomalies, failing to distinguish specific anomalies such as signal distortion, crosstalk, silence, and overload, making it difficult to trigger targeted control strategies.

[0006] In summary, the existing technology has defects in multi-channel synchronous acquisition, multi-dimensional feature extraction, standard space modeling, and anomaly detection and classification. There is an urgent need for a real-time monitoring method and system for multi-channel output status based on voiceprint recognition to improve the accuracy, robustness and practicality of monitoring. Summary of the Invention

[0007] In order to overcome the shortcomings of the existing technology, the purpose of the present invention is to provide a real-time monitoring method and system for multi-channel output status based on voiceprint recognition, which is used to solve the technical problems existing in the real-time monitoring of existing multi-channel output systems, such as insufficient synchronization acquisition accuracy, single feature extraction dimension, incomplete standard state space modeling, poor anomaly detection robustness and inability to accurately classify anomaly types.

[0008] In order to solve the above problems, the technical solutions adopted by the present invention are as follows: A method for real-time monitoring of multi-channel output status based on voiceprint recognition, comprising the following steps: The real-time sound signals output by each channel are synchronously collected through the array microphone group distributed at the multi-channel output terminal; The real-time sound signals of each channel are enhanced, framed, feature extracted, and fused in sequence to form a multi-dimensional voiceprint feature vector for each channel; Collect the voiceprint feature vectors of the standard test tone output by each channel, and obtain the standard state feature space of each channel through support vector machine training. The standard state feature space includes the feature mean vector, covariance matrix and confidence threshold of the normal output state; Through the multi-dimensional voiceprint feature vector and standard state feature space, a comprehensive anomaly index is obtained to identify the anomaly type; Generate control instructions based on the abnormality type, and transmit the abnormality type, occurrence time, and channel position to the monitoring terminal in real time through the bus communication unit.

[0009] Preferably, forming the multi-dimensional voiceprint feature vector of each channel includes: Pre-emphasis processing is used to enhance high-frequency components. The peak value of the signal after high-frequency enhancement does not exceed 90% of the quantization upper limit. Frame division and windowing to form analysis frames; Calculate the MFCC, fundamental frequency period, and formant frequency of each frame signal as basic voiceprint features; Based on time series analysis, ΔMFCC and ΔΔMFCC between adjacent frames are extracted as temporal correlation features; The basic voiceprint features are integrated with the time series correlation features to form a multi-dimensional voiceprint feature vector for each channel; Among them, the pre-emphasis coefficient is dynamically adjusted according to preset rules.

[0010] Preferably, when forming an analysis frame by dividing the frame and adding windows, the following steps are included: Synchronize the frame starting points of multi-channel sound signals; When a voiced segment is detected in the speech signal, the first frame length is used to capture the periodic characteristics; when an unvoiced segment, a white noise signal, or a swept frequency signal is detected, the second frame length is used to improve the time resolution; Use Hamming window for speech signals and use Heining window for swept frequency signals and white noise signals; Set the frame shift to form an overlapping frame structure.

[0011] Preferably, the calculation of basic voiceprint features includes: The windowed analysis frame is subjected to fast Fourier transform, nonlinear compression and discrete cosine transform, and the order coefficient is extracted as the MFCC main feature; Search for the peak position within the fundamental frequency range, take the period corresponding to the maximum peak as the fundamental frequency period for voiced segments, and mark the fundamental frequency period as an invalid value for unvoiced segments, white noise signals, and swept frequency signals; The prediction error filter coefficients are obtained through LPC analysis, and the candidate resonance peak frequencies are determined; the resonance peak bandwidth threshold and amplitude threshold are set to screen the valid resonance peaks, and the first several main resonance peak frequencies are retained; The center frequency and bandwidth parameters of the MFCC filter banks of each channel are unified, and the cross-channel consistency check of the fundamental frequency period and formant frequency calculation results is performed.

[0012] Preferably, forming the multi-dimensional voiceprint feature vector of each channel includes: Standardize the basic voiceprint features and time series correlation features respectively; The basic voiceprint feature vector and the corresponding time series correlation feature vector in the same time window are spliced ​​in chronological order to form an initial fusion vector; Dynamically adjust the weight coefficient of each feature dimension according to the current sound signal type; Calculate the Euclidean distance of the fusion vectors of each channel at the same time point. If the distance between any two channels exceeds the cross-channel feature difference threshold, the feature correction mechanism is triggered; The fusion vector is reduced in dimension through linear discriminant analysis to form the final multi-dimensional voiceprint feature vector.

[0013] Preferably, obtaining the standard state feature space of each channel includes: The multi-dimensional voiceprint feature vectors corresponding to the standard white noise signal, standard swept frequency signal, and standard speech signal are extracted respectively, and weighted coefficients are assigned based on the importance of the signal type to construct a multimodal training sample set; A cross-channel similarity regularization term is introduced into the SVM objective function to constrain the Euclidean distance in the standard state feature space of adjacent physical channels. The polynomial kernel function, radial basis kernel function and linear kernel function are used for the training samples dominated by speech signal, frequency sweep signal and white noise signal respectively; Part of the training samples were used as the validation set, and the SVM classification interval parameters were adjusted to make the upper limit of the 95% confidence interval of the Mahalanobis distance of the normal state samples as the final confidence threshold; The mean vector, covariance matrix and confidence threshold of each channel feature obtained through training are associated with the physical position coordinates according to the channel number to form a standard state feature space.

[0014] Preferably, when identifying the abnormal type, it includes: Calculate the Mahalanobis distance between the multi-dimensional voiceprint feature vector of each channel in the current frame and the corresponding standard state feature space; A spatial weight matrix is ​​constructed based on the physical position of the vocal tract, and a comprehensive anomaly index is calculated based on the distance change rate within the time window. If the comprehensive abnormality index exceeds the preset threshold, the difference vector between the multi-dimensional voiceprint feature vector of the current frame of the abnormal channel and the corresponding standard state feature space is extracted for abnormality identification.

[0015] Preferably, the Mahalanobis distance is calculated including: Obtain the multidimensional voiceprint feature vector of the current frame after dimensionality reduction by linear discriminant analysis; Call the feature mean vector and covariance matrix stored in the standard state feature space of the corresponding vocal channel; Combined with the weight coefficients of each feature dimension dynamically assigned when forming the multidimensional voiceprint feature vector, the multidimensional voiceprint feature vector, the feature mean vector, and the covariance matrix are weighted and adjusted to form a weighted feature vector, a weighted mean vector, and a weighted covariance matrix; Based on the weighted covariance matrix, the Mahalanobis distance between the weighted eigenvector and the weighted mean vector is calculated; A neighborhood relationship is established based on the physical position of the channels, and the deviation rate between the Mahalanobis distance of the current channel and the adjacent channels is calculated. If the deviation exceeds the set value, a smoothing correction is performed.

[0016] Preferably, the calculation of the comprehensive abnormality index includes: Get the physical position coordinates of each channel, calculate the Euclidean distance between any two channels, and identify neighboring channels; The Gaussian weight function is used to calculate the spatial weight to form a symmetric spatial weight matrix; Select a sliding time window and record the time series of Mahalanobis distances of each channel within the window; Obtain the Mahalanobis distance change rate of each channel in the frame interval time, and obtain the average Mahalanobis distance change rate in the sliding time window; For the current frame channel, based on the spatial weight matrix, obtain the sum of the weighted Mahalanobis distances between it and all neighboring channels; By defining the spatial anomaly component and the temporal anomaly component, a comprehensive anomaly index is generated.

[0017] A multi-channel output status real-time monitoring system based on voiceprint recognition, comprising: Synchronous acquisition module: including an array microphone group with a clock synchronization interface and a multi-channel digital-to-analog conversion module; Feature extraction module: integrated digital signal processor, equipped with adaptive pre-emphasis filter, frame windowing unit, Hamming window generation unit, Haining window generation unit, MFCC calculation unit, fundamental frequency period extraction unit, formant frequency extraction unit and time series feature extraction unit; Benchmark modeling module: Contains non-volatile memory for storing the standard state feature space of each channel and an SVM training engine for initial training; Anomaly detection module: It consists of a Mahalanobis distance calculation unit, a spatiotemporal correlation analysis unit, and anomaly recognition unit implemented by FPGA; Feedback control module: integrated microcontroller, equipped with control instruction generation unit, bus communication unit and actuator drive interface; Monitoring terminal: It has a human-computer interaction interface for displaying the status parameters of each channel and abnormal alarm information.

[0018] Compared with the prior art, the present invention has the following beneficial effects: (1) Improve the accuracy of multi-channel signal acquisition and feature alignment Through an array microphone group with a clock synchronization interface and a multi-channel digital-to-analog conversion module, strict synchronization is achieved between the pickup units of each channel and the digital-to-analog conversion clock, ensuring the alignment of the starting points of the multi-channel sound signal frames. This solves the problem of characteristic time dimension misalignment caused by clock asynchrony in traditional methods and significantly improves the consistency of multi-channel collaborative analysis.

[0019] (2) Enhance the comprehensiveness and adaptability of multi-dimensional voiceprint features A layered extraction strategy is employed, fusing time-frequency domain features (MFCC, fundamental frequency period, formant frequency) with dynamic temporal features (ΔMFCC, ΔΔMFCC). Frame length, window function, and feature weights are dynamically adjusted based on the signal type (speech, white noise, frequency sweep). Cross-channel consistency checks are also performed to correct feature deviations. This design enables the features to fully reflect the time-varying nature of the vocal channel state and the differences between multiple signal types, improving their adaptability and discriminability in complex scenarios.

[0020] (3) Optimizing the robustness and generalization ability of the standard state feature space A training sample set is constructed based on multimodal test tones (white noise, frequency sweeps, and speech). A cross-channel similarity regularization term is introduced to constrain the Euclidean distance between adjacent channel feature spaces. SVM kernel functions (polynomial, radial basis, and linear) are adaptively selected for different signal types. Combined with a dynamically calibrated confidence threshold (95% confidence interval upper limit), the standard state feature space can more accurately represent the normal state of multi-channel collaboration, effectively improving the model's generalization and interference resistance.

[0021] (4) Improve the robustness and classification accuracy of anomaly detection A comprehensive anomaly index is generated through weighted Mahalanobis distance calculation (combining feature dimension weights with spatial smoothing correction) and spatiotemporal correlation analysis (spatial weight matrix and temporal rate of change), avoiding the risk of misjudgment associated with single-dimensional detection. Furthermore, based on feature difference dimension analysis and rule matching (such as high-frequency band differences, adjacent channel feature similarity, and energy thresholds), it can accurately distinguish anomalies such as signal distortion, crosstalk, silence, and overload, providing a reliable basis for triggering targeted control strategies (such as gain adjustment, filtering activation, and switching to backup channels).

[0022] (5) Enhance system real-time performance and practicality Linear discriminant analysis (LDA) is used to reduce the dimensionality of fused features (retaining 85%-90% of the discriminant information). This, combined with an FPGA to implement Mahalanobis distance calculation and spatiotemporal correlation analysis, significantly reduces computational complexity and meets the low-latency requirements of real-time monitoring. Furthermore, the monitoring terminal displays the status of each channel and abnormality information in real time through a human-computer interaction interface, improving system operability and troubleshooting efficiency.

[0023] The present invention comprehensively improves the accuracy, robustness and practicality of multi-channel output status monitoring through multi-link technical optimization, and provides an effective solution for the stable operation and rapid fault location of the multi-channel system.

[0024] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a step diagram of a method for real-time monitoring of multi-channel output status according to an embodiment of the present invention; Figure 2 This is a flowchart of forming a multi-dimensional voiceprint feature vector for each channel according to an embodiment of the present invention; Figure 3 This is a flowchart of an embodiment of the present invention for identifying abnormal types; Figure 4 This is a module interaction diagram of a multi-channel output status real-time monitoring system according to an embodiment of the present invention.

[0026] Explanation of the accompanying figures: 600, multi-channel output status real-time monitoring system; 601, synchronous acquisition module; 602, feature extraction module; 603, benchmark modeling module; 604, anomaly detection module; 605, feedback control module; 606, monitoring terminal. DETAILED DESCRIPTION

[0027] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0028] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0029] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0030] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.

[0031] Example 1, see Figure 1 The present invention provides a method for real-time monitoring of multi-channel output status. Figure 1 The method for real-time monitoring of multi-channel output status based on voiceprint recognition shown includes the following steps: S100, synchronous multi-channel voiceprint acquisition: Using an array microphone group distributed across a multi-channel output terminal, the real-time sound signal outputted by each channel is synchronously acquired. The array microphone group is configured with independent pickup units equal to the number of channels, and the sampling clock of each pickup unit is synchronized with the clock of the digital-to-analog conversion module of the corresponding channel. S200, voiceprint feature layered extraction: The real-time sound signal of each channel is sequentially enhanced, framed, feature extracted, and fused to form a multi-dimensional voiceprint feature vector for each channel; S300, Multi-channel State Benchmark Modeling: During the system initialization phase, the voiceprint feature vectors of standard test tones (including standard white noise signals, standard swept frequency signals, and standard speech signals) output by each channel are collected. A support vector machine (SVM) is then trained to obtain the standard state feature space of each channel. The standard state feature space contains the feature mean vector, covariance matrix, and confidence threshold of the normal output state. S400, real-time anomaly detection: obtains a comprehensive anomaly index and identifies anomaly types through multi-dimensional voiceprint feature vectors and standard state feature space; S500 , dynamic feedback control: generating a control instruction according to the abnormality type, and transmitting the abnormality type, occurrence time, and channel position to the monitoring terminal 606 in real time via the bus communication unit.

[0032] Background description: In the real-time monitoring scenario of multi-channel output systems, accurately capturing the characteristics of the acoustic signals of each channel and performing effective analysis are the key to achieving status monitoring. Due to the complexity of the acoustic signal types in multi-channel systems (such as speech, white noise, swept frequency signals, etc.), and the significant differences in the time-frequency characteristics of different signal types, a single-dimensional feature is difficult to fully reflect the actual working status of the channel. In addition, when multiple channels work together, the signal acquisition of each channel needs to be synchronized, and the consistency of the characteristics between channels needs to be considered to avoid misjudgment due to synchronization errors or characteristic deviations. Based on this: See Figure 2 The flowchart of forming a multi-dimensional voiceprint feature vector for each channel is as follows: S210, pre-emphasis processing to enhance high frequency components; S220, dividing the frames into 20-40 ms analysis frames by windowing; S230, calculating the Mel-Frequency Cepstral Coefficient (MFCC), fundamental frequency period, and formant frequency of each frame signal as basic voiceprint features; S240, extracting MFCC dynamic first-order difference coefficients (ΔMFCC) and MFCC dynamic second-order difference coefficients (ΔΔMFCC) between adjacent frames as temporal correlation features based on time series analysis; S250: Fusing the basic voiceprint features with the time series correlation features to form a multi-dimensional voiceprint feature vector for each channel.

[0033] It is important to further explain the embodiment of the present invention. This embodiment extracts multidimensional voiceprint features in layers, integrates time-frequency domain information (such as Mel-frequency cepstral coefficients, fundamental frequency period, and formant frequency) with dynamic timing information (such as first- and second-order differential coefficients of MFCCs), and normalizes, fuses, and optimizes dimensionality reduction to form a multidimensional feature vector that comprehensively characterizes the state of the vocal tract. This process enhances the adaptability of the features to different signal types, improving the accuracy and robustness of subsequent benchmark modeling and anomaly detection, thereby meeting the demand for high-precision feature analysis in multi-channel real-time monitoring.

[0034] Background description: In the process of multi-channel sound signal acquisition and feature extraction, the attenuation of high-frequency components is one of the key issues affecting the accuracy of voiceprint features. Due to factors such as microphone pickup characteristics, signal transmission loss and environmental noise, the energy of the high-frequency part of the sound signal (such as the 8kHz-20kHz frequency band) is usually lower than that of the mid- and low-frequency bands, resulting in subsequent analysis of mid- and high-frequency features (such as the details of clear consonants in speech, the high-frequency response of sweep signals, etc.) It is difficult to be effectively captured, which may cause feature loss or distortion. For multi-channel systems, the acquisition characteristics of the independent pickup units of each channel may be different, and the degree of high-frequency attenuation is inconsistent. If targeted compensation is not performed, the inconsistency of the features between channels will be further aggravated, affecting the accuracy of multi-channel collaborative monitoring. Based on this: In a possible embodiment, the pre-emphasis processing to enhance high-frequency components includes: S211. Calculate the high-frequency energy ratio of the real-time sound signal collected by the independent sound pickup unit of each channel, where the high-frequency energy ratio is the ratio of the energy in the 8kHz-20kHz frequency band to the total energy in the entire frequency band in the signal; S212, based on the clock synchronization characteristics of multi-channel synchronous acquisition, an adaptive pre-emphasis filter is used, whose transfer function is H(z)=1-α(z)˙z -1 , where z -1 is the unit delay operator in discrete-time signal processing, and the pre-emphasis coefficient α(z) is dynamically adjusted according to the following rules: When the high-frequency energy of a single channel accounts for less than 15%, α(z) is set to 0.95; When the high-frequency energy of a single channel accounts for 15%-30%, α(z) is set to 0.90; When the high-frequency energy of a single channel accounts for more than 30%, α(z) is set to 0.85; S213. Through the 24-bit high-precision quantization characteristics of the digital-to-analog conversion module, the dynamic range of the pre-emphasized signal is compressed to ensure that the peak value of the signal after high-frequency enhancement does not exceed 90% of the quantization upper limit, avoiding overload distortion.

[0035] In the embodiments of the present invention, it is important to further explain that pre-emphasis, as the primary step in hierarchical voiceprint feature extraction, aims to adaptively enhance high-frequency components and increase the energy proportion of high-frequency signals, ensuring that subsequent steps such as framing and feature extraction fully utilize this high-frequency information. Given the characteristics of synchronous multi-channel acquisition (where the clock of each pickup unit is synchronized with the corresponding channel's digital-to-analog conversion module), pre-emphasis dynamically adapts the high-frequency attenuation of each channel (by calculating the high-frequency energy proportion and adjusting the pre-emphasis coefficient) to avoid over- or under-compensation issues caused by uniform parameters. Furthermore, combined with the 24-bit high-precision quantization of the digital-to-analog conversion module, the dynamic range compression step prevents high-frequency signal peaks from exceeding the quantization limit after high-frequency enhancement, thus preventing overload distortion.

[0036] Background description: Acoustic signals are inherently short-term stationary (e.g., voiced segments of speech are periodic, and the frequency of swept-frequency signals varies over time). Directly processing long-term signals will obscure local time-varying characteristics. Therefore, continuous signals need to be segmented into short time frames. However, in this invention, the design of framing and windowing must take into account the multi-channel synchronization characteristics, signal type differences, and feature continuity requirements. Based on this: In a possible embodiment, when framing and windowing are performed to form 20-40 ms analysis frames, the following steps are included: S221. Align the sampling start time of the independent sound pickup unit of each channel based on the synchronous clock signal of the digital-to-analog conversion module to ensure synchronization of the frame starting points of the multi-channel sound signal; S222. Dynamically adjust the frame length according to the acoustic signal type: when a voiced segment is detected in the speech signal, use a frame length of 35-40 ms to capture periodic features; when an unvoiced segment, a white noise signal, or a swept frequency signal is detected, use a frame length of 20-25 ms to improve time resolution; S223, using adaptive switching of multiple window functions to add windows: using a Hamming window for the acoustic signal frames containing speech signals to suppress boundary effects; using a Haining window for the acoustic signal frames of swept frequency signals and white noise signals to optimize the spectrum main lobe width and suppress spectrum leakage at the frame boundaries; S224. Set the frame shift to 60%-75% of the frame length to form an overlapping frame structure, ensure the continuity of features between adjacent frames, and avoid feature loss caused by non-overlapping framing.

[0037] What needs to be further explained in the embodiment of the present invention is that this embodiment divides the continuous signal into short time frames (20-40ms) so that the signal in each frame is approximately smooth, which is convenient for subsequent spectrum analysis and feature extraction, and aligns the starting point of the frame through the synchronous clock signal of the analog conversion module, so that the features of each channel are aligned in the time dimension, ensuring the consistency of multi-channel collaborative analysis (S221).

[0038] The time-frequency characteristics of different acoustic signal types vary significantly: voiced speech segments have obvious periodicity (such as the fundamental frequency period of vocal cord vibration), requiring a longer frame length (35-40ms) to fully capture the periodic characteristics; while unvoiced speech segments (such as fricative sounds), white noise (wideband random signals) and swept frequency signals (linear frequency changes) have more prominent time-varying characteristics and require a shorter frame length (20-25ms) to improve temporal resolution and avoid feature blurring caused by long frames (S222).

[0039] Framing introduces abrupt changes at signal boundaries (e.g., the signal amplitude at either end of a frame suddenly drops from non-zero to zero), causing sidelobe leakage (spectral energy spreading to adjacent frequencies) during spectral analysis, affecting feature accuracy. Different window functions have different effects on the mainlobe width and sidelobe suppression: the Hamming window is better at suppressing boundary effects in speech signals (preserving periodicity), while the Haining window is more effective at optimizing the mainlobe width of swept / white noise signals (reducing leakage) (S223).

[0040] If non-overlapping framing is used (frame shift = frame length), there is no information overlap between adjacent frames, which may lead to feature discontinuity in the temporal dimension (for example, key information is lost when the voice pitch period changes across frames). Setting the frame shift to 60%-75% of the frame length to form an overlapping framing structure ensures that adjacent frames share some signal, enhances feature temporal continuity, and avoids feature loss caused by excessively large framing intervals (S224).

[0041] The dynamic design of framing and windowing (synchronous alignment, frame length adjustment, window function switching, overlapping framing) is to adapt to the multi-channel synchronization characteristics and the time-frequency characteristics of different sound signals.

[0042] Background description: In the process of multi-channel voiceprint feature extraction, it is necessary to capture the core characteristics of the acoustic signal from different dimensions to meet the requirements of multi-channel real-time monitoring for feature comprehensiveness, discrimination and robustness. Based on this: In a possible embodiment, the calculation of the Mel-Frequency Cepstral Coefficients (MFCC), fundamental frequency period, and formant frequency of each frame signal as basic voiceprint features includes: S231, MFCC coefficient calculation: Perform a 512-1024-point fast Fourier transform (FFT) on the windowed analysis frame to obtain a spectrogram. Perform nonlinear compression on the spectral energy using a 24-32 MFCC filter bank with an equimel-scale distribution. The center frequency of the MFCC filter bank covers 100 Hz-8 kHz. Take the natural logarithm of the filtered energy value, then perform a 12-16th-order discrete cosine transform (DCT) to extract the 2nd-13th-order coefficients as the main MFCC features. S232, fundamental frequency period extraction: using the autocorrelation method to calculate the normalized autocorrelation function, searching for the peak position within the fundamental frequency range of 50 Hz-500 Hz; combining the voiced and unvoiced sound determination results in the framing step, taking the period corresponding to the maximum peak value as the fundamental frequency period for the voiced sound segment, and marking the fundamental frequency period as an invalid value for the unvoiced sound segment, white noise signal, and swept frequency signal; S233, Formant Frequency Detection: 12-16th order linear predictive coding (LPC) analysis is used to obtain prediction error filter coefficients. The roots of the LPC polynomial are solved to determine candidate formant frequencies. The formant bandwidth threshold (≤500 Hz) and amplitude threshold (≥60% of the forward prediction error energy) are set to screen for valid formants, retaining the top 3-5 major formant frequencies. S234, cross-channel feature calibration: Based on the clock alignment characteristics of multi-channel synchronous acquisition, the center frequency and bandwidth parameters of the MFCC filter group of each channel are unified; the cross-channel consistency of the fundamental frequency period and resonance peak frequency calculation results are checked, and the channel features with deviations exceeding 5% are marked as values ​​to be corrected.

[0043] What needs to be further explained in the embodiments of the present invention is that for different signal types such as speech, white noise, and swept frequency in a multi-channel system, MFCC can effectively capture their spectral envelope characteristics (such as the resonance peak distribution of speech and the frequency change trend of the swept frequency signal), and has strong robustness to environmental noise and signal distortion. It is the core dimension of the basic voiceprint feature.

[0044] Fundamental frequency period is a key prosodic feature of speech signals, particularly in voiced segments (such as vowels), where periodicity is evident. For multi-channel speech signals, the fundamental frequency period can reflect the speaker's pitch characteristics or the degree of signal distortion. However, in non-periodic signals such as unvoiced segments and white noise, the fundamental frequency period is meaningless (marked as an invalid value), thus avoiding the introduction of redundant features. Extracting the fundamental frequency period through the autocorrelation method combined with voiced / unvoiced speech determination can accurately distinguish valid and invalid features, improving feature quality.

[0045] Linear Predictive Coding (LPC) analysis extracts formant frequencies, capturing the peak frequency of vocal tract resonances and reflecting the content characteristics or degree of distortion of the speech signal (for example, formant shifts may indicate signal distortion). For non-speech signals (such as swept frequency signals), formant frequencies do not directly correspond to speech content but can reflect the distribution of the signal's frequency components, helping to identify signal type or anomalies (such as frequency shifts).

[0046] By unifying the MFCC filter bank parameters (center frequency, bandwidth) and performing cross-channel consistency checks on the fundamental frequency and formant frequency (deviations exceeding 5% are marked for correction), we can eliminate feature inconsistencies caused by hardware differences, ensure the comparability of channel features, and provide a reliable foundation for subsequent multi-channel collaborative analysis (such as spatial correlation in anomaly detection).

[0047] Background: Basic voiceprint features (such as MFCC coefficients, fundamental frequency period, and formant frequency) and temporal correlation features (ΔMFCC, ΔΔMFCC) have significant differences in physical dimensions (for example, MFCC is a dimensionless coefficient, while fundamental frequency period is a time unit). Direct concatenation will lead to an imbalance in the importance of different dimensions in the feature space (features with larger dimensions may dominate the analysis results).

[0048] The acquisition and framing of multi-channel sound signals must be strictly synchronized (clock alignment of the digital-to-analog conversion module), but the extraction paths of basic features (reflecting the spectral characteristics of the current frame) and timing features (reflecting the dynamic changes of adjacent frames) are different, which may lead to misalignment of the time dimension.

[0049] The feature contributions of different sound signal types (speech, white noise, and frequency sweep) vary significantly: the fundamental frequency period and formant of speech signals are key to distinguishing content (such as the difference between vowels and consonants), the MFCC of white noise and its differential features can better reflect the random characteristics of the signal (such as frequency band distribution), and the ΔMFCC (frequency change rate) of the frequency sweep signal is key to determining whether its linear frequency sweep is normal.

[0050] In a multi-channel system, channels at adjacent physical locations should have similar acoustic signal characteristics (such as the multi-channel output of the same sound source). If the characteristics of a channel differ significantly from those of adjacent channels (possibly due to microphone failure, signal crosstalk, etc.), subsequent abnormality detection may be misjudged.

[0051] The initial fusion vector has a high dimension (24-26 dimensional MFCC and difference + fundamental frequency + 3-5 resonance peaks). Directly using it for support vector machine (SVM) modeling or anomaly detection will significantly increase the computational complexity and may reduce the model generalization ability due to feature redundancy.

[0052] Based on this: In a possible embodiment, forming a multi-dimensional voiceprint feature vector for each channel includes: S251. Feature normalization: Standardize the basic voiceprint features (Mel-frequency cepstral coefficients (MFCC), fundamental frequency period, formant frequency) and time-series correlation features (ΔMFCC, ΔΔMFCC) respectively: calculate the global mean and standard deviation of each feature dimension, and convert the feature values ​​into a standard normal distribution with a mean of 0 and a standard deviation of 1 to eliminate the differences in different feature dimensions; S252, Time Dimension Alignment: Based on the synchronization characteristics of the frame starting point of multi-channel synchronous acquisition, the basic voiceprint feature vector (12-13 dimensional MFCC main features + fundamental frequency period + 3-5 formant frequencies) in the same time window is spliced ​​with the corresponding time-series correlation feature vector (12-13 dimensional ΔMFCC + 12-13 dimensional ΔΔMFCC) in chronological order to form an initial fusion vector with the dimension of (24-26 dimensional MFCC and differential features + fundamental frequency period + 3-5 formant frequencies); S253, Dynamic Weight Allocation: Dynamically adjust the weight coefficients of each feature dimension based on the current sound signal type (speech / white noise / swept frequency signal): for speech signals, enhance the weights of the fundamental frequency period and formant frequency (weight coefficient 0.8-1.2), for white noise signals, enhance the weights of MFCC and its differential features (weight coefficient 0.9-1.1), and for swept frequency signals, enhance the weight of ΔMFCC (weight coefficient 1.0-1.3); S254, cross-channel consistency check: Based on the multi-channel clock synchronization characteristics, the Euclidean distance of the fusion vectors of each channel at the same time point is calculated. If the distance between any two channels exceeds the cross-channel feature difference threshold (preset to be twice the standard deviation of the basic voiceprint feature), the feature correction mechanism is triggered, and the abnormal channel is interpolated and corrected using the mean of the fusion vectors of adjacent normal channels; S255, feature dimensionality reduction optimization: Reduce the dimensionality of the fusion vector through linear discriminant analysis (LDA), retaining 85%-90% of the discriminant information to form the final 15-20 dimensional multi-dimensional voiceprint feature vector.

[0053] It should be further explained in the embodiments of the present invention that feature normalization (S251) eliminates dimensional differences through standardization processing (mean is 0, standard deviation is 1), ensures that each feature dimension has equal weight in subsequent modeling and detection, and improves the stability of the feature space.

[0054] Time dimension alignment (S252) synchronizes the starting point of the frame division and splices the basic feature vectors (such as 12-13-dimensional MFCC + fundamental frequency + formant) in the same time window with the corresponding time series feature vectors (ΔMFCC, ΔΔMFCC) in chronological order to form a coherent time series feature. This avoids feature discontinuity caused by time misalignment and ensures the timeliness of multi-channel collaborative analysis.

[0055] Dynamic weight allocation (S253) adjusts the weight of each feature dimension (such as the fundamental frequency weight for speech enhancement and the MFCC weight for white noise enhancement) according to the signal type, highlighting the discriminative power of key features, avoiding interference from secondary features, and improving the adaptability of features to different signal types.

[0056] Cross-channel consistency check (S254) calculates the Euclidean distance of the fusion vectors of each channel and performs interpolation correction (using the average of adjacent normal channels) on abnormal channels (differences exceeding the threshold) to ensure the spatial consistency of multi-channel features and enhance the robustness of collaborative monitoring.

[0057] Feature dimensionality reduction optimization (S255) retains 85%-90% of the discriminant information through linear discriminant analysis (LDA), compressing the dimension to 15-20 dimensions, while maintaining feature discriminability while improving computing efficiency to meet the low-latency requirements of real-time monitoring.

[0058] The process of forming a multi-dimensional voiceprint feature vector (normalization, time alignment, weight distribution, consistency check, and dimensionality reduction) is designed to address challenges such as feature dimension differences, time misalignment, signal type adaptation, spatial inconsistency, and computational efficiency in multi-channel systems, and ultimately generate a feature vector that can comprehensively, accurately, and efficiently characterize the state of each channel.

[0059] Background description: In the real-time monitoring scenario of multi-channel output systems, establishing an accurate and robust standard state feature space is the core prerequisite for achieving anomaly detection. Since the normal working state of a multi-channel system needs to cover a variety of typical signal types (such as white noise, swept frequency signals, and voice signals), and each channel may have a natural discreteness in feature distribution due to differences in physical location and hardware characteristics, it is difficult to fully characterize the normal state by relying solely on the features of a single signal type or an independent channel. In addition, as a classification model, the performance of the support vector machine (SVM) is significantly affected by the multimodal characteristics of the training samples, the selection of kernel functions, and the setting of confidence thresholds. If it is not optimized for multiple signal types, it may lead to blurred classification boundaries or sensitivity to environmental noise. At the same time, when multiple channels work together, the normal states of adjacent channels should have a certain spatial consistency, otherwise misjudgment may be caused due to the discreteness of the feature space. Based on this: In the above step S300, when the standard state feature space of each channel is obtained by support vector machine (SVM) training, it includes: S310, Multi-type Standard Test Tone Feature Fusion: Extract multi-dimensional voiceprint feature vectors corresponding to standard white noise signals, standard swept frequency signals, and standard speech signals, assign weighting coefficients based on the importance of signal types (0.3 for white noise signals, 0.4 for swept frequency signals, and 0.3 for speech signals), and construct a multimodal training sample set; S320, cross-channel consistency constraint: Introducing a cross-channel similarity regularization term into the SVM objective function, constraining the Euclidean distance in the standard state feature space of adjacent physical channels to not exceed 1.5 times the standard deviation of the basic voiceprint feature, thereby enhancing the consistency of multi-channel collaborative work; S330, Adaptive kernel function selection: For training samples dominated by speech signals, a polynomial kernel function (kernel parameter d = 3) is used; for training samples dominated by swept frequency signals, a radial basis kernel function (σ is the mean of the feature space variance) is used; for training samples dominated by white noise signals, a linear kernel function is used to optimize the classification boundaries of different signal types; S340, Dynamic Calibration of Confidence Threshold: Using 10% of the training samples as the validation set, by adjusting the SVM classification interval parameters, the upper limit of the 95% confidence interval of the Mahalanobis distance of the normal state samples is used as the final confidence threshold to ensure the robustness of the feature space to environmental noise; S350, feature space storage: The trained feature mean vectors, covariance matrices, and confidence thresholds of each channel are stored in a non-volatile memory by associating the channel numbers with the physical position coordinates to form a traceable standard state feature space.

[0060] What needs to be further explained in the embodiments of the present invention is that this embodiment constructs a standard state feature space including feature mean, covariance matrix and confidence threshold through steps such as fusion of multi-type test sound features, cross-channel consistency constraints, adaptive selection of kernel functions and dynamic calibration of confidence thresholds, which can effectively improve the accuracy and robustness of subsequent anomaly detection.

[0061] Background description: In real-time monitoring of multi-channel output, simply detecting abnormal conditions (such as an excessive Mahalanobis distance) cannot meet actual application needs. The key is to identify the specific type of abnormality (such as signal distortion, crosstalk, silence, overload, etc.) in order to trigger targeted control strategies. Because the manifestations of abnormalities in multi-channel systems are complex and diverse: signal distortion may be caused by equipment aging or transmission loss, manifesting as attenuation of high-frequency components; crosstalk may be caused by electromagnetic coupling between channels, manifesting as mixing of adjacent channel signals; silence may be caused by line disconnection or signal source failure, manifesting as abnormally low energy; overload may be caused by excessive gain settings, manifesting as signal peak exceeding the limit. A single abnormality detection indicator (such as the Mahalanobis distance) can only reflect the degree of feature deviation and cannot directly distinguish the type of abnormality. At the same time, when multiple channels work together, abnormalities may have spatiotemporal correlations (for example, crosstalk may simultaneously affect the feature timing of adjacent channels), requiring a comprehensive analysis combining the spatial weight matrix (reflecting the physical position relationship of the channels) and the time change rate (reflecting the development trend of the abnormality). Based on this: See Figure 3 The flowchart of identifying the abnormality type, in the above step S400, when identifying the abnormality type, includes: S410, calculating the Mahalanobis distance between the multi-dimensional voiceprint feature vector of each channel of the current frame and the corresponding standard state feature space; S420, performing spatiotemporal correlation analysis on Mahalanobis distances of multiple channels: constructing a spatial weight matrix based on the physical positions of the channels, and calculating a comprehensive anomaly index based on the distance change rate within the time window; S430: If the comprehensive abnormality index exceeds a preset threshold, abnormality type determination is triggered: extracting the voiceprint features of the abnormal channel for abnormality identification, and distinguishing at least one abnormality type among signal distortion, crosstalk, silence, and overload.

[0062] It should be further explained in the embodiments of the present invention that this embodiment accurately identifies the type of anomaly by calculating a comprehensive anomaly index and combining feature difference dimensional analysis (such as differences in MFCC, fundamental frequency, and resonance peaks) with rule matching. This is the core prerequisite for achieving dynamic feedback control (such as adjusting gain and starting filtering), and directly affects the operational reliability and fault handling efficiency of the multi-channel system.

[0063] Background description: In the real-time monitoring of multi-channel output status, accurately quantifying the degree of deviation of the current voiceprint features from the standard state is a key link in anomaly detection. Since the voiceprint features of multi-channel systems have high dimensions (15-20 dimensions), there is correlation between features (such as the temporal correlation between MFCC and ΔMFCC), and different signal types (speech / white noise / sweep frequency) have different contributions to the features, the traditional Euclidean distance cannot effectively reflect the distribution characteristics of the feature space (such as covariance information), which can easily lead to abnormal misjudgment. In addition, when multiple channels work together, the characteristics of each channel may have accidental deviations due to hardware differences (such as different microphone sensitivities) or environmental interference (such as local noise). Directly using the original features to calculate the distance may amplify these non-systematic errors. Based on this: In a possible embodiment, the Mahalanobis distance is calculated, including: S411. Based on the clock alignment characteristics of multi-channel synchronous acquisition, obtain a 15-20 dimensional multi-dimensional voiceprint feature vector of the current frame after dimensionality reduction by linear discriminant analysis (LDA); S412, calling the feature mean vector and covariance matrix stored in the standard state feature space of the corresponding channel (from the non-volatile memory stored in S350); S413. Combining the weight coefficients of each feature dimension dynamically allocated in S253, weighted adjustment is performed on the multi-dimensional voiceprint feature vector, feature mean vector, and covariance matrix to form a weighted feature vector, a weighted mean vector, and a weighted covariance matrix; S414. Calculate the Mahalanobis distance between the weighted eigenvector and the weighted mean vector based on the weighted covariance matrix; S415. Perform spatial consistency check on the Mahalanobis distance of multiple channels: Build a neighborhood relationship based on the physical location of the channels, calculate the deviation rate between the Mahalanobis distance of the current channel and the adjacent channels, and if the deviation exceeds 1.5 times the standard deviation of the basic voiceprint feature, use the mean of the adjacent channel distances to smooth the Mahalanobis distance of the current channel to enhance the robustness of multi-channel collaborative detection.

[0064] Further explanation is required regarding the embodiments of the present invention. First, based on the clock alignment characteristics of multi-channel synchronous acquisition, this embodiment obtains feature vectors after LDA dimensionality reduction (retaining 85%-90% of the discriminant information) to balance feature dimensionality and computational efficiency. Second, the standard state feature space parameters (mean vector, covariance matrix) stored during the system initialization phase are called to ensure the accuracy and traceability of the benchmark. Third, dynamic weight allocation (adjusting feature weights based on signal type) is combined to highlight the discriminative power of key features. Fourth, a spatial consistency check (using the mean distance between adjacent channels to correct for abnormal channels) is required to eliminate the impact of accidental deviations on the detection results. Through these steps, the Mahalanobis distance can more accurately reflect the degree of deviation of the feature vector from the standard distribution, providing a reliable basis for subsequent calculation of the comprehensive anomaly index and identification of anomaly types.

[0065] Background description: In real-time monitoring of multi-channel output status, judging anomalies only by the Mahalanobis distance of a single channel (the degree to which the feature deviates from the standard state) may lead to misjudgment due to accidental interference (such as instantaneous noise, hardware fluctuations), or missed detection due to ignoring the spatiotemporal correlation of anomalies (such as coordinated anomalies in adjacent channels, and the trend of anomalies developing over time). For example, a slight increase in the Mahalanobis distance of a channel may be accidental interference, but if the Mahalanobis distance of its adjacent channels increases synchronously, it is more likely to be a systematic anomaly; a sudden surge in the Mahalanobis distance of a channel may be a momentary fluctuation, but if it continues to rise, it is more likely to be a device failure. Based on this: In a possible embodiment, the calculation of the comprehensive abnormality index includes: S421, spatial weight matrix construction: Get the physical position coordinates of each channel and calculate the Euclidean distance d_ij between any two channels i and j; Define the neighborhood range as d_max (take the average of all channel spacing), when d_ij≤d_max, the two channels are neighboring channels; The Gaussian weight function is used to calculate the spatial weight w_ij=exp(-d_ij 2 / (2σ 2 )), where σ is the neighborhood influence factor (taken as d_max / 2), forming a symmetric spatial weight matrix W, and the matrix element W(i,j)=w_ij; S422, time change rate calculation: Select a sliding time window of length N (N ≥ 5) and record the time series of Mahalanobis distances of each channel in the window D_i(t) = [d_i(t-N+1),d_i(t-N+2),…,d_i(t)]; Get the Mahalanobis distance change rate of each channel in the frame interval time, and get the average Mahalanobis distance change rate v_i_avg in the sliding time window; S423. Calculation of comprehensive abnormality index: For the current frame channel i, based on the spatial weight matrix W, obtain the weighted Mahalanobis distance S_i between it and all neighboring channels; Define the spatial anomaly component: A_i=|d_i(t)-S_i| / max(S_i,d_i(t)); Define the temporal anomaly component: B_i = |v_i_avg| / v_threshold (v_threshold is the preset rate of change threshold); Generate a comprehensive anomaly index: E_i=α×A_i + β×B_i (α+β=1, α is the spatial weight coefficient and is 0.6, β is the temporal weight coefficient and is 0.4).

[0066] It needs to be further explained in the embodiment of the present invention that this embodiment generates a comprehensive anomaly index by combining information of spatial and temporal dimensions, thereby achieving a more comprehensive anomaly assessment.

[0067] First, the spatial dimension: In a multi-channel system, channels in adjacent physical locations should normally exhibit a certain degree of synergy due to the common signal source or similar environmental interference (e.g., multi-channel output from the same source). If the Mahalanobis distance of a channel differs significantly from that of adjacent channels (e.g., due to signal congestion caused by crosstalk), this embodiment quantifies this spatial inconsistency using a spatial weight matrix (building neighborhood relationships based on the physical locations of the channels) to avoid misjudgment in a single dimension.

[0068] Second, the temporal dimension: Anomalies often evolve (for example, distortion gradually increases due to equipment aging). Analyzing only the Mahalanobis distance of the current frame cannot reflect the anomaly's development trend. This embodiment calculates the rate of change (such as the average rate of change) of the Mahalanobis distance over a sliding time window. This allows distinguishing between instantaneous fluctuations (such as sudden noise) and persistent anomalies (such as equipment failure), improving detection robustness.

[0069] The implementation steps of the comprehensive anomaly index (construction of spatial weight matrix, calculation of temporal change rate, and generation of comprehensive index) address the limitations of single-dimensional analysis by integrating spatial synergy and temporal evolution.

[0070] Background description: In the real-time monitoring of multi-channel output status, when the comprehensive anomaly index exceeds the threshold, indicating the presence of an anomaly, accurately identifying the specific type of anomaly (such as signal distortion, crosstalk, silence, overload) is the core prerequisite for achieving targeted regulation. Due to the significant differences in the causes and manifestation characteristics of different anomaly types: signal distortion may be caused by equipment aging or transmission loss, resulting in attenuation of high-frequency components, crosstalk may cause signal mixing due to electromagnetic coupling between channels, silence may be caused by line disconnection or signal source failure, resulting in energy loss, and overload may be caused by excessive gain setting, resulting in signal peak exceeding the limit. These specific anomaly types cannot be directly distinguished by the comprehensive anomaly index (reflecting the overall degree of deviation), and detailed analysis must be conducted in combination with the dimensional differences of voiceprint features and rule matching. Based on this: In a possible embodiment, extracting the voiceprint features of the abnormal vocal channel for abnormality identification includes: S431, feature difference dimensional analysis: extract the difference vector between the multi-dimensional voiceprint feature vector of the current frame of the abnormal channel and the corresponding standard state feature space, and calculate the dimensional difference values ​​of the MFCC main feature, ΔMFCC, ΔΔMFCC, fundamental frequency period and formant frequency respectively; S432, exception type rule matching: Signal distortion judgment: If the difference value of the first high-frequency band of the MFCC main feature (corresponding to the Mel filter bank center frequency > 5kHz) exceeds 2.5 times the mean of the high-frequency band of the standard feature space, and the formant frequency offset is > 10% (relative to the standard state formant frequency), it is judged as signal distortion; Crosstalk determination: If the Euclidean distance between the MFCC main feature of the abnormal channel and the MFCC main feature of the current frame of the adjacent channel is less than 30% of the Mahalanobis distance between the channel and its own standard feature, and the temporal correlation between the ΔMFCC feature and the ΔMFCC feature of the adjacent channel is greater than 0.7, it is determined to be crosstalk; Silence determination: If the total energy of the sound signal in the current frame (calculated by integrating the full-band FFT energy) is lower than the minimum energy threshold for the standard state (5% of the energy of the standard test tone), and the fundamental frequency period is marked as an invalid value (unvoiced segment determination) for more than three consecutive frames, it is determined to be silence; Overload determination: If the peak value of the pre-emphasized signal exceeds 90% of the quantization upper limit (monitored by S213 dynamic range compression), and the energy proportion of the second high-frequency band (>8kHz) of the MFCC main feature decreases by >40% compared to the standard state, it is determined to be overloaded.

[0071] It is important to further explain in the embodiments of the present invention that different anomaly types exhibit specific variations in different dimensions of voiceprint features. For example, signal distortion may be reflected by differences in the high-frequency band of MFCCs and formant shifts, crosstalk may be reflected by the similarity of MFCCs of adjacent channels and the temporal correlation of ΔMFCCs, silence may be identified by low energy and invalid fundamental frequency period markers, and overload may be detected by exceeding signal peak limits and decreasing high-frequency energy. Therefore, it is necessary to extract multidimensional voiceprint features (such as main MFCC features, ΔMFCC, fundamental frequency period, etc.) of the abnormal channel and analyze the difference values ​​in each dimension to provide data support for rule matching.

[0072] Abnormal behavior in multichannel systems exhibits predictable patterns (e.g., a strong correlation between excessive high-band differences and distortion, and a correlation between similarity in adjacent channel features and crosstalk). By setting clear judgment rules (e.g., high-band difference thresholds and adjacent channel feature distance thresholds), feature differences can be converted into specific anomaly labels, avoiding the uncertainty associated with subjective judgment.

[0073] This embodiment extracts the voiceprint features of abnormal vocal channels and performs implementation steps for abnormality identification (feature difference dimension analysis and rule matching). By mining the specific differences in feature dimensions and combining empirical rules, accurate classification of abnormal types is achieved.

[0074] In the above step S500, when generating a control instruction according to the exception type, it includes: Adjust the gain coefficient of the digital-to-analog conversion module for the distorted channel; Enable adaptive notch filtering for the crosstalk channel; Trigger backup channel switching for silent channels; Implement limiting protection for overloaded channels.

[0075] Example 2, see Figure 4 The present invention provides a multi-channel output state real-time monitoring system module interaction diagram. Figure 4 The system 600 for real-time monitoring of multi-channel output status based on voiceprint recognition includes: a synchronous acquisition module 601 , a feature extraction module 602 , a benchmark modeling module 603 , an anomaly detection module 604 , a feedback control module 605 and a monitoring terminal 606 .

[0076] The synchronous acquisition module 601 includes an array microphone group with a clock synchronization interface and a multi-channel digital-to-analog conversion module; wherein the array microphone group is configured with independent pickup units consistent with the number of channels, the sampling rate of the digital-to-analog conversion module is ≥48kHz, and the quantization bit number is ≥24bit.

[0077] Feature extraction module 602: an integrated digital signal processor (DSP), equipped with an adaptive pre-emphasis filter, a frame windowing unit, a Hamming window generation unit, a Haining window generation unit, an MFCC calculation unit, a fundamental frequency period extraction unit, a formant frequency extraction unit, and a time series feature extraction unit.

[0078] The reference modeling module 603 includes a non-volatile memory for storing the standard state feature space of each channel and an SVM training engine for initialization training.

[0079] Anomaly detection module 604: consists of a Mahalanobis distance calculation unit, a spatiotemporal correlation analysis unit, and anomaly identification unit implemented by FPGA.

[0080] Feedback control module 605: an integrated microcontroller (MCU), equipped with a control instruction generation unit, a bus communication unit and an actuator drive interface.

[0081] Monitoring terminal 606: has a human-computer interaction interface for displaying the status parameters of each channel and abnormal alarm information.

[0082] In one possible embodiment, the transfer function of the adaptive pre-emphasis filter is H(z)=1-α(z)˙z -1 , where z -1 is the unit delay operator in discrete-time signal processing, and the pre-emphasis coefficient α(z) is dynamically adjusted according to the following rules: When the high-frequency energy of a single channel accounts for less than 15%, α(z) is set to 0.95; When the high-frequency energy of a single channel accounts for 15%-30%, α(z) is set to 0.90; When the high-frequency energy of a single channel accounts for more than 30%, α(z) is set to 0.85.

[0083] In a possible embodiment, the MFCC calculation unit is provided with 24-32 MFCC filter groups with equal Mel-scale distribution, which are used to perform nonlinear compression on spectral energy. The center frequency of the MFCC filter group covers 100 Hz-8 kHz.

[0084] The above embodiments are only preferred embodiments of the present invention and cannot be used to limit the scope of protection of the present invention. Any non-substantial changes and replacements made by technicians in this field on the basis of the present invention fall within the scope of protection required by the present invention.

Claims

1. A method for real-time monitoring of multi-channel output status based on voiceprint recognition, characterized in that: The following steps are involved: The real-time sound signals output by each channel are synchronously collected through the array microphone group distributed at the multi-channel output terminal; The real-time sound signals of each channel are enhanced, framed, feature extracted, and fused in sequence to form a multi-dimensional voiceprint feature vector for each channel; Collect the voiceprint feature vectors of the standard test tone output by each channel, and obtain the standard state feature space of each channel through support vector machine training. The standard state feature space includes the feature mean vector, covariance matrix and confidence threshold of the normal output state; Through the multi-dimensional voiceprint feature vector and standard state feature space, a comprehensive anomaly index is obtained to identify the anomaly type; Generate control instructions based on the abnormality type, and transmit the abnormality type, occurrence time, and channel position to the monitoring terminal in real time through the bus communication unit.

2. The method according to claim 1, characterized in that When forming the multi-dimensional voiceprint feature vector of each channel, it includes: Pre-emphasis processing is used to enhance high-frequency components. The peak value of the signal after high-frequency enhancement does not exceed 90% of the quantization upper limit. Frame division and windowing to form analysis frames; Calculate the MFCC, fundamental frequency period, and formant frequency of each frame signal as basic voiceprint features; Based on time series analysis, ΔMFCC and ΔΔMFCC between adjacent frames are extracted as temporal correlation features; The basic voiceprint features are integrated with the time series correlation features to form a multi-dimensional voiceprint feature vector for each channel; Among them, the pre-emphasis coefficient is dynamically adjusted according to preset rules.

3. The method according to claim 2, characterized in that When forming analysis frames by splitting frames and adding windows, the following steps are included: Synchronize the frame starting points of multi-channel sound signals; When a voiced segment is detected in the speech signal, the first frame length is used to capture the periodic characteristics; when an unvoiced segment, a white noise signal, or a swept frequency signal is detected, the second frame length is used to improve the time resolution; Use Hamming window for speech signals and use Heining window for swept frequency signals and white noise signals; Set the frame shift to form an overlapping frame structure.

4. The method according to claim 2, characterized in that Calculation of basic voiceprint features includes: The windowed analysis frame is subjected to fast Fourier transform, nonlinear compression and discrete cosine transform, and the order coefficient is extracted as the MFCC main feature; Search for the peak position within the fundamental frequency range, take the period corresponding to the maximum peak as the fundamental frequency period for voiced segments, and mark the fundamental frequency period as an invalid value for unvoiced segments, white noise signals, and swept frequency signals; The prediction error filter coefficients are obtained through LPC analysis, and the candidate resonance peak frequencies are determined; the resonance peak bandwidth threshold and amplitude threshold are set to screen the valid resonance peaks, and the first several main resonance peak frequencies are retained; The center frequency and bandwidth parameters of the MFCC filter banks of each channel are unified, and the cross-channel consistency check of the fundamental frequency period and formant frequency calculation results is performed.

5. The method according to claim 2, characterized in that When forming the multi-dimensional voiceprint feature vector of each channel, it includes: Standardize the basic voiceprint features and time series correlation features respectively; The basic voiceprint feature vector and the corresponding time series correlation feature vector in the same time window are spliced ​​in chronological order to form an initial fusion vector; Dynamically adjust the weight coefficient of each feature dimension according to the current sound signal type; Calculate the Euclidean distance of the fusion vectors of each channel at the same time point. If the distance between any two channels exceeds the cross-channel feature difference threshold, the feature correction mechanism is triggered; The fusion vector is reduced in dimension through linear discriminant analysis to form the final multi-dimensional voiceprint feature vector.

6. The method according to claim 1, characterized in that When obtaining the standard state feature space of each channel, it includes: The multi-dimensional voiceprint feature vectors corresponding to the standard white noise signal, standard swept frequency signal, and standard speech signal are extracted respectively, and weighted coefficients are assigned based on the importance of the signal type to construct a multimodal training sample set; A cross-channel similarity regularization term is introduced into the SVM objective function to constrain the Euclidean distance in the standard state feature space of adjacent physical channels. The polynomial kernel function, radial basis kernel function and linear kernel function are used for the training samples dominated by speech signal, frequency sweep signal and white noise signal respectively; Part of the training samples were used as the validation set, and the SVM classification interval parameters were adjusted to make the upper limit of the 95% confidence interval of the Mahalanobis distance of the normal state samples as the final confidence threshold; The mean vector, covariance matrix and confidence threshold of each channel feature obtained through training are associated with the physical position coordinates according to the channel number to form a standard state feature space.

7. The method according to claim 1, characterized in that When identifying the type of exception, include: Calculate the Mahalanobis distance between the multi-dimensional voiceprint feature vector of each channel in the current frame and the corresponding standard state feature space; A spatial weight matrix is ​​constructed based on the physical position of the vocal tract, and a comprehensive anomaly index is calculated based on the distance change rate within the time window. If the comprehensive abnormality index exceeds the preset threshold, the difference vector between the multi-dimensional voiceprint feature vector of the current frame of the abnormal channel and the corresponding standard state feature space is extracted for abnormality identification.

8. The method according to claim 7, characterized in that When calculating the Mahalanobis distance, include: Obtain the multidimensional voiceprint feature vector of the current frame after dimensionality reduction by linear discriminant analysis; Call the feature mean vector and covariance matrix stored in the standard state feature space of the corresponding vocal channel; Combined with the weight coefficients of each feature dimension dynamically assigned when forming the multidimensional voiceprint feature vector, the multidimensional voiceprint feature vector, the feature mean vector, and the covariance matrix are weighted and adjusted to form a weighted feature vector, a weighted mean vector, and a weighted covariance matrix; Based on the weighted covariance matrix, the Mahalanobis distance between the weighted eigenvector and the weighted mean vector is calculated; A neighborhood relationship is established based on the physical position of the channels, and the deviation rate between the Mahalanobis distance of the current channel and the adjacent channels is calculated. If the deviation exceeds the set value, a smoothing correction is performed.

9. The method according to claim 7, characterized in that When calculating the comprehensive abnormality index, the following are included: Get the physical position coordinates of each channel, calculate the Euclidean distance between any two channels, and identify neighboring channels; The Gaussian weight function is used to calculate the spatial weight to form a symmetric spatial weight matrix; Select a sliding time window and record the time series of Mahalanobis distances of each channel within the window; Obtain the Mahalanobis distance change rate of each channel in the frame interval time, and obtain the average Mahalanobis distance change rate in the sliding time window; For the current frame channel, based on the spatial weight matrix, obtain the sum of the weighted Mahalanobis distances between it and all neighboring channels; By defining the spatial anomaly component and the temporal anomaly component, a comprehensive anomaly index is generated.

10. A real-time monitoring system for multi-channel output status based on voiceprint recognition, characterized in that: include: Synchronous acquisition module: including an array microphone group with a clock synchronization interface and a multi-channel digital-to-analog conversion module; Feature extraction module: integrated digital signal processor, equipped with adaptive pre-emphasis filter, frame windowing unit, Hamming window generation unit, Haining window generation unit, MFCC calculation unit, fundamental frequency period extraction unit, formant frequency extraction unit and time series feature extraction unit; Benchmark modeling module: Contains non-volatile memory for storing the standard state feature space of each channel and an SVM training engine for initial training; Anomaly detection module: It consists of a Mahalanobis distance calculation unit, a spatiotemporal correlation analysis unit, and anomaly recognition unit implemented by FPGA; Feedback control module: integrated microcontroller, equipped with control instruction generation unit, bus communication unit and actuator drive interface; Monitoring terminal: It has a human-computer interaction interface for displaying the status parameters of each channel and abnormal alarm information.

Images