Intelligent audio analysis and voice recognition cloud fusion computing service platform

Abstract

The invention discloses an intelligent audio analysis and voice recognition cloud fusion computing service platform. The platform comprises the following modules: an audio acquisition module acquires multi-channel audio data from multi-terminal equipment; the Gammatone filtering module is used for filtering the audio data and extracting preliminary audio features; the feature optimization module refines the frequency spectrum information and generates optimized audio features; the state detection module analyzes the features and generates a state report or alarm in combination with historical archives; the database module stores audio records, acoustic files and model parameters; the recognition module selects an edge or cloud model to perform voice recognition according to the state report, and outputs a recognition result; and the output module receives various results and generates a unified service report. According to the invention, adaptive identification and analysis of audio information in multiple scenes can be realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of speech signal processing and intelligent speech recognition, and particularly relates to an intelligent audio analysis and speech recognition cloud fusion computing service platform. BACKGROUND

[0002] With the wide application of intelligent speech technology, speech recognition and audio analysis play an increasingly key role in mobile terminals, vehicle-mounted systems, intelligent sound boxes, remote conferences and industrial voice monitoring in multiple scenarios. Multi-channel audio acquisition and real-time processing systems are gradually replacing traditional single-channel speech recognition architectures to meet higher requirements for speech interaction quality, system response speed and recognition accuracy in complex environments. However, in the actual deployment process, the current mainstream audio recognition system still faces many technical problems, which restricts its landing efficiency and robustness in edge-cloud collaborative computing scenarios.

[0003] In the prior art, the mainstream speech recognition system is mostly based on a fixed feature extraction algorithm, preset bandwidth filter parameters and a static model path, and relies on centralized computing resources for unified recognition. Such a method has the following disadvantages in the face of complex acoustic environments, dynamic computing conditions and diversified application requirements: the traditional audio preprocessing module uses Gammatone or mel filter with fixed parameter design, which cannot dynamically adjust the bandwidth structure according to the frame-level noise characteristics, resulting in a significant performance decline in low signal-to-noise ratio environments. The existing SincNet model often uses unified frequency initialization and a general window function, which is difficult to model high-frequency details and low-frequency envelopes respectively, resulting in insufficient feature optimization expression. The recognition module in the current edge computing architecture is mostly a fixed model link, which fails to intelligently switch the edge-side and cloud-end model paths according to the actual state, causing resource waste or delay increase.

[0004] Therefore, how to provide an intelligent audio analysis and speech recognition cloud fusion computing service platform is a problem that those skilled in the art need to solve. SUMMARY

[0005] The present application provides an intelligent audio analysis and speech recognition cloud fusion computing service platform, which adopts a method combining multi-channel audio acquisition, Gammatone filtering, double-path SincNet feature refinement, real-time state detection and edge-cloud distributed recognition to adaptively process environmental noise and channel differences, generate high-quality optimized audio features and intelligently select edge or cloud models to complete speech recognition.

[0006] According to the intelligent audio analysis and speech recognition cloud fusion computing service platform provided by the embodiment of the present application, the following steps are included:

[0007] The audio acquisition module is configured to acquire multi-channel audio data from a mobile terminal, a vehicle-mounted device, a smart speaker, a conference system, and an industrial voice acquisition device.

[0008] The Gammatone filter module is configured to perform Gammatone filtering on the audio data to form preliminary audio features.

[0009] The SincNet feature optimization module is configured to perform spectral refinement on the preliminary audio features to generate optimized audio features.

[0010] The state detection module is configured to receive the optimized audio features and generate a state report in combination with historical acoustic archives stored in a database, and generate an alarm report if an abnormality is detected.

[0011] The database module is configured to store historical audio data, acoustic archives, and model parameters.

[0012] The distributed recognition module is configured to dynamically select an edge-side or cloud-side model to perform speech recognition processing on the optimized audio features according to the state report, and generate a recognition result.

[0013] The output module is configured to receive the recognition result, the state report, and the alarm report, and uniformly output an audio analysis and speech recognition service report.

[0014] Optionally, the Gammatne filter module specifically includes:

[0015] The input multi-channel audio data is divided into a frame signal sequence, the frame length is set to L sampling points, and the starting points of adjacent frames are spaced by (1-r)·L sampling points, where r is a frame overlap rate, and the divided sequence x k [n] is formed, where k is a frame number, and n is a frame sampling point number.

[0016] For each continuous sampling point within a frame k, the zero-crossing rate ZCR k is calculated, that is, the number of sign changes between adjacent sampling points is counted and normalized to the frame length ratio.

[0017] The frame signal-to-noise ratio SNR k is calculated, which is defined as the ratio of the effective signal power of the current frame to the reference noise power.

[0018] The ZCR k and the SNR k are proportionally weighted to generate a hybrid noise indicator η k .

[0019] The current frame filter bandwidth b k is modulated according to η k , and the modulation is: b k =b0·(1+α·log 10 ηk ), wherein b0 is an initial bandwidth, and a is a bandwidth modulation sensitivity coefficient;

[0020] On the basis of bandwidth modulation, bandwidth contraction adjustment is directly triggered according to the internal statistical characteristics of input data, that is, whether the proportion of abnormal frames in the current frame set exceeds a preset threshold θ is calculated, and when the proportion of abnormal frames is greater than θ, the bandwidth of the current frame and subsequent frames is immediately narrowed;

[0021] According to the adjusted bandwidth parameters, a complex-valued Gammatone filter bank is constructed, and the impulse response of each filter includes a real part and an imaginary part; the real part is defined as: on the non-negative time axis, the real part is composed of a Gamma envelope function and a center frequency modulation cosine term; the imaginary part is defined as: the real part is convolved with a Hilbert-modulated delay kernel;

[0022] After the filter bank is generated, orthogonal compression optimization is performed, which includes: judging whether the center frequency difference between adjacent Gammatone filters is less than a frequency interval threshold Δf, if less than the threshold, merging the filter weights and recalibrating the normalization factor, and controlling the cross-correlation coefficient between any filter pairs in the Gammatone filter bank to be less than a threshold ∈;

[0023] The frame signal and the filter bank are convolved one by one to generate: an amplitude envelope vector A k , and an instantaneous phase vector φ k ;

[0024] Three types of preliminary audio features are extracted: sub-band intra-frame energy, inter-channel correlation, and inter-frame phase difference;

[0025] The above three types of preliminary audio features are normalized by Z-scre, and the normalized preliminary audio feature sequence is output as the final result of the Gammatne filter module.

[0026] Optionally, the method for generating the imaginary part specifically includes:

[0027] Discrete-time sampling is performed on the real part of each Gammatone filter to generate a real part sequence , wherein i represents the filter number, and n∈[-N,N] represents the intra-frame sampling point index;

[0028] A Hilbert-modulated delay kernel FIR convolution kernel sequence h HMDK [m] is constructed, wherein h HMDK [m] is an odd-symmetric finite impulse response convolution kernel, M is the half length of the odd-symmetric finite impulse response convolution kernel, and the value determines the kernel length as 2M+1, is the discrete kernel of an ideal Hilbert filter; a sine window function for modulating the frequency domain characteristics;

[0029] performing a convolution operation of the real part sequence of the i-th Gammatone filter with the Hilbert-modulated delay kernel FIR convolution kernel to generate a virtual part sequence of the i-th filter, that is, multiplying the value of each time point in the real part with the value of the corresponding position in the modulation kernel within the symmetric interval respectively, and summing all the products as the value of the corresponding position of the virtual part;

[0030] constructing a complex number form of the i-th Gammatone filter, combining the real part with the generated virtual part response in the form of a complex number, wherein the real part is the original response value of the filter, and the virtual part is the convolution result of the Hilbert-modulated delay kernel;

[0031] The Gammatone filter module is based on a complex number g i The SincNet feature optimization module performs a multi-channel filtering operation on the input frame signal to output a frame-level complex number feature sequence.

[0032] Optionally, the SincNet feature optimization module specifically includes:

[0033] receiving the output preliminary audio feature sequence V k , wherein k represents a frame number, V k contains frame-level multi-channel preliminary audio feature data;

[0034] initializing the heterogeneous double-path SincNet convolution module, including:

[0035] The first path SincNet convolution channel: the number of convolution kernels is fixed, the center frequency of the convolution kernel is initialized according to the full frequency band linear equal interval, the impulse response of the convolution kernel is windowed by the Hamming window function, and the low cutoff frequency parameter and the high cutoff frequency parameter of each convolution kernel are limited to the range of half Nyquist frequency to half Nyquist frequency for training;

[0036] The second path SincNet convolution channel: the number of convolution kernels is fixed, the center frequency of the convolution kernel is initialized according to the Mel frequency scale, the impulse response of the convolution kernel is windowed by the Blackman window function, and the low cutoff frequency parameter and the high cutoff frequency parameter of each convolution kernel are limited to the range of 0Hz to half Nyquist frequency for training;

[0037] An asymmetric deformable structure is adopted for each SincNet convolution kernel, and independent time offset parameters are introduced into the positive time axis and the negative time axis of the convolution kernel impulse response

[0038] statistically calculating the frame energy distribution width of the input frame sequence V k , forming a statistical quantity STD frameand the offset adjustment value Δδ is calculated through a gating network i , the offset adjustment value Δδ i The time offset parameter of the convolution kernel is updated to ensure that the positive and negative time axis offsets of the convolution kernel are adaptively adjusted according to the width of the frame energy distribution.

[0039] After the convolution and adaptive adjustment are completed, the first-path convolution output features are subjected to intra-frame local normalization processing, and the mean value and standard deviation are calculated to perform standardization; and the second-path convolution output features are subjected to inter-frame sliding weighted average smoothing processing.

[0040] The first-path and second-path convolution output features are fused by weighting, and the fusion manner includes weighted summation according to a preset weighting coefficient or channel equalization weighted summation, and the fusion result is further obtained by performing residual connection superposition on the input preliminary audio feature sequence V k to generate optimized audio features.

[0041] Optionally, the state detection module specifically includes:

[0042] The optimized audio feature sequence is received, and the sequence is divided into frame groups with fixed lengths, and each frame contains a group of spectral energy features and a group of acoustic parameter features.

[0043] The historical acoustic archives under the corresponding scene are called from the database module, and reference frame features with the same dimension as the current frame group are extracted, including a historical spectral energy mean value sequence and a historical acoustic parameter mean value sequence.

[0044] For each frame, the frame energy deviation between the current frame spectral energy features and the historical spectral energy mean value is calculated, and the frame parameter deviation between the current frame acoustic parameter features and the historical acoustic parameter mean value is calculated.

[0045] The frame energy deviation and the frame parameter deviation are weighted and summed according to a preset weight to generate a comprehensive deviation score of the frame.

[0046] The comprehensive deviation score is compared with a set threshold value, and when the comprehensive deviation score is greater than the threshold value, the frame is marked as an abnormal frame, otherwise it is marked as a normal frame.

[0047] A sliding window is set in the continuous frames, and the number of abnormal frames in the sliding window is counted, and when the proportion of abnormal frames exceeds a proportion threshold value, the current window is determined to be in an abnormal state, an alarm report is generated, otherwise a normal state report is generated.

[0048] The generated state report includes a detection time, an abnormal frame proportion, main deviation indicators and a reference historical archive number.

[0049] Optionally, the database module is configured to store multi-channel audio data with sampling time, channel number and frame sequence number, historical acoustic archives composed of audio features output by the Gammatone filtering module and the SincNet feature optimization module, index entries including frame-level energy feature vectors, inter-channel correlation features and normalized comprehensive feature vectors, model structure configuration files and weight file summaries provided by the distributed recognition module and their version numbers, and complete asynchronous data synchronization, incremental tracking and difference checking between the edge and the cloud database based on the center frequency range, feature dimension and scene identification.

[0050] Optionally, the distributed recognition module specifically includes:

[0051] receive the state report output by the state detection module, and parse the current detection state mark;

[0052] According to the state mark in the state report, if the state mark is normal, the edge speech recognition model is selected, and if the state mark is abnormal, the cloud speech recognition model is selected;

[0053] The optimized audio features output by the SincNet feature optimization module are divided into frame-level feature sequences in chronological order, and each frame feature is normalized according to the preset mean and standard deviation and then input into the selected speech recognition model;

[0054] On the edge, the speech recognition processing includes the following sub-steps: receiving the normalized frame-level feature sequence, calculating the classification probability distribution of each frame on the predefined phoneme set using a lightweight acoustic model, and selecting the maximum probability corresponding phoneme as the frame label; merging the continuous frames with the same adjacent frame label to generate a preliminary phoneme sequence; applying the edge-preinstalled finite state transition dictionary to each preliminary phoneme sequence to match the legal word entries and form the edge recognition text result;

[0055] In the cloud, the speech recognition processing includes the following sub-steps: receiving the normalized frame-level feature sequence, calculating the classification probability distribution of each frame on the extended phoneme set using a full-depth acoustic model, and using the Viterbi decoding algorithm to find the optimal frame path to generate a preliminary frame-level phoneme label sequence; inputting the preliminary label sequence into the cloud context-independent language model to perform context probability correction and generate a corrected phoneme sequence; converting the corrected phoneme sequence into a recognition text result according to the phoneme-to-word mapping rule;

[0056] When the edge and the cloud both generate available recognition text results, the average frame-level confidence of each result is calculated;

[0057] Compare the average frame-level confidence of the edge and the cloud recognition text, and select the text with higher confidence as the final recognition result;

[0058] The final recognition result record includes a frame sequence, a corresponding timestamp, a recognition path identifier, and an average confidence, and is output to an output module.

[0059] Optionally, the lightweight acoustic model specifically includes:

[0060] The normalized frame-level feature sequence is received as input, and the input tensor dimension is (T, D), where T represents the frame sequence length, and D represents the feature dimension;

[0061] The input frame-level feature sequence is processed by a single-layer bidirectional long short-term memory network, and the output hidden state tensor dimension is (T, 2H);

[0062] Each frame hidden state vector output by the single-layer bidirectional long short-term memory network is mapped to a phoneme class space through matrix multiplication, forming a score tensor with a dimension of (T, N E ) where N E is a predefined phoneme class number;

[0063] The score vector of each frame of the score tensor is calculated using a softmax function to obtain a probability distribution, the phoneme class with the maximum probability of each frame is selected as the frame label, and the phoneme prediction result of the frame sequence is recorded as the edge preliminary recognition result.

[0064] Optionally, the full-amount deep acoustic model specifically includes:

[0065] The normalized frame-level feature sequence is received as input, and the input tensor dimension is (T, D), where T represents the frame sequence length, and D represents the feature dimension;

[0066] The input frame-level feature sequence is convolved in the time dimension by a one-dimensional convolutional neural network, the number of convolution kernels is K, and the convolution output tensor dimension is (T, K);

[0067] The convolution output is input into a multi-layer bidirectional long short-term memory network, the number of layers of the multi-layer bidirectional long short-term memory network is L C , the number of single-direction hidden units of each layer is H C , and the final output hidden state tensor dimension is (T, 2H C );

[0068] Each frame hidden state vector output by the final layer of the multi-layer bidirectional long short-term memory network is mapped to an extended phoneme class space through matrix multiplication, forming a frame-level score tensor with a dimension of (T, N C ) where N C is the number of extended phoneme classes;

[0069] The softmax function is used to calculate the probability distribution of each class of the frame-level score tensor, and the Viterbi decoding algorithm is used to solve the maximum path probability sequence of the frame-level probability matrix of the full sequence, and the frame label sequence is output;

[0070] inputting the frame label sequence into a context-free n-gram language model, correcting the frame label sequence according to a language probability, and generating a corrected phoneme sequence;

[0071] by means of a predefined phoneme-to-word mapping table, converting the phoneme sequence into a cloud recognition text result one by one, and outputting the recognition text and frame-level confidence statistical information.

[0072] Optionally, the output module is configured to receive a final recognition result generated by the distributed recognition module, a state report and an alarm report generated by the state detection module, and to aggregate frame sequence, time stamp, recognition path and abnormality index and the like information to form a standardized audio analysis and speech recognition service report output to a terminal interface.

[0073] The present application has the following advantages:

[0074] (1) The present application combines Gammatone filtering with noise indicators such as zero-crossing rate and signal-to-noise ratio, introduces a bandwidth adaptive modulation mechanism and an abnormal driving bandwidth narrowing mechanism, so that the filter bank has dynamic adjustment capability in different noise environments, enhances the robustness and discriminability of feature extraction, and significantly improves the stability and recognition preprocessing capability of the system in unstructured sound environments.

[0075] (2) The present application constructs a dual-path heterogeneous SincNet feature optimization structure, introduces local normalization and sliding smoothing mechanism in frame and inter-frame respectively, and introduces a context statistical driven asymmetric deformable convolution kernel adjustment path on this basis, which effectively enhances the expression ability of audio features in time-frequency distribution dimension, and adapts to multi-scene changes and different speaker features.

[0076] (3) The present application introduces a state detection module, realizes automatic identification of abnormal sound field in combination with historical acoustic archives, and drives the distributed recognition module to dynamically select edge or cloud speech recognition model, so as to improve the response speed and resource scheduling capability of the system in the environment with limited computing resources while ensuring the recognition accuracy, and realize the optimization of edge-cloud collaborative recognition. BRIEF DESCRIPTION OF DRAWINGS

[0077] The accompanying drawings are included to provide a further understanding of the present application, and constitute a part of the specification, which together with the embodiments of the present application, serve to explain the present application, and do not constitute a limitation on the present application. In the drawings:

[0078] Fig. 1 a flow chart of an intelligent audio analysis and speech recognition cloud fusion computing service platform according to the present application;

[0079] Fig. 2 a processing flow chart of a Gammatone filtering module according to the present application;

[0080] Fig. 3 A structure flow chart of the SincNet feature optimization module proposed in the present application. DETAILED DESCRIPTION

[0081] The present application will now be further described in greater detail in connection with the accompanying drawings. These drawings are simplified schematic illustrations of the basic structure of the present application and therefore only show the components relevant to the present application.

[0082] REFERENCE Figs. 1-3 An intelligent audio analysis and speech recognition cloud fusion computing service platform, comprising the following steps:

[0083] The audio acquisition module is used to acquire multi-channel audio data from mobile terminals, vehicle-mounted devices, smart sound boxes, conference systems, and industrial voice acquisition devices;

[0084] The Gammatone filtering module is used to perform Gammatone filtering processing on the audio data to form preliminary audio features;

[0085] The SincNet feature optimization module is used to perform spectral refinement on the preliminary audio features to generate optimized audio features;

[0086] The state detection module is used to receive the optimized audio features and combine the historical acoustic archives stored in the database to generate a state report, and generate an alarm report if an abnormality is detected;

[0087] The database module is used to store historical audio data, acoustic archives, and model parameters;

[0088] The distributed recognition module is used to dynamically select an edge-end or cloud-end model to perform speech recognition processing on the optimized audio features according to the state report to generate a recognition result;

[0089] The output module is used to receive the recognition result, the state report, and the alarm report, and uniformly output an audio analysis and speech recognition service report.

[0090] In the present embodiment, the Gammatne filtering module specifically comprises:

[0091] The input multi-channel audio data is divided into a frame signal sequence, the frame length is set to L sampling points, and the starting points of adjacent frames are spaced by (1-r)·L sampling points, where r is the frame overlap rate. After division, a sequence x k [n] is formed, where k is the frame number and n is the sampling point number within the frame.

[0092] For each continuous sampling point within a frame k, the zero-crossing rate ZCR k is calculated, that is, the number of sign changes between adjacent sampling points is counted and normalized to the frame length ratio, and the zero-crossing rate is defined as:

[0093]

[0094] where sgn(·) is the sign function, outputting 1, -1 or 0, ZCR k capture the intra-frame high-frequency variation, reflecting the high-frequency component density of the frame signal;

[0095] calculate the frame signal-to-noise ratio SNR k , defined as the ratio of the effective signal power of the current frame to the reference noise power, the effective power being the mean of the sample square sum within the frame, and the reference noise power N0 being derived from system preset or historical non-speech segment statistics:

[0096]

[0097] where the numerator is the average power of the current frame, and the denominator is the reference noise power;

[0098] scale the ZCR k and SNR k , and generate a hybrid noise indicator η k to reflect the comprehensive noise environment within the frame:

[0099] η k = γ1·SNR k + γ2·ZCR k ;

[0100] where γ1, γ2 are weight coefficients, satisfying 0 ≤ γ1, γ2 ≤ 1, γ1+ γ2 = 1, γ1 controls the contribution of low-frequency background noise, and γ2 controls the contribution of high-frequency interference, and the hybrid noise indicator finely characterizes the noise environment;

[0101] According to η k , the current frame filter bandwidth b k is dynamically modulated, and the modulation is: b k = b0·(1+ α·log 10 η k ), where b0 is the initial bandwidth, and α is the bandwidth modulation sensitivity coefficient (0.1-0.3), when the noise environment deteriorates, i.e. η k increases, the bandwidth is moderately expanded to cover more frequency details; when the environment is stable, the bandwidth is narrowed to enhance the feature resolution;

[0102] On the basis of adaptive modulation of bandwidth, it is allowed to directly trigger bandwidth contraction adjustment according to the internal statistical characteristics of the input data, i.e. by calculating whether the proportion of abnormal frames in the current frame set exceeds the preset threshold θ, defined as the proportion of frames whose hybrid noise indicator η k exceeds the preset noise threshold η thframe ratio is greater than θ, immediately narrow the bandwidth of the current frame and subsequent frames, and the adjustment formula is:

[0103] b k ← β·b k ;

[0104] Wherein, β is the bandwidth narrowing factor (0.6-0.8), and the normal frame ratio is automatically generated based on the intra-frame index;

[0105] According to the dynamically adjusted bandwidth parameter, a complex-valued Gammatone filter bank is constructed, and the impulse response of each filter includes real and imaginary parts:

[0106] The real part is defined as: on the non-negative time axis, the real part is composed of the Gamma envelope function and the center frequency modulation cosine term:

[0107]

[0108] Wherein, n is the filter order, b k is the bandwidth parameter corresponding to the kth frame, f 0,i is the center frequency of the ith filter, and t is the time variable, the domain is t≥0;

[0109] The imaginary part is defined as: the real part is convolved with the Hilbert-modulated delay kernel;

[0110] After the filter bank is generated, orthogonal compression optimization is performed, which includes: judging whether the center frequency difference between adjacent Gammatone filters is less than the frequency interval threshold Δf (80-120 Hz), if less than the threshold, merging the filter weights and recalibrating the normalization factor, and controlling the cross-correlation coefficient between any filter pairs in the Gammatone filter bank to be less than the threshold ∈ (0.1-0.3);

[0111] The frame signal and the filter bank are convolved one by one to generate: k The envelope amplitude A k , which is the envelope amplitude output by the filter real part convolution; the instantaneous phase φ k , which is the instantaneous phase output by the filter imaginary part convolution, capturing the fast phase change feature;

[0112] Three types of preliminary audio features are extracted: sub-band intra-frame energy, calculated by the square sum of A k ; inter-channel correlation, obtained based on the correlation coefficient matrix of multi-channel amplitude features; inter-frame phase difference, calculated based on the inter-frame difference of φ

[0113] The three types of preliminary audio features are normalized by Z-score normalization, so that each dimension output feature is output in a standardized form with a mean of 0 and a standard deviation of 1, ensuring that the numerical range and distribution of the input features of the subsequent module are consistent.

[0114] The normalized preliminary audio feature sequence is output as the final result of the Gammatone filter module, which contains multi-dimensional, cross-channel, and amplitude and phase information preliminary audio features for subsequent SincNet feature optimization module calls.

[0115] In this embodiment, the generation method of the imaginary part specifically includes:

[0116] Discrete-time sampling of the real part of each Gammatone filter to generate a real part sequence Where i represents the filter number, and n∈[-N, N] represents the sampling point index within the frame.

[0117] Constructing a Hilbert-modulation delay kernel FIR convolution kernel sequence h HMDK [m], which is defined as follows:

[0118]

[0119] Where: h HMDK [m] is an odd-symmetric finite impulse response convolution kernel; M is the half length of the odd-symmetric finite impulse response convolution kernel, and the value determines the kernel length as 2M+1. is the discrete kernel of the ideal Hilbert filter; is a sine window function used to modulate frequency domain characteristics, which limits the frequency response within the Nyquist half-band;

[0120] Performing a convolution operation of the real part sequence of the i-th Gammatone filter with the Hilbert-modulation delay kernel FIR convolution kernel to generate the imaginary part sequence of the i-th filter, i.e., multiplying the value of each time point in the real part with the value of the corresponding position in the modulation kernel within the symmetric interval, and summing all the products as the value of the corresponding position of the imaginary part.

[0121] Constructing the complex form of the i-th Gammatone filter by combining the real part and the generated imaginary part in complex form, where the real part is the original response value of the filter, and the imaginary part is the convolution result of the Hilbert-modulation delay kernel;

[0122] The Gammatone filter module performs a multi-channel filtering operation on the input frame signal based on a complex form, outputs a frame-level complex feature sequence, the real part is used to extract a sub-band energy feature, and the imaginary part is used to calculate an instantaneous phase feature in a frame, which is used as an input of a subsequent SincNet feature optimization module.

[0123] In the embodiment, the SincNet feature optimization module specifically includes:

[0124] The preliminary audio feature sequence V k output by the Gammatone filter module is received, where k represents a frame number, V k is a frame-level multi-channel preliminary audio feature data.

[0125] The heterogeneous double-path SincNet convolution module is initialized, including:

[0126] The first-path SincNet convolution channel: the number of convolution kernels is fixed, the center frequency of the convolution kernel is initialized according to linear equal intervals of the full frequency band, the impulse response of the convolution kernel is windowed by a Hamming window function, and the low cutoff frequency parameter and the high cutoff frequency parameter of each convolution kernel are limited to a range of half the Nyquist frequency to the Nyquist frequency for training;

[0127] The second-path SincNet convolution channel: the number of convolution kernels is fixed, the center frequency of the convolution kernel is initialized according to a Mel frequency scale, the impulse response of the convolution kernel is windowed by a Blackman window function, and the low cutoff frequency parameter and the high cutoff frequency parameter of each convolution kernel are limited to a range of 0 Hz to half the Nyquist frequency for training;

[0128] An asymmetric deformable structure is used for each SincNet convolution kernel, and independent time offset parameters are introduced into the positive time axis and the negative time axis of the impulse response of the convolution kernel The impulse response of the convolution kernel is defined as:

[0129]

[0130] where h i (t) is the impulse response of the i th convolution kernel at time t, f h,i represents the high cutoff frequency, f l,i represents the low cutoff frequency, respectively, are positive and negative time axis offset parameters, used to adjust the shape of the convolution kernel.

[0131] The frame energy distribution width of the input frame sequence V k is counted to form a statistical quantity STD frame , and an offset adjustment value Δδ i is calculated through a gating network:

[0132] Δδ i =η·σ(W·STD frame );

[0133] wherein, η represents an adjustment learning rate, W is a trainable weight matrix, σ(·) is a Sigmoid activation function, the output value is limited in the interval [0, 1], and Δδ i The time offset parameter for dynamically updating the convolution kernel: The positive and negative time axis offsets of the convolution kernel are dynamically and adaptively adjusted according to the width of the frame energy distribution, forming a context-driven adaptive adjustment closed loop.

[0134] After the above convolution and adaptive adjustment are completed, the first-path convolution output features are subjected to intra-frame local normalization processing, and the mean value and standard deviation are calculated for frame-by-frame normalization; and the second-path convolution output features are subjected to inter-frame sliding weighted average smoothing processing.

[0135] The first-path and second-path convolution output features are fused by weighting, and the fusion manner includes weighted summation according to a preset weighting coefficient or channel equalization weighted summation, and the fusion result can be further superimposed with the input preliminary audio feature sequence V k by residual connection, to form a fusion result.

[0136] The optimized audio features are generated as the final output of the SincNet feature optimization module.

[0137] In the embodiment, the state detection module specifically comprises:

[0138] The optimized audio feature sequence is received, and the sequence is divided into frame groups with fixed length, each frame containing a group of spectral energy features and a group of acoustic parameter features;

[0139] The historical acoustic archives under the corresponding scene are called from the database module, and reference frame features with the same dimension as the current frame group are extracted, including a historical spectral energy mean value sequence and a historical acoustic parameter mean value sequence;

[0140] For each frame, the frame energy deviation between the current frame spectral energy features and the historical spectral energy mean value is calculated, and the frame parameter deviation between the current frame acoustic parameter features and the historical acoustic parameter mean value is calculated, the frame energy deviation being defined as the average of the absolute values of the energy differences of each frequency band, and the frame parameter deviation being defined as the average of the absolute values of the parameter value differences of each parameter;

[0141] The frame energy deviation and the frame parameter deviation are weighted and summed according to a preset weight to generate a comprehensive deviation score of the frame; the comprehensive deviation score is compared with a set threshold value, and when the comprehensive deviation score is greater than the threshold value, the frame is marked as an abnormal frame, otherwise it is marked as a normal frame;

[0142] A sliding window is set in continuous frames, and the number of abnormal frames in the window is counted. When the proportion of abnormal frames exceeds the proportion threshold, the current window is determined to be in an abnormal state, and an alarm report is generated. Otherwise, a normal state report is generated.

[0143] The generated report includes detection time, abnormal frame proportion, main deviation index, and reference history archive number, and the report is transmitted to the distributed recognition module and the output module.

[0144] In the embodiment, the database module is used to store multi-channel audio data with sampling time, channel number and frame sequence number, historical acoustic archives composed of audio features output by the Gammatone filtering module and the SincNet feature optimization module, index entries including frame-level energy feature vectors, inter-channel correlation features and normalized comprehensive feature vectors, model structure configuration files and weight file summaries provided by the distributed recognition module and their version numbers, and based on the center frequency range, feature dimension and scene identification, asynchronous data synchronization, incremental tracking and difference verification between the edge and the cloud database are completed.

[0145] In the embodiment, the distributed recognition module specifically includes:

[0146] The state report output by the state detection module is received, and the current detection state mark is parsed;

[0147] According to the state mark in the state report, if the state mark is normal, the edge voice recognition model is selected, and if the state mark is abnormal, the cloud voice recognition model is selected;

[0148] The optimized audio features output by the SincNet feature optimization module are divided into frame-level feature sequences in time sequence, and each frame feature is normalized according to the preset mean and standard deviation and then input into the selected voice recognition model;

[0149] On the edge, the voice recognition process includes the following sub-steps: receiving the normalized frame-level feature sequence, calculating the classification probability distribution of each frame on the predefined phoneme set using a lightweight acoustic model, and selecting the maximum probability corresponding phoneme as the frame label; merging the continuous frames with the same adjacent frame label to generate a preliminary phoneme sequence; applying the edge-preinstalled finite state transition dictionary to each preliminary phoneme sequence to match the legal word entries and form the edge recognition text result;

[0150] In the cloud, the speech recognition processing includes the following sub-steps: receiving a normalized frame-level feature sequence, calculating a classification probability distribution of each frame on an extended phoneme set using a full-amount deep acoustic model, using a Viterbi decoding algorithm to obtain an optimal frame path, and generating a preliminary frame-level phoneme label sequence; inputting the preliminary label sequence into a cloud context-independent language model, performing context probability correction, and generating a corrected phoneme sequence; and converting the corrected phoneme sequence into a recognition text result according to a phoneme-to-word mapping rule;

[0151] When the edge and the cloud both generate available recognition text results, the average frame-level confidence of each result is calculated, and the confidence calculation formula is:

[0152]

[0153] wherein C k represents the confidence value of the kth frame, max(p k ) is the highest class prediction probability of the frame, is the sum of all class prediction probabilities of the frame, and N is the total number of classes;

[0154] The average frame-level confidences of the recognition texts of the edge and the cloud are compared, and the text with higher confidence is selected as the final recognition result;

[0155] The final recognition result record contains a frame sequence, a corresponding timestamp, a recognition path identifier, and an average confidence, and is output to an output module for generating an audio analysis and speech recognition service report.

[0156] In the embodiment, the lightweight acoustic model specifically includes:

[0157] A normalized frame-level feature sequence is received as input, and the input tensor dimension is (T, D), wherein T represents the frame sequence length, and D represents the feature dimension;

[0158] The input sequence is processed by a single-layer bidirectional long short-term memory network, and the output hidden state tensor dimension is (T, 2H E ), wherein H E is the number of one-way hidden units;

[0159] Each frame hidden state vector output by the single-layer bidirectional long short-term memory network is mapped to a phoneme class space through matrix multiplication to form a score tensor with a dimension of (T, N E ), wherein N E is the number of predefined phoneme classes;

[0160] The score vector of each frame of the score tensor is calculated using a softmax function to obtain a probability distribution;

[0161] Select the most probable phoneme category of each frame as the frame label, and record the phoneme prediction results of the frame sequence as the edge preliminary recognition result.

[0162] In this embodiment, the full-quantity deep acoustic model specifically comprises:

[0163] The normalized frame-level feature sequence is received as input, and the input tensor dimension is (T, D), wherein T represents the frame sequence length, and D represents the feature dimension;

[0164] The input sequence is convolved in the time dimension by a one-dimensional convolutional neural network, the number of convolution kernels is K, and the convolution output tensor dimension is (T, K);

[0165] The convolution output is input into a multi-layer bidirectional long short-term memory network, the number of layers of the multi-layer bidirectional long short-term memory network is L C , the number of single-direction hidden units of each layer is H C , and the final output hidden state tensor dimension is (T, 2H C );

[0166] Each frame hidden state vector output by the final layer of the multi-layer bidirectional long short-term memory network is mapped to an extended phoneme category space through matrix multiplication, forming a frame-level score tensor with a dimension of (T, N C ), wherein N C is the number of extended phoneme categories;

[0167] The softmax is used to calculate the probability distribution of each category for the frame-level score tensor, the Viterbi decoding algorithm is used to solve the maximum path probability sequence of the frame-level probability matrix of the whole sequence, and the frame label sequence is output;

[0168] The frame label sequence is input into a context-independent n-gram language model, the frame label sequence is corrected according to the language probability, and a corrected phoneme sequence is generated;

[0169] The phoneme sequence is converted into a cloud recognition text result one by one through a predefined phoneme-to-word mapping table, and the recognition text and frame-level confidence statistical information are output.

[0170] In this embodiment, the output module is used to receive the final recognition result generated by the distributed recognition module, the state report and the alarm report generated by the state detection module, and to summarize the frame sequence, the time stamp, the recognition path and the abnormal index and other information, so as to form a standardized audio analysis and speech recognition service report output to the terminal interface.

[0171] Embodiment:

[0172] To verify the effectiveness and practical application value of the intelligent audio analysis and speech recognition cloud fusion computing service platform proposed in the present application, a large urban rail transit dispatching center in China is taken as a pilot scene for deployment test. The trial operation started in the first quarter of 2025. Due to the dense voice flow, complex noise interference and various device interface types in the rail transit dispatching environment, the traditional speech recognition system faces the problems of low accuracy, high delay and poor scalability in this scenario, and an intelligent speech service platform with fine audio feature processing, dynamic state perception and distributed recognition capability is urgently needed to support efficient dispatching and real-time risk response.

[0173] In the present embodiment, the overall system of the present application is deployed between the front-end voice acquisition nodes and the back-end cloud computing center of the dispatching center. Multiple voice sources such as dispatching consoles, conference microphones, wayside devices and passenger help systems are uniformly accessed through audio acquisition modules to form multi-channel audio input data streams. The Gammatone filtering module preprocesses these raw audio data on the edge node, extracting frame-level preliminary audio features including amplitude envelope, instantaneous phase, inter-channel correlation, etc. The module realizes a hybrid noise indicator based on the joint construction of frame-level SNR and ZCR in the inference stage, and adaptively adjusts the filter bandwidth, cooperates with the Hilbert-modulation delay kernel to construct the complex filter output, and improves the energy stability and resolution in the strong non-stationary background such as subway cars. The processed features are significantly reduced in feature dimension after executing orthogonal compression, improving the throughput efficiency of subsequent modules.

[0174] The SincNet feature optimization module further refines the spectrum locally. In the present embodiment, a heterogeneous double-path SincNet structure is used: one path uses linear frequency scaling and is multiplied by a Hamming window for initialization of the convolution kernel, learns the cutoff frequency of the upper half of the frequency band, and the other path uses Mel scaling and is multiplied by a Blackman window, learns the cutoff frequency of the lower half of the frequency band. The first path output is processed by intra-frame normalization to enhance local energy contrast, and the second path output is processed by inter-frame weighted smoothing to improve continuity, and the fusion of the two forms an optimized audio feature. The asymmetric deformable kernel structure of this module adaptively adjusts the offset of the positive and negative time axes according to the intra-frame energy variance, bringing stronger time sensitivity, and the recognition rate of high semantic density short sentences such as "drive out" and "stop" in dispatching instructions is significantly improved.

[0175] The state detection module dynamically compares the above-optimized audio features with the historical acoustic archives to form a state report for each period. The historical acoustic archives are provided by the database module and include feature distribution in different device environments, typical abnormal samples, scheduling scene labels, etc. If a significant deviation is found (for example, a dramatic increase in the proportion of input audio noise, a spectral mean shift, etc.), the module automatically generates an alarm report and uploads it to the central cloud platform. By cross-comparing with previous waveform samples in the database, the system can infer whether the abnormal source is from device failure, human intervention, or environmental mutation.

[0176] The distributed recognition module selects a recognition path according to the current state report. If the state label is "normal", the local edge recognition model is called to quickly respond and output instruction text. If the state label is "abnormal", the cloud recognition model is switched to for redundant recognition and context reconstruction through a high-capacity deep network. The entire process maintains millisecond-level switching without human intervention, significantly improving system robustness and scalability.

[0177] The output module uniformly summarizes the recognition results, state reports, and alarm reports, and structures the comprehensive service report according to the timestamp, audio source ID, recognition path label, etc. This report not only supports standard interface transmission into the rail transit command system, but also supports historical data backtracking and security log auditing, providing data support for later risk tracing and model evaluation.

[0178] To quantify the technical effect of the present application in practical application, the dispatching center conducted a three-month comparative experiment from April to June 2025. The experimental object was a conventional rail transit dispatching task, covering about 200 hours of voice data. The traditional MFCC+deep acoustic model recognition system currently deployed was selected as the comparison group, and the present application platform was the experimental group. The core evaluation indicators included speech recognition accuracy (WER inverse ratio), average recognition delay, abnormal response time, feature processing time consumption, and overall system satisfaction.

[0179] The following is the experimental data statistical result:

[0180] Table 1 Comparison data table of traditional scheme and present application platform in scheduling scene

[0181]

[0182]

[0183] From the data table, it can be seen that in the peak period of strong noise interference, the optimized audio features can improve the recognition accuracy by 7.1 percentage points, and the improvement is particularly obvious in short-time high-command-density sentences such as "stop immediately" and "switch track to east". The core factor of significant delay reduction is that the system can dynamically switch the edge recognition path, so that most normal scenes do not need to upload to the cloud, avoiding unnecessary transmission delay. Through bandwidth adaptation and asymmetric convolution mechanism, the recognition effect in abnormal scenes does not decrease significantly, and the abnormal response time is reduced to 38.9% of the original. Although the Gammatone processing time of the platform is slightly increased, the average single-frame processing delay is still controlled within the acceptable range of 70ms, and this module supports batch processing, which has parallel processing advantages in multi-channel scenarios.

[0184] The system identified 23,416 dispatch passwords in three months, automatically recorded 34,288 status reports, and generated 352 alarm reports, with a false positive rate of less than 3% after manual verification. The system has a stable operation rate of 99.97%, and no hardware overload or model out-of-control phenomenon occurs. Dispatch commanders generally feedback that the recognition response speed is fast, the misrecognition rate is low, the log query and responsibility backtracking functions are supported, and the safety of the dispatch process is significantly improved.

[0185] In summary, the intelligent audio analysis and voice recognition cloud fusion computing service platform proposed by the present application is significantly superior to existing traditional solutions in terms of audio feature processing accuracy, voice recognition accuracy, system response speed, and state anomaly perception, and has good engineering feasibility and actual deployment value. Through the modular structure and adaptive mechanism, the system has strong scalability and is suitable for high-noise, multi-source, high-density voice scenarios, providing efficient voice service solutions for rail transit, industrial control, intelligent customer service and other fields.

[0186] The above describes only the preferred embodiments of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can make equivalent replacements or changes to the technical solutions and inventive concepts of the present application within the technical scope disclosed by the present application, which should be covered within the protection scope of the present application.

Claims

1. A cloud-based fusion computing service platform for intelligent audio analysis and speech recognition, characterized in that, It includes an audio acquisition module, a Gammatone filtering module, a SincNet feature optimization module, a state detection module, a database module, a distributed recognition module, and an output module; The audio acquisition module is used to acquire multi-channel audio data from mobile terminals, in-vehicle devices, smart speakers, conference systems, and industrial voice acquisition devices; The Gammatone filtering module is used to perform Gammatone filtering on audio data to form preliminary audio features; The SincNet feature optimization module is used to refine the spectrum of the initial audio features and generate optimized audio features. The status detection module receives optimized audio features and combines them with historical acoustic files stored in the database to generate a status report. If an anomaly is detected, an alarm report is generated. The database module is used to store historical audio data, acoustic archives, and model parameters; The distributed recognition module is used to dynamically select the edge or cloud model based on the status report to perform speech recognition processing on the optimized audio features and generate recognition results; The output module is used to receive recognition results, status reports and alarm reports, and output audio analysis and speech recognition service reports in a unified manner.

2. The intelligent audio analysis and speech recognition cloud fusion computing service platform according to claim 1, characterized in that, The Gammatne filtering module specifically includes: The input multi-channel audio data is divided into a sequence of frame signals, with a frame length of L sampling points and a starting point interval of (1-r)·L sampling points between adjacent frames, where r is the frame overlap rate. The resulting sequence is x. k [n], where k is the frame number and n is the intra-frame sampling point number; For each frame k consecutive sampling points, calculate the zero-crossing rate ZCR. k That is, to count the number of sign changes between adjacent sampling points and normalize them to the frame length ratio; Calculate frame signal-to-noise ratio (SNR) k , defined as the ratio of the effective signal power of the current frame to the reference noise power; ZCR k and SNR k The mixed noise index η is generated by proportional weighting. k ; According to η k Modulate the current frame filter bandwidth b k Modulation: b k =b0·(1+α·log) 10 η k ), where b0 is the initial bandwidth and α is the bandwidth modulation sensitivity coefficient; Based on bandwidth modulation, bandwidth contraction adjustment can be directly triggered according to the internal statistical characteristics of the input data. That is, by calculating whether the proportion of abnormal frames in the current frame set exceeds a preset threshold θ, when the proportion of abnormal frames is greater than θ, the bandwidth of the current frame and subsequent frames is immediately narrowed. Based on the adjusted bandwidth parameters, a complex-valued Gammatone filter bank is constructed. The impulse response of each filter includes a real part and an imaginary part: the real part is defined as the Gamma envelope function and the center frequency modulation cosine term on the non-negative time axis; the imaginary part is defined as the real part convolved with the Hilbert modulation delay kernel. After the Gammatone filter bank is generated, orthogonal compression optimization is performed. Orthogonal compression optimization includes: determining whether the center frequency difference between adjacent Gammatone filters is less than the frequency interval threshold Δf. If it is less than the threshold, the filter weights are merged and the normalization factor is recalibrated to control the cross-correlation coefficient between any filter pair in all Gammatone filter banks to not exceed the threshold ∈. Convolution operations are performed on the frame signal and the filter bank one by one to generate the amplitude envelope vector A. k Instantaneous phase vector φ k ; Three types of preliminary audio features were extracted: sub-band intra-frame energy; inter-channel correlation; and inter-frame phase difference. The three types of preliminary audio features are Z-scre normalized, and the normalized preliminary audio feature sequence is output as the final result of the Gammatne filtering module.

3. The intelligent audio analysis and speech recognition cloud fusion computing service platform according to claim 2, characterized in that, The method for generating the imaginary part specifically includes: Discrete-time sampling is performed on the real part of each Gammatone filter to generate a sequence of real parts. Where i represents the filter number, and n∈[-N,N] represents the intra-frame sampling point index; Constructing Hilbert-modulation delay kernel FIR convolution kernel sequence h HMDK [m]], where h HMDK [m] represents the odd-symmetric finite impulse response convolution kernel, and M is the half-length of the odd-symmetric finite impulse response convolution kernel. The value of M determines the kernel length, which is 2M+1. For the discrete kernel of the ideal Hilbert filter; This is a sinusoidal window function used to modulate the frequency domain characteristics; The real part sequence of the i-th Gammatone filter is convolved with the Hilbert-modulation delay kernel FIR convolution kernel by performing a term-by-term product and summing the results to generate the imaginary part sequence of the i-th filter. This is achieved by multiplying the value of each time point in the real part with the value of the modulation kernel at the corresponding position in the symmetrical interval, and summing all the products to obtain the value at the corresponding position in the imaginary part. Construct the complex form of the i-th Gammatone filter by combining the real part with the generated imaginary part in complex form, where the real part is the original response value of the filter and the imaginary part is the result of Hilbert-modulation delay kernel convolution. The Gammatone filtering module is based on the complex number g. i [n] performs multi-channel filtering on the input frame signal and outputs a frame-level complex feature sequence.

4. The intelligent audio analysis and speech recognition cloud fusion computing service platform according to claim 1, characterized in that, The SincNet feature optimization module specifically includes: Received initial audio feature sequence V k Where k represents the frame number, V k Includes frame-level multi-channel preliminary audio feature data; Initialize the heterogeneous dual-path SincNet convolutional module, including: The first path of SincNet convolutional channels: the number of convolutional kernels is fixed, the center frequency of the convolutional kernels is initialized linearly and equally at intervals across the entire frequency band, the impulse response of the convolutional kernels is windowed by the Hamming window function, and the low cutoff frequency parameters and high cutoff frequency parameters of each convolutional kernel are limited to the range of half the Nyquist frequency to the Nyquist frequency for training. The second path, SincNet convolutional channels, has a fixed number of convolutional kernels. The center frequency of the convolutional kernels is initialized according to the Mel frequency scale. The convolutional kernel impulse response is windowed by the Blackman window function. The low cutoff frequency parameters and high cutoff frequency parameters of each convolutional kernel are limited to the range of 0Hz to half of the Nyquist frequency for training. Each SincNet convolutional kernel employs an asymmetric deformable structure, and the kernel impulse response incorporates independent time-shifting parameters on both the positive and negative time axes. Statistical input frame sequence V k The frame energy distribution width forms the statistical quantity STD. frame The offset adjustment value Δδ is calculated through a gating network. i Offset adjustment value Δδ i Used to update the temporal offset parameter of the convolution kernel, ensuring that the positive and negative temporal axis offsets of the convolution kernel are adaptively adjusted according to the frame energy distribution width; After completing convolution and adaptive adjustment, the output features of the first path convolution are subjected to intra-frame local normalization, and the mean and standard deviation are calculated frame by frame for standardization; the output features of the second path convolution are subjected to inter-frame sliding weighted average smoothing. The convolutional output features of the first and second paths are weighted and fused. The fusion method includes weighted summation according to preset weighting coefficients or channel-equalized weighted summation. This summation is then further combined with the initial input audio feature sequence V. k Residual connections are superimposed to form a fusion result, generating optimized audio features.

5. The intelligent audio analysis and speech recognition cloud fusion computing service platform according to claim 1, characterized in that, The status detection module specifically includes: Receive an optimized audio feature sequence and divide the sequence into fixed-length frame groups, each frame containing a set of spectral energy features and a set of acoustic parameter features; Retrieve historical acoustic archives for the corresponding scenario from the database module, and extract the baseline frame features with the same dimensions as the current frame group, including the historical spectral energy mean sequence and the historical acoustic parameter mean sequence; For each frame, calculate the frame energy deviation between the current frame spectral energy characteristics and the historical average spectral energy, and calculate the frame parameter deviation between the current frame acoustic parameter characteristics and the historical average acoustic parameters. The frame energy deviation and frame parameter deviation are weighted and summed according to preset weights to generate a comprehensive deviation score for the frame. The overall deviation score is compared with a set threshold. If the overall deviation score is greater than the threshold, the frame is marked as an abnormal frame; otherwise, it is marked as a normal frame. Set a sliding window in consecutive frames, count the number of abnormal frames in the sliding window, and when the proportion of abnormal frames exceeds the proportion threshold, determine that the current window is in an abnormal state and generate an alarm report; otherwise, generate a normal state report. The generated status report includes the detection time, the proportion of abnormal frames, the main deviation indicators, and the reference historical file number.

6. The intelligent audio analysis and speech recognition cloud fusion computing service platform according to claim 1, characterized in that, The database module is used to store multi-channel audio data with sampling time, channel number and frame sequence number, historical acoustic archives composed of audio features output by the Gammatone filtering module and the SincNet feature optimization module, index entries including frame-level energy feature vectors, inter-channel correlation features and normalized comprehensive feature vectors, model structure configuration files and weight file summaries and their version numbers provided by the distributed recognition module, and to complete asynchronous data synchronization, incremental tracking and difference verification between the edge and cloud databases based on the center frequency range, feature dimension and scene identifier.

7. The intelligent audio analysis and speech recognition cloud fusion computing service platform according to claim 1, characterized in that, The distributed identification module specifically includes: Receive the status report output by the status detection module and parse the current detection status flag; Based on the status markers in the status report, if the status marker is normal, the edge speech recognition model is selected; if the status marker is abnormal, the cloud speech recognition model is selected. The optimized audio features output by the SincNet feature optimization module are divided into frame-level feature sequences in chronological order, and each frame feature is normalized according to a preset mean and standard deviation before being input into the selected speech recognition model. At the edge, speech recognition processing includes the following sub-steps: receiving a normalized frame-level feature sequence, using a lightweight acoustic model to calculate the classification probability distribution of each frame on a predefined set of phonemes, and selecting the phoneme with the highest probability as the frame label; merging consecutive frames with the same adjacent frame labels to generate a preliminary phoneme sequence; applying a finite state transition dictionary pre-set at the edge to match each legal word in the preliminary phoneme sequence to form the edge-recognized text result; In the cloud, speech recognition processing includes the following sub-steps: receiving a normalized frame-level feature sequence; calculating the classification probability distribution of each frame on an extended phoneme set using a full deep acoustic model; finding the optimal frame path using the Viterbi decoding algorithm; generating a preliminary frame-level phoneme label sequence; inputting the preliminary label sequence into a cloud-based context-free language model; performing context probability correction; generating a corrected phoneme sequence; and converting the corrected phoneme sequence into recognized text results according to the phoneme-to-word mapping rules. When usable recognizable text results are generated at both the edge and the cloud, the average frame-level confidence of each result is calculated separately. Compare the average frame-level confidence scores of the text recognized at the edge and in the cloud, and select the text with higher confidence scores as the final recognition result; The final recognition result record includes the frame sequence, corresponding timestamp, recognition path identifier, and average confidence score, and is output to the output module.

8. The intelligent audio analysis and speech recognition cloud fusion computing service platform according to claim 7, characterized in that, The lightweight acoustic model specifically includes: The input is a normalized frame-level feature sequence, with the input tensor dimension being (T,D), where T represents the frame sequence length and D represents the feature dimension. The input frame-level feature sequence is processed by a single-layer bidirectional long short-term memory network, and the hidden state tensor dimension is output. The hidden state vector of each frame output by a single-layer bidirectional long short-term memory network is mapped to the phoneme category space through matrix multiplication, forming a dimension (T, N) E The score tensor of ), where N E The number of predefined phoneme categories; The probability distribution of the score vector for each frame of the score tensor is calculated using the softmax function. The phoneme category with the highest probability in each frame is selected as the frame label, and the phoneme prediction results of the frame sequence are recorded as the preliminary identification results at the edge.

9. The intelligent audio analysis and speech recognition cloud fusion computing service platform according to claim 7, characterized in that, The full-volume deep acoustic model specifically includes: The input is a normalized frame-level feature sequence, with the input tensor dimension being (T,D), where T represents the frame sequence length and D represents the feature dimension. The input frame-level feature sequence is slidably convolved in the time dimension using a one-dimensional convolutional neural network with K convolution kernels and the output tensor dimension being (T,K). The convolutional output is input into a multi-layer bidirectional long short-term memory network, which has L layers. C The number of one-way hidden units in each layer is H C The final output hidden state tensor has a dimension of (T, 2H). C ); The hidden state vector of each frame output from the final layer of the multilayer bidirectional long short-term memory network is mapped to the extended phoneme category space through matrix multiplication, forming a dimension (T, N) C The frame-level score tensor of ), where N C To expand the number of phoneme categories; The frame-level score tensor is used to calculate the probability distribution of each category using softmax. The Viterbi decoding algorithm is used to solve the maximum path probability sequence of the frame-level probability matrix of the entire sequence, and the frame label sequence is output. The frame label sequence is input into the context-free n-gram language model, and the frame label sequence is corrected according to the language probability to generate the corrected phoneme sequence. Using a predefined phoneme-to-word mapping table, the phoneme sequence is converted one by one into cloud-based text recognition results, and the recognized text and frame-level confidence statistics are output.

10. The intelligent audio analysis and speech recognition cloud fusion computing service platform according to claim 1, characterized in that, The output module is used to receive the final recognition result generated by the distributed recognition module, the status report and alarm report generated by the status detection module, and to summarize the frame sequence, timestamp, recognition path and abnormal indicator information to form a standardized audio analysis and speech recognition service report and output it to the terminal interface.

Images