An intelligent audio analysis and speech recognition cloud fusion computing service platform

Abstract

The application discloses a kind of intelligent audio analysis and speech recognition cloud fusion computing service platform, including following module: audio acquisition module gathers the multichannel audio data from multiple terminal devices;Gammatone filtering module filters audio data, extracts preliminary audio features;Feature optimization module refines spectral information, generates optimized audio features;State detection module analyzes features and generates state reports or alarms in combination with historical records;Database module stores audio recordings, acoustic records and model parameters;Recognition module selects edge or cloud model for speech recognition based on state reports, and outputs recognition results;Output module receives various results and generates a unified service report.The application can realize adaptive recognition and analysis of audio information in multiple scenarios.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of speech signal processing and intelligent speech recognition, and particularly relates to an intelligent audio analysis and speech recognition cloud fusion computing service platform. BACKGROUND

[0002] With the wide application of intelligent speech technology, speech recognition and audio analysis play an increasingly key role in mobile terminals, vehicle-mounted systems, intelligent sound boxes, remote conferences and industrial voice monitoring and other scenarios. Multi-channel audio acquisition and real-time processing systems are gradually replacing traditional single-channel speech recognition architectures to meet higher requirements for speech interaction quality, system response speed and recognition accuracy in complex environments. However, in the actual deployment process, the current mainstream audio recognition system still faces many technical problems, which restricts its landing efficiency and robustness in edge-cloud collaborative computing scenarios.

[0003] In the prior art, the mainstream speech recognition system is mostly based on a fixed feature extraction algorithm, preset bandwidth filter parameters and a static model path, and relies on centralized computing resources for unified recognition. Such a method has the following disadvantages in the face of complex acoustic environments, dynamic computing conditions and diversified application requirements: the traditional audio preprocessing module uses Gammatone or mel filter with fixed parameter design, which cannot dynamically adjust the bandwidth structure according to the frame-level noise characteristics, resulting in a significant performance decline in low signal-to-noise ratio environments. The existing SincNet model often uses uniform frequency initialization and a general window function, which is difficult to model high-frequency details and low-frequency envelopes respectively, resulting in insufficient feature optimization expression. The recognition module in the current edge computing architecture is mostly a fixed model link, which fails to intelligently switch the edge-side and cloud-end model paths according to the actual state, causing resource waste or delay increase.

[0004] Therefore, how to provide an intelligent audio analysis and speech recognition cloud fusion computing service platform is a problem that those skilled in the art need to solve. SUMMARY

[0005] The present application provides an intelligent audio analysis and speech recognition cloud fusion computing service platform, which adopts a method combining multi-channel audio acquisition, Gammatone filtering, double-path SincNet feature refinement, real-time state detection and edge-cloud distributed recognition to adaptively process environmental noise and channel differences, generate high-quality optimized audio features and intelligently select edge or cloud models to complete speech recognition.

[0006] According to the intelligent audio analysis and speech recognition cloud fusion computing service platform of an embodiment of the present application, the following steps are included:

[0007] The audio acquisition module is configured to acquire multi-channel audio data from a mobile terminal, a vehicle-mounted device, a smart speaker, a conference system, and an industrial voice acquisition device.

[0008] The Gammatone filter module is configured to perform Gammatone filtering on the audio data to form preliminary audio features.

[0009] The SincNet feature optimization module is configured to perform spectral refinement on the preliminary audio features to generate optimized audio features.

[0010] The state detection module is configured to receive the optimized audio features and generate a state report in combination with historical acoustic archives stored in a database, and generate an alarm report if an abnormality is detected.

[0011] The database module is configured to store historical audio data, acoustic archives, and model parameters.

[0012] The distributed recognition module is configured to dynamically select an edge-side or cloud-side model to perform speech recognition processing on the optimized audio features according to the state report, and generate a recognition result.

[0013] The output module is configured to receive the recognition result, the state report, and the alarm report, and uniformly output an audio analysis and speech recognition service report.

[0014] Optionally, the Gammatone filter module specifically includes:

[0015] The input multi-channel audio data is divided into a sequence of frame signals, the frame length is set to Ns samples, and the starting points of adjacent frames are spaced by Ns*O samples, where Ns is a positive integer, and 0 is a frame overlap rate, and the division forms a sequence of frames , where i is a frame sequence number, and j is a frame sample point sequence number.

[0016] For each frame , the zero-crossing rate of the continuous sampling points in the frame is calculated, that is, the number of sign changes between adjacent sampling points is counted and normalized to the frame length ratio.

[0017] The frame signal-to-noise ratio SNR is calculated, which is defined as the ratio of the effective signal power of the current frame to the reference noise power.

[0018] The and are proportionally weighted to generate a hybrid noise indicator . The current frame filter bandwidth is modulated according to

[0019] ​​​​​​​​​, modulated as: wherein is an initial bandwidth, is a bandwidth modulation sensitivity coefficient;

[0020] On the basis of bandwidth modulation, it is allowed to directly trigger bandwidth contraction adjustment according to the internal statistical characteristics of input data, that is, by calculating whether the proportion of abnormal frames in the current frame set exceeds a preset threshold When the proportion of abnormal frames is greater than , the bandwidth of the current frame and the subsequent frame is immediately narrowed;

[0021] According to the adjusted bandwidth parameter, a complex-valued Gammatone filter bank is constructed, and the impulse response of each filter includes a real part and an imaginary part: the real part is defined as: on the non-negative time axis, the real part is composed of a Gamma envelope function and a center frequency modulation cosine term; the imaginary part is defined as: the real part is convolved with a Hilbert-modulated delay kernel;

[0022] After the filter bank is generated, orthogonal compression optimization is performed, which includes: judging whether the center frequency difference between adjacent Gammatone filters is less than a frequency interval threshold If it is less than the threshold, the filter weights are merged and the normalization factor is recalibrated to control the cross-correlation coefficient between any filter pairs in the Gammatone filter bank to be less than a threshold ;

[0023] The frame signal and the filter bank are convolved one by one to generate: an amplitude envelope vector , an instantaneous phase vector ;

[0024] Three types of preliminary audio features are extracted: sub-band intra-frame energy; inter-channel correlation; inter-frame phase difference;

[0025] The above three types of preliminary audio features are subjected to Z-scre normalization, and the normalized preliminary audio feature sequence is output as the final result of the Gammatone filtering module.

[0026] Optionally, the method for generating the imaginary part specifically includes:

[0027] Discrete-time sampling is performed on the real part of each Gammatone filter to generate a real part sequence , wherein represents the filter number, represents the intra-frame sampling point index;

[0028] A Hilbert-modulated delay kernel FIR convolution kernel sequence is constructed, wherein odd symmetric finite impulse response convolution kernel, odd symmetric finite impulse response convolution kernel half length, the value determines the kernel length as , discrete kernel of ideal Hilbert filter; sine window function for modulating frequency domain characteristics;

[0029] the real part sequence of the first Perform the convolution operation of the real part sequence of the first Perform the convolution operation of the real part sequence of the first

[0030] construct the complex form of the first Combine the real part and the generated imaginary part in complex form, where the real part is the original response value of the filter, and the imaginary part is the convolution result of the Hilbert-modulated delay kernel;

[0031] The Gammatone filter module is based on complex Perform multi-channel filtering operation on the input frame signal, and output frame-level complex feature sequence.

[0032] Optionally, the SincNet feature optimization module specifically includes:

[0033] receive the output preliminary audio feature sequence wherein indicates the frame number, contains frame-level multi-channel preliminary audio feature data;

[0034] Initialize the heterogeneous double-path SincNet convolution module, including:

[0035] First path SincNet convolution channel: the number of convolution kernels is fixed, the center frequency of the convolution kernel is initialized according to the full frequency band linear equal interval, the convolution kernel impulse response is windowed by Hamming window function, and the low cutoff frequency parameter and the high cutoff frequency parameter of each convolution kernel are limited to the range of half Nyquist frequency to half Nyquist frequency for training;

[0036] Second path SincNet convolution channel: the number of convolution kernels is fixed, the center frequency of the convolution kernel is initialized according to the Mel frequency scale, the convolution kernel impulse response is windowed by Blackman window function, and the low cutoff frequency parameter and the high cutoff frequency parameter of each convolution kernel are limited to the range of 0Hz to half Nyquist frequency for training;

[0037] An asymmetric deformable structure is adopted for each SincNet convolution kernel, and independent time offset parameters are introduced to the positive and negative time axes of the convolution kernel impulse response 、 ;

[0038] The frame energy distribution width of the statistical input frame sequence is calculated to form a statistical quantity , and an offset adjustment value is calculated through a gating network , which is used to update the time offset parameters of the convolution kernel to ensure that the positive and negative time axis offsets of the convolution kernel are adaptively adjusted according to the frame energy distribution width. After the convolution and adaptive adjustment are completed, the first-path convolution output features are subjected to intra-frame local normalization processing, and the mean and standard deviation are calculated for each frame to perform standardization; the second-path convolution output features are subjected to inter-frame sliding weighted average smoothing processing.

[0039] The first-path and second-path convolution output features are fused by weighting, and the fusion method includes weighted summation according to a preset weighting coefficient or channel equalization weighted summation, and further residual connection superposition with the input preliminary audio feature sequence to form a fusion result and generate optimized audio features.

[0040] Optionally, the state detection module specifically includes:

[0041] The optimized audio feature sequence is received, and the sequence is divided into fixed-length frame groups, each frame containing a group of spectral energy features and a group of acoustic parameter features.

[0042] The historical acoustic archives under the corresponding scene are called from the database module, and reference frame features with the same dimension as the current frame group are extracted, including historical spectral energy mean sequence and historical acoustic parameter mean sequence.

[0043] For each frame, the frame energy deviation between the current frame spectral energy features and the historical spectral energy mean is calculated, and the frame parameter deviation between the current frame acoustic parameter features and the historical acoustic parameter mean is calculated.

[0044] The frame energy deviation and the frame parameter deviation are weighted and summed according to a preset weight to generate a comprehensive deviation score of the frame; the comprehensive deviation score is compared with a set threshold value, and when the comprehensive deviation score is greater than the threshold value, the frame is marked as an abnormal frame, otherwise it is marked as a normal frame.

[0045] A sliding window is set in the continuous frames, and the number of abnormal frames in the sliding window is counted, and when the proportion of abnormal frames exceeds a proportion threshold value, the current window is determined to be in an abnormal state, an alarm report is generated, otherwise a normal state report is generated.

[0046] ​​

[0047] The generated status report includes detection time, abnormal frame proportion, main deviation index and reference history file number.

[0048] Optionally, the database module is configured to store multi-channel audio data with sampling time, channel number and frame sequence number, a historical acoustic archive composed of audio features output by the Gammatone filtering module and the SincNet feature optimization module, index entries including frame-level energy feature vectors, inter-channel correlation features and normalized comprehensive feature vectors, model structure configuration files and weight file summaries provided by the distributed recognition module and their version numbers, and complete asynchronous data synchronization, incremental tracking and difference checking between the edge and the cloud database based on the center frequency range, feature dimension and scene identification.

[0049] Optionally, the distributed recognition module specifically includes:

[0050] Receive the status report output by the state detection module and parse the current detection state marker;

[0051] According to the state marker in the status report, if the state marker is normal, select the edge voice recognition model, and if the state marker is abnormal, select the cloud voice recognition model;

[0052] Divide the optimized audio features output by the SincNet feature optimization module into frame-level feature sequences in chronological order, and input the normalized frame-level features into the selected voice recognition model;

[0053] On the edge, the voice recognition process includes the following sub-steps: receiving the normalized frame-level feature sequence, calculating the classification probability distribution of each frame on the predefined phoneme set using a lightweight acoustic model, and selecting the maximum probability corresponding phoneme as the frame label; merge the continuous frames with the same adjacent frame label to generate a preliminary phoneme sequence; apply the edge-preinstalled finite state transition dictionary to each preliminary phoneme sequence to match the legal word entries and form the edge recognition text result;

[0054] In the cloud, the voice recognition process includes the following sub-steps: receiving the normalized frame-level feature sequence, calculating the classification probability distribution of each frame on the extended phoneme set using a full-depth acoustic model, and using the Viterbi decoding algorithm to find the optimal frame path to generate a preliminary frame-level phoneme label sequence; input the preliminary label sequence into the cloud context-independent language model to perform context probability correction and generate a corrected phoneme sequence; convert the corrected phoneme sequence into a recognition text result according to the phoneme-to-word mapping rule;

[0055] When the edge and the cloud both generate available recognition text results, calculate the average frame-level confidence of each result respectively;

[0056] The average frame-level confidence of the text identified by the edge end and the cloud end is compared, and the text with higher confidence is selected as the final identification result.

[0057] The final identification result record includes a frame sequence, a corresponding timestamp, an identification path identifier, and an average confidence, and is output to an output module.

[0058] Optionally, the lightweight acoustic model specifically includes:

[0059] The normalized frame-level feature sequence is received as input, and the input tensor dimension is , wherein represents the frame sequence length, represents the feature dimension;

[0060] The frame-level feature sequence input is processed by a single-layer bidirectional long short-term memory network, and the output hidden state tensor dimension is

[0061] Each frame hidden state vector output by the single-layer bidirectional long short-term memory network is mapped to the phoneme class space through matrix multiplication to form a score tensor with a dimension of , wherein is a predefined phoneme class number;

[0062] The score vector of each frame of the score tensor is calculated using a softmax function to obtain a probability distribution, and the phoneme class with the maximum probability of each frame is selected as the frame label. The phoneme prediction result of the frame sequence is recorded as the edge end preliminary identification result.

[0063] Optionally, the full-depth acoustic model specifically includes:

[0064] The normalized frame-level feature sequence is received as input, and the input tensor dimension is , wherein represents the frame sequence length, represents the feature dimension;

[0065] The frame-level feature sequence input is processed by a one-dimensional convolutional neural network in the time dimension sliding convolution, the number of convolution kernels is , and the convolution output tensor dimension is ;

[0066] The convolution output is input into a multi-layer bidirectional long short-term memory network, the number of layers of the multi-layer bidirectional long short-term memory network is , the number of single-direction hidden units of each layer is , and the final output hidden state tensor dimension is ;

[0067] Each frame hidden state vector output by the final layer of the multi-layer bidirectional long short-term memory network is mapped to the extended phoneme class space through matrix multiplication to form a score tensor with a dimension of The frame-level score tensor, where To expand the number of phoneme categories;

[0068] The frame-level score tensor is used to calculate the probability distribution of each category using softmax. The Viterbi decoding algorithm is used to solve the maximum path probability sequence of the frame-level probability matrix of the entire sequence, and the frame label sequence is output.

[0069] The frame label sequence is input into the context-free n-gram language model, and the frame label sequence is corrected according to the language probability to generate the corrected phoneme sequence.

[0070] Using a predefined phoneme-to-word mapping table, the phoneme sequence is converted one by one into cloud-based text recognition results, and the recognized text and frame-level confidence statistics are output.

[0071] Optionally, the output module is used to receive the final recognition result generated by the distributed recognition module, the status report and alarm report generated by the status detection module, and to summarize information such as frame sequence, timestamp, recognition path and abnormal indicators to form a standardized audio analysis and speech recognition service report and output it to the terminal interface.

[0072] The beneficial effects of this invention are:

[0073] (1) This invention integrates Gammatone filtering with noise indicators such as zero-crossing rate and signal-to-noise ratio, and introduces a bandwidth adaptive modulation mechanism and an abnormal driving bandwidth narrowing mechanism, so that the filter bank has dynamic adjustment capability under different noise environments, which enhances the robustness and discriminability of feature extraction and significantly improves the stability and pre-processing capability of the system in unstructured sound environments.

[0074] (2) This invention constructs a dual-path heterogeneous SincNet feature optimization structure, introduces local normalization and sliding smoothing mechanisms in intra-frame and inter-frame respectively, and introduces context statistics-driven asymmetric deformable convolution kernels to adjust the path, which effectively enhances the expressive power of audio features in the time-frequency distribution dimension and adapts to multiple scene changes and different speaker features.

[0075] (3) This invention introduces a state detection module and combines it with historical acoustic archives to achieve automatic identification of abnormal sound fields and drive the distributed recognition module to dynamically select edge or cloud speech recognition models. This ensures recognition accuracy while improving the system's response speed and resource scheduling capabilities in a computing resource-constrained environment, and achieves optimal edge-cloud collaborative recognition. Attached Figure Description

[0076] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0077] Figure 1 This is a flowchart of an intelligent audio analysis and speech recognition cloud fusion computing service platform proposed in this invention;

[0078] Figure 2 This is a flowchart of the Gammatone filtering module proposed in this invention.

[0079] Figure 3 This is a flowchart illustrating the structure of the SincNet feature optimization module proposed in this invention. Detailed Implementation

[0080] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0081] refer to Figures 1-3 A cloud-based fusion computing service platform for intelligent audio analysis and speech recognition includes the following steps:

[0082] The audio acquisition module is used to acquire multi-channel audio data from mobile terminals, in-vehicle devices, smart speakers, conference systems, and industrial voice acquisition devices;

[0083] The Gammatone filtering module is used to perform Gammatone filtering on audio data to form preliminary audio features;

[0084] The SincNet feature optimization module is used to refine the spectrum of the initial audio features and generate optimized audio features.

[0085] The status detection module receives optimized audio features and combines them with historical acoustic files stored in the database to generate a status report. If an anomaly is detected, an alarm report is generated.

[0086] The database module is used to store historical audio data, acoustic archives, and model parameters;

[0087] The distributed recognition module is used to dynamically select the edge or cloud model based on the status report to perform speech recognition processing on the optimized audio features and generate recognition results;

[0088] The output module is used to receive recognition results, status reports and alarm reports, and output audio analysis and speech recognition service reports in a unified manner.

[0089] In this embodiment, the Gammatone filtering module specifically includes:

[0090] The input multi-channel audio data is divided into a sequence of frame signals, with the frame length set to [value missing]. There are sampling points, and the interval between the start points of adjacent frames is . There are 1 sampling points, among which The frame overlap rate is used to form a sequence after segmentation. ,in The frame number, The sequence number of the sampling point within the frame;

[0091] For each frame Calculate the zero-crossing rate using continuous sampling points inside. That is, counting the number of sign changes between adjacent sampling points and normalizing it to a proportion of the frame length. The zero-crossing rate is defined as:

[0092] ;

[0093] in, For symbolic functions, output , or , It captures high-frequency changes within a frame, reflecting the high-frequency component density of the frame signal;

[0094] Calculate frame signal-to-noise ratio The effective signal power is defined as the ratio of the effective signal power to the reference noise power in the current frame. The effective power is the mean of the sum of squares of the samples within the frame, and the reference noise power is... Sourced from system presets or historical no-speech-segment statistics:

[0095] ;

[0096] Where the numerator is the average power of the current frame, and the denominator is the reference noise power;

[0097] Will and Generate mixed noise index by weighting proportionally. This is used to reflect the overall noise environment within the frame:

[0098] ;

[0099] in, For the weighting coefficients, satisfying , , Controlling the contribution of low-frequency background noise Controlling the contribution of high-frequency interference, and finely characterizing the noise environment through mixed noise indicators;

[0100] according to Dynamically modulate the current frame filter bandwidth Modulated as: ,in For the initial bandwidth, The bandwidth modulation sensitivity coefficient is (0.1–0.3), which is relevant when the noise environment deteriorates. When the bandwidth is increased, it is moderately expanded to cover more frequency details; when the environment is relatively stable, the bandwidth is narrowed to enhance feature resolution.

[0101] Based on bandwidth adaptive modulation, it allows for direct triggering of bandwidth contraction adjustments based on the internal statistical characteristics of the input data, i.e., by calculating whether the proportion of abnormal frames in the current frame set exceeds a preset threshold. Defined as the mixed noise index in consecutive frames. Exceeding the preset noise threshold The frame ratio, when the abnormal frame ratio is greater than When this happens, the bandwidth of the current frame and subsequent frames is immediately narrowed, and the adjustment formula is as follows:

[0102] ;

[0103] in, This is a bandwidth narrowing factor (0.6–0.8), where the constant frame ratio is automatically generated based on intra-frame metrics.

[0104] Based on the dynamically adjusted bandwidth parameters, a complex-valued Gammatone filter bank is constructed, where the impulse response of each filter includes both real and imaginary parts:

[0105] The real part is defined as follows: On the non-negative time axis, the real part consists of the Gamma envelope function and the center frequency modulation cosine term.

[0106] ;

[0107] in, Let the filter order be . For the first The bandwidth parameters corresponding to the frame, For the first The center frequency of the filter, It is a time variable, and its domain is ;

[0108] The imaginary part is defined as the real part convolved with the Hilbert modulation delay kernel;

[0109] After the filter bank is generated, orthogonal compression optimization is performed. Orthogonal compression optimization includes: determining whether the center frequency difference between adjacent Gammatone filters is less than the frequency spacing threshold. (80–120Hz) If the value is less than the threshold, the filter weights are merged and the normalization factor is recalibrated to control the cross-correlation coefficient between any filter pair within all Gammatone filter banks to not exceed the threshold. (0.1–0.3);

[0110] Convolution operations are performed on the frame signal and the filter bank one by one to generate the amplitude envelope vector. The envelope amplitude output by the convolution of the real part of the filter; the instantaneous phase vector. The instantaneous phase output by the convolution of the imaginary part of the filter captures the characteristics of rapid phase changes;

[0111] Three types of preliminary audio features were extracted: sub-band intra-frame energy, from The sum of squares is calculated; inter-channel correlation is obtained based on the correlation coefficient matrix of multi-channel amplitude characteristics; inter-frame phase difference is calculated based on... The inter-frame difference is calculated;

[0112] Z-scre normalization is performed on the above three types of preliminary audio features so that the output features of each dimension are output in a standardized form with a mean of 0 and a standard deviation of 1, ensuring that the numerical range and distribution of the input features of subsequent modules are consistent.

[0113] The normalized preliminary audio feature sequence is output as the final result of the Gammatone filtering module. This sequence contains multi-dimensional, cross-channel preliminary audio features with amplitude and phase information, which can be used by the subsequent SincNet feature optimization module.

[0114] In this embodiment, the method for generating the imaginary part specifically includes:

[0115] Discrete-time sampling is performed on the real part of each Gammatone filter to generate a sequence of real parts. ,in Indicates the filter number. Indicates the index of the intra-frame sampling point;

[0116] Constructing Hilbert-modulation delay kernel FIR convolution kernel sequences The Hilbert-modulation delay kernel is defined as follows:

[0117] ;

[0118] in: It is an odd-symmetric finite impulse response convolution kernel; The half-length of the odd-symmetric finite impulse response convolution kernel is determined by the value of . ; For the discrete kernel of the ideal Hilbert filter; This is a sinusoidal window function used to modulate the frequency domain characteristics, thereby limiting the frequency response to the Nyquist half-band;

[0119] The first The real part sequence of the Gammatone filter is convolved with the Hilbert-modulation delay kernel FIR convolution kernel by performing a term-by-term product and summation operation to generate the _th_ ... The imaginary part sequence of the filter is obtained by multiplying the value of the real part at each time point by the value of the modulation kernel at the corresponding position in the symmetric interval, and summing all the products to get the value at the corresponding position of the imaginary part.

[0120] Construct the first The complex form of a Gammatone filter is obtained by combining the real part with the generated imaginary part in complex form, where the real part is the original response value of the filter and the imaginary part is the result of Hilbert-modulation delay kernel convolution.

[0121] The Gammatone filtering module performs multi-channel filtering operations on the input frame signal based on complex numbers, and outputs a frame-level complex feature sequence. The real part is used to extract sub-band energy features, and the imaginary part is used to calculate intra-frame instantaneous phase features, which serve as input to the subsequent SincNet feature optimization module.

[0122] In this embodiment, the SincNet feature optimization module specifically includes:

[0123] Receive the initial audio feature sequence output from the Gammatone filtering module. ,in Indicates the frame number. Includes frame-level multi-channel preliminary audio feature data;

[0124] Initialize the heterogeneous dual-path SincNet convolutional module, including:

[0125] The first path of SincNet convolutional channels: the number of convolutional kernels is fixed, the center frequency of the convolutional kernels is initialized linearly and equally at intervals across the entire frequency band, the impulse response of the convolutional kernels is windowed by the Hamming window function, and the low cutoff frequency parameters and high cutoff frequency parameters of each convolutional kernel are limited to the range of half the Nyquist frequency to the Nyquist frequency for training.

[0126] The second path, SincNet convolutional channels, has a fixed number of convolutional kernels. The center frequency of the convolutional kernels is initialized according to the Mel frequency scale. The convolutional kernel impulse response is windowed by the Blackman window function. The low cutoff frequency parameters and high cutoff frequency parameters of each convolutional kernel are limited to the range of 0Hz to half of the Nyquist frequency for training.

[0127] Each SincNet convolutional kernel employs an asymmetric deformable structure, and the kernel impulse response incorporates independent time-shifting parameters on both the positive and negative time axes. , The convolution kernel impulse response is defined as:

[0128] ;

[0129] in, For the first Strip convolution kernel in time The impulse response, Indicates a high cutoff frequency. Indicates a low cutoff frequency. , These are the positive and negative time axis offset parameters, used to adjust the shape of the convolution kernel;

[0130] Statistical input frame sequence The frame energy distribution width forms a statistical quantity. The offset adjustment value is calculated through a gating network. :

[0131] ;

[0132] in, This indicates an adjustment to the learning rate. For trainable weight matrix, The Sigmoid activation function is used, and the output value is limited to the interval [0,1]. The offset adjustment value is... Used for dynamically updating the time offset parameter of the convolution kernel: This ensures that the positive and negative time axis offsets of the convolution kernel are dynamically and adaptively adjusted according to the frame energy distribution width, forming a context-driven adaptive adjustment closed loop;

[0133] After completing the above convolution and adaptive adjustment, the output features of the first path convolution are subjected to intra-frame local normalization, and the mean and standard deviation are calculated frame by frame for standardization; the output features of the second path convolution are subjected to inter-frame sliding weighted average smoothing.

[0134] The convolutional output features of the first and second paths are weighted and fused. The fusion method includes weighted summation according to preset weighting coefficients or channel-equalized weighted summation. This can be further combined with the initial input audio feature sequence. Residual connections are superimposed to form a fusion result;

[0135] Optimized audio features are generated as the final output of the SincNet feature optimization module.

[0136] In this embodiment, the state detection module specifically includes:

[0137] Receive an optimized audio feature sequence and divide the sequence into fixed-length frame groups, each frame containing a set of spectral energy features and a set of acoustic parameter features;

[0138] Retrieve historical acoustic archives for the corresponding scenario from the database module, and extract the baseline frame features with the same dimensions as the current frame group, including the historical spectral energy mean sequence and the historical acoustic parameter mean sequence;

[0139] For each frame, calculate the frame energy deviation between the current frame spectral energy characteristics and the historical spectral energy average, and calculate the frame parameter deviation between the current frame acoustic parameter characteristics and the historical acoustic parameter average. The frame energy deviation is defined as the average of the absolute values ​​of the energy differences in each frequency band, and the frame parameter deviation is defined as the average of the absolute values ​​of the differences in each parameter value.

[0140] The frame energy deviation and frame parameter deviation are weighted and summed according to preset weights to generate a comprehensive deviation score for the frame. The comprehensive deviation score is compared with a set threshold. If the comprehensive deviation score is greater than the threshold, the frame is marked as an abnormal frame; otherwise, it is marked as a normal frame.

[0141] Set a sliding window in consecutive frames, count the number of abnormal frames in the window, and when the proportion of abnormal frames exceeds the proportion threshold, determine that the current window is in an abnormal state and generate an alarm report; otherwise, generate a normal state report.

[0142] The generated report includes the detection time, the proportion of abnormal frames, the main deviation indicators, and the reference historical file number, and the report is transmitted to the distributed recognition module and the output module.

[0143] In this embodiment, the database module is used to store multi-channel audio data with sampling time, channel number and frame sequence number, historical acoustic archives composed of audio features output by the Gammatone filtering module and the SincNet feature optimization module, index entries including frame-level energy feature vectors, inter-channel correlation features and normalized comprehensive feature vectors, model structure configuration files and weight file summaries and their version numbers provided by the distributed recognition module, and to complete asynchronous data synchronization, incremental tracking and difference verification between the edge and cloud databases based on the center frequency range, feature dimensions and scene identifiers.

[0144] In this embodiment, the distributed identification module specifically includes:

[0145] Receive the status report output by the status detection module and parse the current detection status flag;

[0146] Based on the status markers in the status report, if the status marker is normal, the edge speech recognition model is selected; if the status marker is abnormal, the cloud speech recognition model is selected.

[0147] The optimized audio features output by the SincNet feature optimization module are divided into frame-level feature sequences in chronological order, and each frame feature is normalized according to a preset mean and standard deviation before being input into the selected speech recognition model.

[0148] At the edge, speech recognition processing includes the following sub-steps: receiving a normalized frame-level feature sequence, using a lightweight acoustic model to calculate the classification probability distribution of each frame on a predefined set of phonemes, and selecting the phoneme with the highest probability as the frame label; merging consecutive frames with the same adjacent frame labels to generate a preliminary phoneme sequence; applying a finite state transition dictionary pre-set at the edge to match each legal word in the preliminary phoneme sequence to form the edge-recognized text result;

[0149] In the cloud, speech recognition processing includes the following sub-steps: receiving a normalized frame-level feature sequence; calculating the classification probability distribution of each frame on an extended phoneme set using a full deep acoustic model; finding the optimal frame path using the Viterbi decoding algorithm; generating a preliminary frame-level phoneme label sequence; inputting the preliminary label sequence into a cloud-based context-free language model; performing context probability correction; generating a corrected phoneme sequence; and converting the corrected phoneme sequence into recognized text results according to the phoneme-to-word mapping rules.

[0150] When usable recognizable text results are generated at both the edge and the cloud, the average frame-level confidence of each result is calculated. The confidence calculation formula is as follows:

[0151] ;

[0152] in, Indicates the first Frame confidence value, Predict the probability of the highest category in this frame. Predict the sum of probabilities for all categories in this frame. Total number of categories;

[0153] Compare the average frame-level confidence scores of the text recognized at the edge and in the cloud, and select the text with higher confidence scores as the final recognition result;

[0154] The final recognition result record includes frame sequence, corresponding timestamp, recognition path identifier and average confidence level, and is output to the output module to generate an audio analysis and speech recognition service report.

[0155] In this embodiment, the lightweight acoustic model specifically includes:

[0156] The input is a normalized frame-level feature sequence, and the input tensor has a dimension of . ,in Indicates the length of the frame sequence. Indicates the feature dimension;

[0157] The input sequence is processed through a single-layer bidirectional long short-term memory network, and the output hidden state tensor has a dimension of [missing value]. ,in This represents the number of one-way hidden units;

[0158] The hidden state vector of each frame output by a single-layer bidirectional long short-term memory network is mapped to the phoneme category space through matrix multiplication, forming a dimension of The score tensor, where The number of predefined phoneme categories;

[0159] The probability distribution of the score vector for each frame of the score tensor is calculated using the softmax function; the phoneme category with the highest probability in each frame is selected as the frame label, and the phoneme prediction results of the frame sequence are recorded as the preliminary identification results at the edge.

[0160] In this embodiment, the full-volume deep acoustic model specifically includes:

[0161] The input is a normalized frame-level feature sequence, and the input tensor dimension is... ,in Indicates the length of the frame sequence. Indicates feature dimension;

[0162] The input sequence is subjected to sliding convolution in the time dimension using a one-dimensional convolutional neural network, with the number of convolutional kernels being [number missing]. The convolution output tensor has a dimension of ;

[0163] The convolutional output is input into a multi-layer bidirectional long short-term memory network. The number of layers in the multi-layer bidirectional long short-term memory network is... The number of one-way hidden units in each layer is The final output hidden state tensor has a dimension of . ;

[0164] The hidden state vector of each frame output from the final layer of the multilayer bidirectional long short-term memory network is mapped to an extended phoneme category space through matrix multiplication, forming a dimension of The frame-level score tensor, where To expand the number of phoneme categories;

[0165] The frame-level score tensor is used to calculate the probability distribution of each category using softmax. The Viterbi decoding algorithm is used to solve the maximum path probability sequence of the frame-level probability matrix of the entire sequence, and the frame label sequence is output.

[0166] The frame label sequence is input into the context-free n-gram language model, and the frame label sequence is corrected according to the language probability to generate the corrected phoneme sequence.

[0167] Using a predefined phoneme-to-word mapping table, the phoneme sequence is converted one by one into cloud-based text recognition results, and the recognized text and frame-level confidence statistics are output.

[0168] In this embodiment, the output module is used to receive the final recognition result generated by the distributed recognition module, the status report and alarm report generated by the status detection module, and to summarize information such as frame sequence, timestamp, recognition path and abnormal indicators to form a standardized audio analysis and speech recognition service report and output it to the terminal interface.

[0169] Example:

[0170] To verify the effectiveness and practical application value of the intelligent audio analysis and speech recognition cloud fusion computing service platform proposed in this invention, this paper conducts a deployment test in a large urban rail transit dispatch center in China as a pilot scenario. Trial operation began in the first quarter of 2025. Due to the dense voice traffic, complex noise interference, and diverse equipment interface types in the rail transit dispatch environment, traditional speech recognition systems face problems such as low accuracy, high latency, and poor scalability in this scenario. Therefore, there is an urgent need for an intelligent speech service platform that combines refined audio feature processing, dynamic state perception, and distributed recognition capabilities to support efficient dispatching and real-time risk response.

[0171] In this embodiment, the overall system of the present invention is deployed between the front-end voice acquisition node of the dispatch center and the back-end cloud computing center. Multiple voice sources, such as the dispatch console, conference microphones, trackside equipment, and passenger assistance system, are uniformly connected through the audio acquisition module, forming a multi-channel audio input data stream. The Gammatone filtering module preprocesses these raw audio data at the edge nodes, extracting frame-level preliminary audio features, including amplitude envelope, instantaneous phase, and inter-channel correlation. During the inference phase, this module implements a hybrid noise index based on the joint construction of frame-level SNR and ZCR, and adaptively adjusts the filter bandwidth. It also constructs a complex filter output with the Hilbert-modulation delay kernel, improving energy stability and resolution under strongly non-stationary backgrounds such as subway cars. The processed features, after orthogonal compression, significantly reduce the feature dimensionality, improving the throughput efficiency of subsequent modules.

[0172] The SincNet feature optimization module further refines the spectrum locally. This embodiment uses a heterogeneous dual-path SincNet structure: one path initializes the convolutional kernel with a linear frequency scale multiplied by a Hamming window to learn the cutoff frequency of the upper half of the frequency band; the other path uses a Mel scale multiplied by a Blackman window to learn the cutoff frequency of the lower half of the frequency band. The output of the first path is processed by intra-frame normalization to enhance local energy contrast, while the output of the second path is processed by inter-frame weighted smoothing to improve continuity. The two are then fused to form optimized audio features. This module's unique asymmetric deformable kernel structure adaptively adjusts the offset of the positive and negative time axes according to the intra-frame energy variance, resulting in stronger temporal sensitivity and a significant improvement in the recognition rate of high semantic density phrases such as "drive out" and "stop" in scheduling instructions.

[0173] The status detection module dynamically compares the optimized audio features with historical acoustic archives to generate a status report for each time period. The historical acoustic archives are provided by the database module and include feature distributions under different device environments, typical anomaly samples, and scheduling scenario labels. If significant deviations are detected (e.g., a sharp increase in the proportion of input audio noise, or a shift in the mean spectrum), the module automatically generates an alarm report and uploads it to the central cloud platform. By cross-referencing with previous waveform samples in the database, the system can infer whether the anomaly originates from equipment failure, human intervention, or sudden environmental changes.

[0174] The distributed recognition module selects the recognition path based on the current status report. If the status is marked as "normal," it calls the local edge recognition model, responds quickly, and outputs the command text. If the status is marked as "abnormal," it switches to the cloud-based recognition model, using a high-capacity deep network for redundant recognition and context reconstruction. The entire process maintains millisecond-level switching, requires no manual intervention, and significantly improves the system's robustness and scalability.

[0175] The output module uniformly summarizes the recognition results, status reports, and alarm reports, and outputs a comprehensive service report in a structured manner based on metadata such as timestamps, audio source IDs, and recognition path markers. This report not only supports standard interface input to the rail transit command system, but also supports historical data backtracking and safety log auditing, providing data support for subsequent risk tracing and model evaluation.

[0176] To quantify the technical effectiveness of this invention in practical applications, the dispatch center conducted a three-month comparative experiment from April to June 2025. The experiment involved routine rail transit dispatching tasks, covering approximately 200 hours of voice data. The traditional MFCC+ deep acoustic model-based speech recognition system currently in use was selected as the control group, while the platform of this invention was selected as the experimental group. Key evaluation metrics included speech recognition accuracy (WER inverse ratio), average recognition latency, anomaly response time, feature processing time, and overall system satisfaction.

[0177] The following are the statistical results of the experimental data:

[0178] Table 1 Comparison of traditional solutions and the platform of this invention in scheduling scenarios.

[0179] Indicator Conventional system average Invention platform average Lift Recognition accuracy (%) 85.7 92.8 +7.1% Recognition latency (ms) 480 290 -39.6% Average time to respond to anomaly (s) 3.6 1.4 -61.1% Gammatone processing duration (ms) 55 68 +23.6% (acceptable) False positive rate for state detection (%) Not supported 2.3 / Satisfaction score (10-point scale) 7.5 9.1 +1.6

[0180] As can be seen from the data table, during peak working hours with strong noise interference, the optimized audio features of this invention can improve the recognition accuracy by 7.1 percentage points, with a particularly significant improvement in command recognition such as "stop immediately" and "switch tracks eastward" for short, high-instruction-density statements. The core factor for the significant reduction in latency is that the system can dynamically switch the edge recognition path, eliminating the need to upload to the cloud in most normal scenarios and avoiding unnecessary transmission delays. Through bandwidth adaptation and asymmetric convolution mechanisms, the recognition performance in abnormal scenarios did not decrease significantly, while the abnormal response time was reduced to 38.9% of the original. Although the Gammatone processing time of this platform increased slightly, the average single-frame processing latency was still controlled within an acceptable range of less than 70ms, and because this module supports batch processing, it actually has the advantage of parallel processing in multi-channel scenarios.

[0181] Within three months, the system identified 23,416 dispatch passwords, automatically recorded 34,288 status reports, and generated 352 alarm reports. The false alarm rate, after manual verification, was less than 3%. The system maintained a stable operating rate of 99.97%, with no hardware overload or model malfunctions observed. Dispatch and command personnel generally reported fast identification and response speeds, low false identification rates, and support for log querying and accountability retrospective functions, significantly improving the security of the dispatch process.

[0182] In summary, the intelligent audio analysis and speech recognition cloud fusion computing service platform proposed in this invention significantly outperforms existing traditional solutions in terms of audio feature processing accuracy, speech recognition accuracy, system response speed, and anomaly detection, demonstrating good engineering feasibility and practical deployment value. Through its modular structure and adaptive mechanism, the system exhibits strong scalability and is suitable for high-noise, multi-source, and high-density voice scenarios, providing efficient voice service solutions for fields such as rail transit, industrial control, and intelligent customer service.

[0183] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A cloud-based fusion computing service platform for intelligent audio analysis and speech recognition, characterized in that, It includes an audio acquisition module, a Gammatone filtering module, a SincNet feature optimization module, a state detection module, a database module, a distributed recognition module, and an output module; The audio acquisition module is used to acquire multi-channel audio data from mobile terminals, in-vehicle devices, smart speakers, conference systems, and industrial voice acquisition devices; The Gammatone filtering module is used to perform Gammatone filtering on audio data to form preliminary audio features; The SincNet feature optimization module is used to refine the spectrum of the initial audio features and generate optimized audio features. The status detection module receives optimized audio features and combines them with historical acoustic files stored in the database to generate a status report. If an anomaly is detected, an alarm report is generated. The database module is used to store historical audio data, acoustic archives, and model parameters; The distributed recognition module is used to dynamically select the edge or cloud model based on the status report to perform speech recognition processing on the optimized audio features and generate recognition results; The output module is used to receive recognition results, status reports and alarm reports, and output audio analysis and speech recognition service reports in a unified manner; Receive the status report output by the status detection module and parse the current detection status flag; Based on the status markers in the status report, if the status marker is normal, the edge speech recognition model is selected; if the status marker is abnormal, the cloud speech recognition model is selected. The optimized audio features output by the SincNet feature optimization module are divided into frame-level feature sequences in chronological order, and each frame feature is normalized according to a preset mean and standard deviation before being input into the selected speech recognition model. At the edge, speech recognition processing includes the following sub-steps: receiving a normalized frame-level feature sequence, using a lightweight acoustic model to calculate the classification probability distribution of each frame on a predefined set of phonemes, and selecting the phoneme with the highest probability as the frame label; merging consecutive frames with the same adjacent frame labels to generate a preliminary phoneme sequence; applying a finite state transition dictionary pre-set at the edge to match each legal word in the preliminary phoneme sequence to form the edge-recognized text result; In the cloud, speech recognition processing includes the following sub-steps: receiving a normalized frame-level feature sequence; calculating the classification probability distribution of each frame on an extended phoneme set using a full deep acoustic model; finding the optimal frame path using the Viterbi decoding algorithm; generating a preliminary frame-level phoneme label sequence; inputting the preliminary label sequence into a cloud-based context-free language model; performing context probability correction; generating a corrected phoneme sequence; and converting the corrected phoneme sequence into recognized text results according to the phoneme-to-word mapping rules. When usable recognizable text results are generated at both the edge and the cloud, the average frame-level confidence of each result is calculated separately. Compare the average frame-level confidence scores of the text recognized at the edge and in the cloud, and select the text with higher confidence scores as the final recognition result; The final recognition result record includes the frame sequence, corresponding timestamp, recognition path identifier, and average confidence score, and is output to the output module.

2. The intelligent audio analysis and speech recognition cloud fusion computing service platform according to claim 1, characterized in that, The Gammatone filtering module specifically includes: The input multi-channel audio data is divided into a sequence of frame signals, with the frame length set to [value missing]. There are sampling points, and the interval between the start points of adjacent frames is . There are 1 sampling points, among which The frame overlap rate is used to form a sequence after segmentation. ,in For frame number, The sequence number of the sampling point within the frame; For each frame Calculate the zero-crossing rate using continuous sampling points inside. That is, to count the number of sign changes between adjacent sampling points and normalize them to the frame length ratio; Calculate frame signal-to-noise ratio , defined as the ratio of the effective signal power of the current frame to the reference noise power; Will and Generate mixed noise index by weighting proportionally. ; according to Modulate the current frame filter bandwidth Modulated as: ,in For the initial bandwidth, This is the bandwidth modulation sensitivity coefficient; Based on bandwidth modulation, bandwidth contraction adjustment can be directly triggered according to the internal statistical characteristics of the input data, that is, by calculating whether the proportion of abnormal frames in the current frame set exceeds a preset threshold. When the proportion of abnormal frames is greater than When this happens, the bandwidth of the current frame and subsequent frames is immediately narrowed. Based on the adjusted bandwidth parameters, a complex-valued Gammatone filter bank is constructed. The impulse response of each filter includes a real part and an imaginary part: the real part is defined as the Gamma envelope function and the center frequency modulation cosine term on the non-negative time axis; the imaginary part is defined as the real part convolved with the Hilbert modulation delay kernel. After the Gammatone filter bank is generated, orthogonal compression optimization is performed. Orthogonal compression optimization includes: determining whether the center frequency difference between adjacent Gammatone filters is less than the frequency spacing threshold. If the cross-correlation coefficient between any two filter pairs within the Gammatone filter bank is less than the threshold, the filter weights are merged and the normalization factor is recalibrated to ensure that the cross-correlation coefficient between any two filter pairs within the Gammatone filter bank does not exceed the threshold. ; Convolution operations are performed on the frame signal and the filter bank one by one to generate the amplitude envelope vector. Instantaneous phase vector ; Three types of preliminary audio features were extracted: sub-band intra-frame energy; inter-channel correlation; and inter-frame phase difference. The three types of preliminary audio features are Z-scre normalized, and the normalized preliminary audio feature sequence is output as the final result of the Gammatone filtering module.

3. The intelligent audio analysis and speech recognition cloud fusion computing service platform according to claim 2, characterized in that, The method for generating the imaginary part specifically includes: Discrete-time sampling is performed on the real part of each Gammatone filter to generate a sequence of real parts. ,in Indicates the filter number. Indicates the index of the intra-frame sampling point; Constructing Hilbert-modulation delay kernel FIR convolution kernel sequences ,in For odd-symmetric finite impulse response convolution kernels, The half-length of the odd-symmetric finite impulse response convolution kernel is determined by the value of . , For the discrete kernel of the ideal Hilbert filter; This is a sinusoidal window function used to modulate the frequency domain characteristics; The first The real part sequence of the Gammatone filter is convolved with the Hilbert-modulation delay kernel FIR convolution kernel by performing a term-by-term product and summation operation to generate the _th_ ... The imaginary part sequence of the filter is obtained by multiplying the value of the real part at each time point by the value of the modulation kernel at the corresponding position in the symmetric interval, and summing all the products to get the value at the corresponding position of the imaginary part. Construct the first The complex form of a Gammatone filter is obtained by combining the real part with the generated imaginary part in complex form, where the real part is the original response value of the filter and the imaginary part is the result of Hilbert-modulation delay kernel convolution. The Gammatone filtering module is based on complex numbers. Perform multi-channel filtering on the input frame signal and output a frame-level complex feature sequence.

4. The intelligent audio analysis and speech recognition cloud fusion computing service platform according to claim 1, characterized in that, The SincNet feature optimization module specifically includes: Preliminary audio feature sequence of the received output ,in Indicates the frame number. Includes frame-level multi-channel preliminary audio feature data; Initialize the heterogeneous dual-path SincNet convolutional module, including: The first path of SincNet convolutional channels: the number of convolutional kernels is fixed, the center frequency of the convolutional kernels is initialized linearly and equally at intervals across the entire frequency band, the impulse response of the convolutional kernels is windowed by the Hamming window function, and the low cutoff frequency parameters and high cutoff frequency parameters of each convolutional kernel are limited to the range of half the Nyquist frequency to the Nyquist frequency for training. The second path, SincNet convolutional channels, has a fixed number of convolutional kernels. The center frequency of the convolutional kernels is initialized according to the Mel frequency scale. The convolutional kernel impulse response is windowed by the Blackman window function. The low cutoff frequency parameters and high cutoff frequency parameters of each convolutional kernel are limited to the range of 0Hz to half of the Nyquist frequency for training. Each SincNet convolutional kernel employs an asymmetric deformable structure, and the kernel impulse response incorporates independent time-shifting parameters on both the positive and negative time axes. , ; Statistical input frame sequence The frame energy distribution width forms a statistical quantity. The offset adjustment value is calculated through a gating network. Offset adjustment value Used to update the temporal offset parameter of the convolution kernel, ensuring that the positive and negative temporal axis offsets of the convolution kernel are adaptively adjusted according to the frame energy distribution width; After completing convolution and adaptive adjustment, the output features of the first path convolution are subjected to intra-frame local normalization, and the mean and standard deviation are calculated frame by frame for standardization; the output features of the second path convolution are subjected to inter-frame sliding weighted average smoothing. The convolutional output features of the first and second paths are weighted and fused. The fusion method includes weighted summation according to preset weighting coefficients or channel-equalized weighted summation. This is then further combined with the initial input audio feature sequence. Residual connections are superimposed to form a fusion result, generating optimized audio features.

5. The intelligent audio analysis and speech recognition cloud fusion computing service platform according to claim 1, characterized in that, The status detection module specifically includes: Receive an optimized audio feature sequence and divide the sequence into fixed-length frame groups, each frame containing a set of spectral energy features and a set of acoustic parameter features; Retrieve historical acoustic archives for the corresponding scenario from the database module, and extract the baseline frame features with the same dimensions as the current frame group, including the historical spectral energy mean sequence and the historical acoustic parameter mean sequence; For each frame, calculate the frame energy deviation between the current frame spectral energy characteristics and the historical average spectral energy, and calculate the frame parameter deviation between the current frame acoustic parameter characteristics and the historical average acoustic parameters. The frame energy deviation and frame parameter deviation are weighted and summed according to a preset weight to generate a comprehensive deviation score for the frame; the comprehensive deviation score is compared with a set threshold, and if the comprehensive deviation score is greater than the threshold, the frame is marked as an abnormal frame, otherwise it is marked as a normal frame; Set a sliding window in consecutive frames, count the number of abnormal frames in the sliding window, and when the proportion of abnormal frames exceeds the proportion threshold, determine that the current window is in an abnormal state and generate an alarm report; otherwise, generate a normal state report. The generated status report includes the detection time, the proportion of abnormal frames, the main deviation indicators, and the reference historical file number.

6. The intelligent audio analysis and speech recognition cloud fusion computing service platform according to claim 1, characterized in that, The database module is used to store multi-channel audio data with sampling time, channel number and frame sequence number, historical acoustic archives composed of audio features output by the Gammatone filtering module and the SincNet feature optimization module, index entries including frame-level energy feature vectors, inter-channel correlation features and normalized comprehensive feature vectors, model structure configuration files and weight file summaries and their version numbers provided by the distributed recognition module, and to complete asynchronous data synchronization, incremental tracking and difference verification between the edge and cloud databases based on the center frequency range, feature dimension and scene identifier.

7. The intelligent audio analysis and speech recognition cloud fusion computing service platform according to claim 1, characterized in that, The lightweight acoustic model specifically includes: The input is a normalized frame-level feature sequence, and the input tensor dimension is... ,in Indicates the length of the frame sequence. Indicates the feature dimension; The input frame-level feature sequence is processed by a single-layer bidirectional long short-term memory network, and the hidden state tensor dimension is output. The hidden state vector of each frame output by a single-layer bidirectional long short-term memory network is mapped to the phoneme category space through matrix multiplication, forming a dimension of The score tensor, where The number of predefined phoneme categories; The probability distribution of the score vector for each frame of the score tensor is calculated using the softmax function. The phoneme category with the highest probability in each frame is selected as the frame label, and the phoneme prediction results of the frame sequence are recorded as the preliminary identification results at the edge.

8. The intelligent audio analysis and speech recognition cloud fusion computing service platform according to claim 1, characterized in that, The full-volume deep acoustic model specifically includes: The input is a normalized frame-level feature sequence, and the input tensor dimension is... ,in Indicates the length of the frame sequence. Indicates the feature dimension; The input frame-level feature sequence is subjected to sliding convolution in the time dimension using a one-dimensional convolutional neural network, with the number of convolutional kernels being... The convolution output tensor has a dimension of ; The convolutional output is input into a multi-layer bidirectional long short-term memory network. The number of layers in the multi-layer bidirectional long short-term memory network is... The number of one-way hidden units in each layer is The final output hidden state tensor has a dimension of . ; The hidden state vector of each frame output from the final layer of the multilayer bidirectional long short-term memory network is mapped to an extended phoneme category space through matrix multiplication, forming a dimension of The frame-level score tensor, where To expand the number of phoneme categories; The frame-level score tensor is used to calculate the probability distribution of each category using softmax. The Viterbi decoding algorithm is used to solve the maximum path probability sequence of the frame-level probability matrix of the entire sequence, and the frame label sequence is output. The frame label sequence is input into the context-free n-gram language model, and the frame label sequence is corrected according to the language probability to generate the corrected phoneme sequence. Using a predefined phoneme-to-word mapping table, the phoneme sequence is converted one by one into cloud-based text recognition results, and the recognized text and frame-level confidence statistics are output.

9. The intelligent audio analysis and speech recognition cloud fusion computing service platform according to claim 1, characterized in that, The output module is used to receive the final recognition result generated by the distributed recognition module, the status report and alarm report generated by the status detection module, and to summarize the frame sequence, timestamp, recognition path and abnormal indicator information to form a standardized audio analysis and speech recognition service report and output it to the terminal interface.

Images