Real-time voice quality inspection and risk early warning system combined with machine learning

Abstract

This invention relates to the field of speech recognition technology, specifically to a real-time speech quality inspection and risk warning system combining machine learning. The system includes: a speech packet loss masking module, used to extract missing time periods based on the signaling sequence number fragmentation of the real-time speech data stream, and to extract the preceding received frequency domain feature matrix before the fragmentation; for the missing time periods, zero vectors are filled at the corresponding fragmentation positions of the preceding received frequency domain feature matrix to generate a damaged acoustic frequency domain feature matrix. In this invention, the calculation of the observation state probability distribution, extraction of the acoustic evolution trajectory, and generation of recognized text characters are performed based on the seamless reconstruction of the frequency domain feature matrix. This improves the stability of text restoration in mixed scenarios of packet loss, fragmented speech, and restored speech. Furthermore, when the matching degree between the recognized text characters and the characters built into the speech quality inspection lexicon exceeds a threshold, a trigger command is generated, enhancing the timeliness of real-time speech quality inspection in intercepting risky statements.

Description

Technical Field

[0001] This invention relates to the field of speech recognition technology, and in particular to a real-time speech quality inspection and risk warning system that combines machine learning. Background Technology

[0002] Speech recognition technology involves converting speech signals into recognizable and processable text information.

[0003] Current speech recognition technologies typically focus on converting speech signals into text information. In practice, they prioritize acoustic feature extraction, pronunciation state recognition, and text output. However, when faced with issues like lost speech frames, broken signaling sequence numbers, short interruptions, and recovery in real-time transmission links, missing segments are easily treated as silence, noise, or unrecognizable fragments. This leads to breaks in acoustic features over time, affecting the continuity of subsequent pronunciation state judgments. Furthermore, when these breaks occur near keywords, negation words, monetary terms, promises, or risk warnings, text output may suffer from missing characters, misclassification of words, and semantic shifts. For example, if a customer's voice includes expressions like "do not process" or "immediate refund," and the intermediate syllables are affected by packet loss, a standard transcription process might only output incomplete text like "process" or "refund," causing quality control judgments to deviate from the true semantic meaning. Therefore, improvements are needed. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by proposing a real-time voice quality inspection and risk warning system that combines machine learning.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: a real-time voice quality inspection and risk warning system combining machine learning includes: The voice packet loss masking module is used to extract missing time periods based on the signaling sequence number faults of the real-time voice data stream and extract the preceding received frequency domain feature matrix before the fault. For the missing time periods, zero vectors are filled at the corresponding fault positions of the preceding received frequency domain feature matrix to generate a damaged acoustic frequency domain feature matrix. The frequency domain feature extrapolation module is used to construct acoustic dynamic variables based on the frequency domain amplitude parameters corresponding to the damaged acoustic frequency domain feature matrix, perform state evolution prediction on the acoustic dynamic variables to obtain extrapolated feature sequences, and concatenate the extrapolated feature sequences with the pre-received frequency domain feature matrix in time sequence to generate a continuous extrapolated frequency domain feature matrix. The cross-blinding reconstruction module is used to extract and recover the subsequent received frequency domain feature matrix, and extract the intersection feature values ​​of the matrix and the continuous extrapolated frequency domain feature matrix within a preset overlap window; establish a smooth blanking weight table containing positive weight coefficients and negative weight coefficients, and perform a summation and merging operation on the two sets of intersection feature values ​​in combination with the positive weight coefficients and negative weight coefficients to generate a seamless reconstruction frequency domain feature matrix. The risk speech decoding module is used to input the seamlessly reconstructed frequency domain feature matrix into the acoustic model for mapping, calculate the observed state probability distribution value, extract the acoustic evolution trajectory through state space search, and decode and generate recognized text characters in combination with the language model; compare the recognized text characters with the built-in characters of the speech quality inspection lexicon, and generate a trigger command when the matching degree exceeds the threshold.

[0006] Preferably, the step of obtaining the damaged acoustic frequency domain feature matrix is ​​as follows: Read the transmission protocol signaling sequence numbers arranged in the arrival order within the real-time voice data stream. Select the transmission protocol signaling sequence number of the current signaling and the transmission protocol signaling sequence number of the preceding signaling that arrives immediately before the current signaling. Calculate the sequence number difference between adjacent transmission protocol signaling sequence numbers. Subtract the normal continuous increment interval from the sequence number difference to obtain the sequence number gap span of the corresponding signaling position. Compare the sequence number gap span item by item with a continuity threshold. Filter out signaling positions where the sequence number gap span exceeds the continuity threshold. Read the signaling arrival time before and after the filtered signaling position. Determine the missing voice data boundary according to the end time of the preceding signaling to the start time of the current signaling to obtain the missing time period. Based on the missing time period, the starting time boundary of the real-time voice data stream interruption is determined. The real-time voice data stream is backtracked to before the starting time boundary according to the time frame arrangement order. Continuous voice frames that have been received and whose timestamps are earlier than the starting time boundary are extracted. Voice frames with duplicate timestamps, inconsistent time frame lengths, or inconsistent frequency points are removed. Each voice frame is arranged from earliest to latest according to the timestamp of the retained voice frames. The frequency domain feature values ​​of each voice frame on all frequency points are read. The frequency domain feature values ​​of the same voice frame are arranged into row vectors. Each row vector is combined row by row according to the time frame arrangement order to generate the pre-received frequency domain feature matrix. The number of frequency point dimensions, time frame interval, and end timestamp of the pre-received frequency domain feature matrix are read. The number of time frames to be filled is determined according to the start time boundary, end time boundary, and time frame interval of the missing time period. For each time frame to be filled, a zero vector with a length equal to the number of frequency point dimensions and all dimensions having zero values ​​is created. All zero vectors are arranged in chronological order to the corresponding break position of the pre-received frequency domain feature matrix, so that the timestamps of the zero vectors sequentially cover the missing time period. The timestamp interval of adjacent time frames after expansion is verified to be equal to the time frame interval. The original frequency domain feature values ​​in the pre-received frequency domain feature matrix are retained to generate the damaged acoustic frequency domain feature matrix.

[0007] Preferably, the steps for obtaining the acoustic dynamic variables are as follows: Based on the damaged acoustic frequency domain feature matrix, locate the continuous time frame corresponding to the preceding real-received frequency domain feature matrix before filling the zero vector. Read the complex feature values ​​at each frequency point position in each continuous time frame, extract the real and imaginary parts of the complex feature values ​​respectively, sum and take the square root of the squares of the real and imaginary parts, arrange the square root results of each frequency point position according to the time frame order to obtain the frequency domain amplitude parameter, calculate the first-order change value of the frequency domain amplitude parameter adjacent to the same frequency point position, and then calculate the second-order change value of the adjacent first-order change value. Bind the frequency domain amplitude parameter, the first-order change value, and the second-order change value item by item according to the corresponding time frame and frequency point position to generate acoustic dynamic variables.

[0008] Preferably, the step of obtaining the continuous extrapolation frequency domain feature matrix is ​​as follows: Based on the acoustic dynamics variables, continuous historical states are extracted for each frequency point position in time frame order. The state backtracking range is determined based on the number of available time frames for continuous historical states. The frequency domain amplitude parameters, first-level change values, and second-level change values ​​corresponding to each time frame within the state backtracking range are arranged sequentially. The state action coefficients corresponding to each historical time frame are adjusted to minimize the cumulative squared deviation of the historical time frame estimated values ​​relative to the corresponding actual received values. The predicted feature values ​​at the starting position of the missing time period are calculated according to the determined state action coefficients. The predicted feature values ​​obtained each time are added to the continuous historical states. The subsequent predicted feature values ​​are repeatedly calculated along the time step direction until the predicted timestamp reaches the termination boundary of the missing time period. All predicted feature values ​​are arranged according to the predicted timestamps to obtain the inferred feature sequence. Retrieve the feature values ​​corresponding to the preceding actual received frequency domain feature matrix from the damaged acoustic frequency domain feature matrix, retain the original time frame order, frequency point arrangement order, and number of feature dimensions of the preceding actual received frequency domain feature matrix, verify whether the frequency point arrangement order and number of feature dimensions of the extrapolated feature sequence correspond to the preceding actual received frequency domain feature matrix, rearrange the extrapolated feature sequence according to the frequency point index of the preceding actual received frequency domain feature matrix for inconsistent positions, delete the padding zero vectors covering the missing time period in the damaged acoustic frequency domain feature matrix, attach the rearranged extrapolated feature sequence to the end time frame of the preceding actual received frequency domain feature matrix, configure continuous timestamps according to a uniform time frame interval, and generate a continuous extrapolated frequency domain feature matrix.

[0009] Preferably, the step of obtaining the subsequent received frequency domain feature matrix is ​​as follows: Based on the subsequent real-time voice data stream that arrives after the missing time period, the end time boundary of the missing time period is used as the starting point for recovery reception. Voice frames that arrive continuously after the starting point for recovery reception are read. The timestamp, frame length, number of frequency points, and feature dimensions of each voice frame are checked. Voice frames with duplicate timestamps, missing frame lengths, and inconsistent number of frequency points are removed. The retained voice frames are arranged in order of earliest to latest timestamp. The frequency domain feature values ​​of each voice frame at each frequency point are extracted. The frequency domain feature values ​​of the same voice frame are arranged into row vectors. All row vectors are combined row by row according to the order of voice frame arrangement to generate the subsequent received frequency domain feature matrix.

[0010] Preferably, the step of obtaining the seamlessly reconstructed frequency domain feature matrix is ​​as follows: The continuous extrapolated frequency domain feature matrix is ​​invoked, and the starting boundary of the preset overlapping window is determined according to the starting timestamp of the subsequent actual received frequency domain feature matrix. The ending boundary of the preset overlapping window is determined according to the preset overlapping duration. The frequency domain feature values ​​of the continuous extrapolated frequency domain feature matrix and the subsequent actual received frequency domain feature matrix within the preset overlapping window are extracted respectively. The extraction results are aligned item by item according to the timestamp, frequency point index and feature dimension to obtain the handover feature values. According to the relative position of each time step within the preset overlapping window from the starting boundary and the ending boundary, the positive weight coefficient is configured incrementally for each frequency point, and the reverse weight coefficient is configured incrementally for each frequency point. The sum of the positive weight coefficient and the reverse weight coefficient at any time step is verified to be equal to one, and a smoothing blanking weight table is established. Based on the handover feature values ​​and the smoothing and blanking weight table, according to the time step index, frequency point index, and feature dimension, the handover feature values ​​of the subsequent received frequency domain feature matrix are multiplied by the corresponding positive weight coefficients one by one. The handover feature values ​​of the continuous extrapolated frequency domain feature matrix are multiplied by the corresponding negative weight coefficients one by one. The two sets of product values ​​obtained under the same time step, the same frequency point position, and the same feature dimension are added one by one. The addition result replaces the original handover feature values ​​in the preset overlapping window. The frequency domain feature values ​​of the continuous extrapolated frequency domain feature matrix before the preset overlapping window are retained. The frequency domain feature values ​​of the subsequent received frequency domain feature matrix after the preset overlapping window are continued. The matrix merging is completed according to the unified timestamp order and frequency point arrangement order to generate a seamless reconstructed frequency domain feature matrix.

[0011] Preferably, the step of obtaining the identified text characters is as follows: The seamlessly reconstructed frequency domain feature matrix is ​​input into the preset acoustic model frame by frame in time frame order. The mapping probability of each time frame in all acoustic phonation states is read. The mapping probabilities are arranged according to the acoustic phonation state index to obtain the observation state probability distribution value of each time frame. The built-in reference value of the acoustic phonation state probability is called item by item to correct the candidate probability of the corresponding acoustic phonation state in the current time frame. According to the preset allowed flow relationship between acoustic phonation states, each flow path is expanded frame by frame from the start time frame to the end time frame. The candidate probability of each flow path is accumulated. Flow paths that cannot reach subsequent acoustic phonation states are deleted. The flow path with the largest accumulated probability in each time frame is retained. The acoustic phonation state sequence corresponding to the maximum accumulated probability value is extracted to obtain the recognized acoustic evolution trajectory. Based on the identified acoustic evolution trajectory, and according to the temporal arrangement of acoustic pronunciation states, the phoneme characters corresponding to each acoustic pronunciation state in the preset pronunciation dictionary are read. Continuously repeated acoustic pronunciation states are merged, and blank acoustic pronunciation states used to mark pause positions are deleted. The remaining acoustic pronunciation states are sequentially converted into candidate phoneme character sequences. Candidate entries that can cover the candidate phoneme character sequences are retrieved from the preset pronunciation dictionary. Based on the connection probability of adjacent candidate entries recorded by the language model, the candidate entry combinations are expanded one by one at each time position. The connection probability of each candidate entry combination is accumulated. Candidate entry combinations that cannot form complete entry boundaries are deleted. The candidate entry combination with the highest accumulated connection probability is selected, and the characters are connected according to the character arrangement order of the candidate entry combination to generate the recognized text characters.

[0012] Preferably, the step of obtaining the trigger command is as follows: Extract continuous character segments according to the character arrangement order, unify the character encoding, character width, and punctuation form of the recognized text characters and the built-in characters of the speech quality inspection dictionary, read the quality inspection entries corresponding to the built-in characters of the speech quality inspection dictionary one by one, slide the quality inspection entries sequentially according to the character starting position, count the number of characters that are completely consistent at each sliding position, record the number of replacements for characters with inconsistent character arrangement positions, the number of deletions for missing recognized text characters, and the number of insertions for added recognized text characters, determine the matching degree based on the proportion of completely consistent characters to the total number of characters in the quality inspection entries, combined with the number of replacements, deletions, and insertions, and compare the matching degree corresponding to each quality inspection entry item by item with the alarm interception threshold. When any matching degree exceeds the alarm interception threshold, a trigger command is generated.

[0013] Compared with the prior art, the advantages and positive effects of the present invention are as follows: In this invention, missing time periods are located by breaking down the signaling sequence number of the real-time voice data stream. Zero vectors are then filled into the corresponding positions of the preceding received frequency domain feature matrix before the break. This converts discontinuous voice segments caused by transmission interruptions into temporally aligned damaged acoustic frequency domain feature matrices, preventing time index errors in subsequent processing due to voice frame gaps. Furthermore, acoustic dynamic variables are constructed using frequency domain amplitude parameters, and state evolution prediction is performed along the time direction for the missing time periods. This allows the missing interval to obtain an inferred feature sequence with preceding and following acoustic correlations, rather than simply discarding or replacing it with silence, thereby improving the continuity of the broken voice before recognition. Finally, the subsequent received frequency domain feature matrix and continuous extrapolated frequency domain... The feature matrix performs feature extraction within a preset overlapping window and uses positive and negative weighting coefficients to achieve smooth summation and merging. This reduces abrupt differences between predicted features and recovered features, ensuring a smooth connection between the seamlessly reconstructed frequency domain feature matrix in terms of frequency points, time frames, and feature dimensions. Based on the seamlessly reconstructed frequency domain feature matrix, the probability distribution of observation states is calculated, acoustic evolution trajectory is extracted, and text characters are generated. This improves the stability of text restoration in mixed scenarios of lost speech, fragmented speech, and recovered speech. Furthermore, when the matching degree between the recognized text characters and the built-in characters in the speech quality inspection lexicon exceeds a threshold, a trigger command is generated, enhancing the timeliness of real-time speech quality inspection in intercepting risky statements. Attached Figure Description

[0014] Figure 1 This is a graph for determining the fault span and continuity threshold of the sequence number. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0016] Please see Figure 1 This invention provides a technical solution: a real-time voice quality inspection and risk warning system combining machine learning, comprising: The voice packet loss masking module is used to extract missing time periods based on the signaling sequence number of the real-time voice data stream and extract the preceding received frequency domain feature matrix before the break. For the missing time period, zero vectors are filled at the corresponding break positions of the preceding received frequency domain feature matrix to generate a damaged acoustic frequency domain feature matrix. The frequency domain feature extrapolation module is used to construct acoustic dynamic variables based on the frequency domain amplitude parameters corresponding to the damaged acoustic frequency domain feature matrix, perform state evolution prediction on the acoustic dynamic variables to obtain the extrapolated feature sequence, and concatenate the extrapolated feature sequence with the pre-received frequency domain feature matrix in time sequence to generate a continuous extrapolated frequency domain feature matrix. The cross-blinding reconstruction module is used to extract and recover the subsequent received frequency domain feature matrix and extract the intersection feature values ​​of it and the continuous extrapolated frequency domain feature matrix within a preset overlap window; establish a smooth blanking weight table containing positive weight coefficients and negative weight coefficients, and perform summation and merging operations on the two sets of intersection feature values ​​in combination with the positive weight coefficients and negative weight coefficients to generate a seamless reconstruction frequency domain feature matrix. The risk speech decoding module is used to input the seamlessly reconstructed frequency domain feature matrix into the acoustic model for mapping, calculate the observed state probability distribution value, extract the acoustic evolution trajectory through state space search, and decode and generate the recognized text characters in combination with the language model; compare the recognized text characters with the built-in characters in the speech quality inspection lexicon, and generate a trigger command when the matching degree exceeds the threshold.

[0017] The steps for obtaining the damaged acoustic frequency domain feature matrix are as follows: Read the transmission protocol signaling sequence numbers arranged in the arrival order within the real-time voice data stream. Select the transmission protocol signaling sequence number of the current signaling and the transmission protocol signaling sequence number of the preceding signaling that arrives immediately before the current signaling. Calculate the sequence number difference between adjacent transmission protocol signaling sequence numbers. Subtract the normal continuous increment interval from the sequence number difference to obtain the sequence number gap span of the corresponding signaling position. Compare the sequence number gap span with a continuity threshold item by item. Filter out signaling positions where the sequence number gap span exceeds the continuity threshold. Read the arrival time of the signaling before and after the filtered signaling position. Determine the missing voice data boundary according to the end time of the preceding signaling to the start time of the current signaling to obtain the missing time period. Based on the missing time period, determine the starting time boundary where the real-time voice data stream breaks. Backtrack to before the starting time boundary according to the time frame arrangement order of the real-time voice data stream, extract continuous voice frames that have been received and whose timestamps are earlier than the starting time boundary, and remove voice frames with duplicate timestamps, inconsistent time frame lengths, and inconsistent number of frequency points. Arrange each voice frame from earliest to latest according to the timestamp of the retained voice frames, read the frequency domain feature values ​​of each voice frame on all frequency points, arrange the frequency domain feature values ​​of the same voice frame into a row vector, and combine each row vector row by row according to the time frame arrangement order to generate the pre-received frequency domain feature matrix. Read the number of frequency point dimensions, time frame interval, and end timestamp of the pre-received frequency domain feature matrix. Determine the number of time frames to be filled according to the start time boundary, end time boundary, and time frame interval of the missing time period. For each time frame to be filled, create a zero vector with a length equal to the number of frequency point dimensions and all dimensions having zero values. Arrange all zero vectors in chronological order to the corresponding break position of the pre-received frequency domain feature matrix, so that the timestamps of the zero vectors sequentially cover the missing time period. Verify that the timestamp interval of adjacent time frames after expansion is equal to the time frame interval. Retain the original frequency domain feature values ​​in the pre-received frequency domain feature matrix to generate the damaged acoustic frequency domain feature matrix.

[0018] Specifically, the process reads the transmission protocol signaling sequence numbers arranged in arrival order from the real-time voice data stream. Before processing begins, a continuity threshold is first set. This threshold is based on a large number of historical voice data stream signaling samples collected under good network conditions, such as 100,000 signaling packets collected within one hour. The sequence number differences between all adjacent signaling packets are calculated to form a difference sample set. After removing large outliers that are obviously caused by network retransmissions, the mean and standard deviation of this sample set are calculated. The continuity threshold is then set to the mean of the difference samples plus three times the standard deviation, with an additional fixed compensation value. For example, if the calculated mean is 0.1, the standard deviation is 0.5, and the fixed compensation value is set to 2 (to tolerate a small number of out-of-order packets), then the threshold is 0.1 + 3 * 0.5 + 2 = 3.6, rounded to 4, means that a sequence number gap exceeding 4 is considered a valid missing voice data. Subsequently, the transmission protocol signaling sequence number of the current signaling and the transmission protocol signaling sequence number of the preceding signaling that arrives immediately before the current signaling are selected one by one, and the sequence number difference between the two is calculated. This difference is then subtracted from the continuously increasing interval of 1 specified under normal circumstances, such as Real-time Transport Protocol (RTP), to obtain the sequence number gap span of the corresponding signaling position. The calculated sequence number gap span is compared one by one with the previously set continuity threshold of 4. When the sequence number gap span of a certain position is greater than 4, the signaling position is filtered and recorded, and the receiving timestamp of the preceding signaling at that position is read as the start time of the missing period, and the receiving timestamp of the current signaling is used as the end time of the missing period. This is used to define the boundary of missing voice data and obtain the missing time period.

[0019] Based on the missing time period, after determining the starting time boundary of the real-time voice data stream interruption, the system searches backward from the starting time boundary according to the time frame arrangement to extract continuous voice frames that were completely received before the starting time boundary. For example, 200 continuous frames are extracted. For these 200 extracted voice frames, data cleaning and normalization are performed. First, a baseline voice frame is set, usually the first valid voice frame within the backtracking range, and its timestamp, frame length (e.g., 20 milliseconds), and number of frequency points (e.g., 256 frequency points) are recorded. Then, the subsequent 199 voice frames are traversed, and consistency checks are performed. Any voice frame with a timestamp that overlaps with the previous frame is identified as redundant. The process involves frame rejection, identifying and removing speech frames with a length not equal to 20 milliseconds or a number of frequency points not equal to 256 as abnormal frames. After removing all non-compliant speech frames, the remaining retained speech frames are rearranged in order of their respective timestamps from earliest to latest to ensure the continuity and monotonicity of the data in time. Then, the frequency domain feature values ​​of each retained speech frame at all 256 frequency points are read sequentially. These feature values ​​are usually complex numbers obtained after Short Time Fourier Transform (STFT). The 256 frequency domain feature values ​​within the same speech frame are arranged into a row vector. Finally, the row vectors corresponding to all speech frames are stacked and combined row by row in chronological order of timestamps to generate the pre-received frequency domain feature matrix.

[0020] First, the pre-received frequency domain feature matrix is ​​read. Its dimensionality information is then obtained from the matrix's metadata, including the number of frequency points (e.g., 256 frequency points), the fixed time interval between time frames (e.g., 20 milliseconds), and the end timestamp corresponding to the last time frame in the matrix. Simultaneously, based on the start and end time boundaries of the missing time period determined in the previous step, the total duration of the missing speech data is calculated. For example, if the start time of the missing time period is 10.50 seconds and the end time is 10.90 seconds, the total missing duration is 400 milliseconds. This is calculated by dividing the total missing duration by the time frame interval, i.e., 400 milliseconds / (400 milliseconds / 10.90 milliseconds). Calculating the number of time frames to be filled in 20 milliseconds, we find that there are 20. Then, for each of these 20 time frames, we create a zero vector with a length equal to the number of frequency point dimensions, i.e., a row vector containing 256 elements, all of which are zero. Then, we insert these 20 zero vectors into the end of the pre-received frequency domain feature matrix in chronological order, corresponding to the position where the speech stream breaks. We assign continuous timestamps to these newly inserted zero vectors, so that their timestamps start after the timestamp at the end of the pre-received frequency domain feature matrix and increase sequentially at 20-millisecond intervals until they completely cover the entire missing time period. After the operation is completed, we verify that the timestamp interval of any adjacent time frames in the expanded matrix is ​​equal to the preset time frame interval of 20 milliseconds, and retain all the original frequency domain feature values ​​in the pre-received frequency domain feature matrix to generate the damaged acoustic frequency domain feature matrix.

[0021] The steps for obtaining acoustic dynamics variables are as follows: Based on the damaged acoustic frequency domain feature matrix, locate the continuous time frames of the corresponding pre-received frequency domain feature matrix before filling the zero vector. Read the complex feature values ​​at each frequency point position in each continuous time frame, extract the real and imaginary parts of the complex feature values ​​respectively, sum and take the square root of the square values ​​of the real and imaginary parts, arrange the square root results of each frequency point position according to the time frame order to obtain the frequency domain amplitude parameter, calculate the first-order change value of the frequency domain amplitude parameter of the same frequency point position, and then calculate the second-order change value of the adjacent first-order change value. Bind the frequency domain amplitude parameter, the first-order change value, and the second-order change value according to the corresponding time frame and frequency point position to generate acoustic dynamic variables.

[0022] Specifically, based on the damaged acoustic frequency domain feature matrix, the continuous time frames corresponding to the preceding real-received frequency domain feature matrix before the zero-filling vector are first located. From these continuous time frames, the frequency domain feature values ​​at each frequency point are read sequentially. These values ​​are in complex form, containing a real part and an imaginary part. For each complex feature value at each frequency point, the real and imaginary parts are extracted. Then, according to the calculation method of complex modulus, the squares of the real and imaginary parts of the complex number are summed. The square root of the sum is then performed to obtain the frequency domain amplitude of that frequency point in that time frame. The square root results of all frequencies in all time frames are then combined with the original time frame order and frequency... The points are arranged in order to obtain the frequency domain amplitude parameter matrix. Next, in order to capture the changing trend of spectral energy, it is necessary to calculate its dynamic characteristics, that is, to calculate the first-order and second-order differences. Specifically, for the same frequency point position, the frequency domain amplitude parameter of the previous time frame is subtracted from the frequency domain amplitude parameter of the later time frame to obtain the first-order change value of that time frame. Then, the same difference calculation is performed on two adjacent first-order change values ​​to obtain the second-order change value. Finally, the original frequency domain amplitude parameters, the calculated first-order change values, and the second-order change values ​​are bound item by item according to their corresponding time frame index and frequency point position to form a multi-dimensional data structure containing amplitude, velocity, and acceleration information, generating acoustic dynamic variables.

[0023] The steps for obtaining the continuous extrapolation frequency domain feature matrix are as follows: Based on acoustic dynamics variables, continuous historical states are extracted for each frequency point position in time frame order. The state backtracking range is determined based on the number of available time frames for continuous historical states. The frequency domain amplitude parameters, first-level change values, and second-level change values ​​corresponding to each time frame within the state backtracking range are arranged sequentially. The state action coefficients corresponding to each historical time frame are adjusted to minimize the cumulative square deviation of the historical time frame estimated values ​​relative to the corresponding actual received values. The predicted feature values ​​at the starting position of the missing time period are calculated according to the determined state action coefficients. The predicted feature values ​​obtained each time are added to the continuous historical states. The subsequent predicted feature values ​​are repeatedly calculated along the time step direction until the predicted timestamp reaches the termination boundary of the missing time period. All predicted feature values ​​are arranged according to the predicted timestamp to obtain the inferred feature sequence. Retrieve the feature values ​​of the corresponding preceding received frequency domain feature matrix from the damaged acoustic frequency domain feature matrix, retain the original time frame order, frequency point arrangement order, and number of feature dimensions of the preceding received frequency domain feature matrix, verify whether the frequency point arrangement order and number of feature dimensions of the extrapolated feature sequence correspond to the preceding received frequency domain feature matrix, rearrange the extrapolated feature sequence according to the frequency point index of the preceding received frequency domain feature matrix for inconsistent positions, delete the padding zero vectors covering the missing time period in the damaged acoustic frequency domain feature matrix, attach the rearranged extrapolated feature sequence to the end time frame of the preceding received frequency domain feature matrix, configure continuous timestamps according to a uniform time frame interval, and generate a continuous extrapolated frequency domain feature matrix.

[0024] Specifically, based on acoustic dynamic variables, an autoregressive model is used to extrapolate the characteristics of the missing segments. Specifically, for each frequency point, the prediction process is executed independently. First, the continuous historical states before the missing frequency point are extracted in time frame order, i.e., acoustic dynamic variables (including frequency domain amplitude parameters, first-order change values, and second-order change values). The state backtracking range is determined based on the number of available historical time frames. For example, if there are 200 available historical time frames, the backtracking range p is set to 10. The acoustic dynamic variables corresponding to the 10 most recent time frames are used as the model input. An optimization problem is solved to adjust the state action coefficients corresponding to these 10 historical time frames. The goal of this optimization problem is to ensure that the extrapolated values ​​generated by using these coefficients to predict known historical data are consistent with the corresponding actual received values. The sum of mean squared errors between them is minimized. This process is equivalent to solving a set of linear equations to obtain the optimal state action coefficient. After obtaining the determined state action coefficient, the first predicted feature value at the start position of the missing time period is calculated using the real state of the last 10 time frames before the missing time period. Then, this newly obtained predicted feature value is added to the end of the continuous historical state as a new historical state, and the oldest historical state is removed. The backtracking range is kept at 10. Based on this updated set of historical states, the calculation process is repeated to predict the feature value of the next time step. This iterative process is repeated continuously along the time step direction until the predicted timestamp reaches and covers the termination boundary of the missing time period. Finally, all the predicted feature values ​​generated in the iteration process are arranged in the order of their predicted timestamps to obtain the inferred feature sequence.

[0025] Retrieve the feature values ​​from the damaged acoustic frequency domain feature matrix that correspond to the original pre-received frequency domain feature matrix. Retain the original time frame order, frequency point order, and number of feature dimensions of this data as the basis for subsequent splicing. Simultaneously, verify the internal structure of the inferred feature sequence generated in the previous step, checking whether its frequency point order and number of feature dimensions are completely consistent with the basis of the pre-received frequency domain feature matrix. If inconsistencies in the frequency point index arrangement are found in the inferred feature sequence, such as due to disordered order caused by parallel computation, then reorder the row vectors of the inferred feature sequence using the frequency point indices of the pre-received frequency domain feature matrix as a template. To ensure one-to-one correspondence of frequency points, after alignment, all padding zero vectors used to cover missing time periods are located and deleted from the damaged acoustic frequency domain feature matrix. Then, the rearranged and aligned extrapolated feature sequence is spliced ​​as a whole to the end time frame of the preceding received frequency domain feature matrix. A unified and continuous timestamp is configured for the entire spliced ​​matrix to ensure that the timestamp interval from the end of the original data to the beginning of the extrapolated sequence, as well as the timestamp interval between adjacent frames within the extrapolated sequence, are equal to the standard time frame interval (e.g., 20 milliseconds). Finally, a matrix that is continuous in time and has consistent feature dimensions is formed, generating a continuous extrapolated frequency domain feature matrix.

[0026] The steps for obtaining the subsequent received frequency domain feature matrix are as follows: Based on the subsequent real-time voice data streams that arrive after the missing time period, the end time boundary of the missing time period is used as the starting point for recovery reception. Voice frames that arrive continuously after the starting point for recovery reception are read. The timestamp, frame length, number of frequency points, and feature dimensions of each voice frame are checked. Voice frames with duplicate timestamps, missing frame lengths, and inconsistent number of frequency points are removed. The retained voice frames are arranged in order of earliest to latest timestamp. The frequency domain feature values ​​of each voice frame at each frequency point are extracted. The frequency domain feature values ​​of the same voice frame are arranged into row vectors. All row vectors are combined row by row according to the order of voice frame arrangement to generate the subsequent received frequency domain feature matrix.

[0027] Specifically, based on the subsequent real-time voice data streams that arrive after the missing voice data period, the end time boundary of the missing period is set as the logical starting point for recovery reception. From this starting point, subsequent arriving voice frames are continuously read, and these voice frames undergo rigorous data quality verification and filtering. First, the metadata of each received voice frame is checked, including its timestamp, frame length, number of frequency points, and feature dimensions. A verification benchmark is set, which is usually derived from the first successfully recovered and fully formatted voice frame in the data stream. For example, the benchmark frame has a frame length of 20 milliseconds and 256 frequency points. Then, all subsequently received voice frames are traversed, and any voice frames whose timestamps overlap with the previously confirmed valid frame are marked. To eliminate redundancy, speech frames with a frame length of less than 20 milliseconds or a frequency point count of less than 256 are marked as having format errors and are removed. Incomplete frames with missing frame length or feature dimension information are also removed. Through this series of filtering operations, only valid speech frames with continuous timestamps and complete formats are retained. All retained speech frames are then sorted in ascending order of their timestamps to reconstruct the correct temporal sequence. Next, frequency domain feature values ​​at all 256 frequency point positions are extracted from each sorted speech frame, and the feature values ​​of the same speech frame are arranged into a row vector. Finally, the row vectors corresponding to all speech frames are combined row by row according to their temporal order to generate the subsequent received frequency domain feature matrix.

[0028] The steps for seamlessly reconstructing the frequency domain feature matrix are as follows: The continuous extrapolation frequency domain feature matrix is ​​invoked. The starting boundary of the preset overlapping window is determined according to the starting timestamp of the subsequent actual received frequency domain feature matrix. The ending boundary of the preset overlapping window is determined according to the preset overlapping duration. The frequency domain feature values ​​of the continuous extrapolation frequency domain feature matrix and the subsequent actual received frequency domain feature matrix within the preset overlapping window are extracted respectively. The extraction results are aligned item by item according to the timestamp, frequency index and feature dimension to obtain the handover feature values. According to the relative position of each time step within the preset overlapping window from the starting boundary and the ending boundary, the positive weight coefficient is configured incrementally for each frequency point and the reverse weight coefficient is configured incrementally for each frequency point. The sum of the positive weight coefficient and the reverse weight coefficient at any time step is verified to be equal to one, and a smooth blanking weight table is established. Based on the handover feature values ​​and the smoothing and blanking weight table, according to the time step index, frequency point index, and feature dimension, the handover feature values ​​of the subsequent received frequency domain feature matrix are multiplied by the corresponding positive weight coefficients one by one. The handover feature values ​​of the continuous extrapolated frequency domain feature matrix are multiplied by the corresponding negative weight coefficients one by one. The two sets of product values ​​obtained under the same time step, the same frequency point position, and the same feature dimension are added one by one. The sum replaces the original handover feature values ​​in the preset overlapping window. The frequency domain feature values ​​of the continuous extrapolated frequency domain feature matrix before the preset overlapping window are retained. The frequency domain feature values ​​of the subsequent received frequency domain feature matrix after the preset overlapping window are continued. The matrix merging is completed according to the unified timestamp order and frequency point arrangement order to generate a seamlessly reconstructed frequency domain feature matrix.

[0029] Specifically, the process involves calling a continuously extrapolated frequency domain feature matrix and, based on the start timestamp of the subsequent received frequency domain feature matrix, precisely determining the starting boundary of a preset overlapping window for smooth transition. Then, based on a preset overlap duration, such as 100 milliseconds, the ending boundary of this window is determined. For example, if the time frame interval is 20 milliseconds, the overlapping window contains 100 / 20 = Five time frames are used. Then, frequency domain feature values ​​within the overlapping window containing these five time frames are extracted from both the continuously extrapolated frequency domain feature matrix and the subsequent received frequency domain feature matrix. Based on the timestamp, frequency index, and feature dimension of each data point, these two sets of extracted feature values ​​are aligned item by item to form one-to-one corresponding handover feature value pairs. Next, a smoothing and blanking weight table is established. This table assigns weight coefficients to each time step (i.e., each time frame) within the overlapping window. The weight coefficients are set using a linear gradient. For subsequent received data, the weight coefficients are the forward weight coefficients, starting from 0 at the beginning of the window and linearly increasing to 1 at the end. For example, in a five-frame window, they are 0, 0.25, 0.5, 0.75, and 1, respectively. For continuously extrapolated data, the weight coefficients are the reverse weight coefficients, starting from 1 at the beginning of the window and linearly decreasing to 0 at the end. They are 1, 0.75, 0.5, and 0.25, respectively. 0, at any time step, the sum of the positive weight coefficient and the negative weight coefficient is always equal to 1, and a smoothing and hidden surface removal weight table is established.

[0030] Based on the handover feature values ​​and the established smoothing and blanking weight table, a weighted fusion operation is performed on the feature data within the overlapping window. Specifically, according to the time step index, frequency point index, and feature dimension, each data point within the overlapping window is traversed. The handover feature value at the corresponding position in the subsequent received frequency domain feature matrix is ​​multiplied item by item by its corresponding positive weight coefficient in the smoothing and blanking weight table. At the same time, the handover feature value at the corresponding position in the continuously extrapolated frequency domain feature matrix is ​​multiplied item by item by its corresponding negative weight coefficient. Taking the third time frame in the overlapping window as an example, its positive weight coefficient is 0.5, and its negative weight coefficient is 0.5. Then, the fused feature value of this frame is the subsequent received feature value * 0.5 + the continuously extrapolated feature value * 0.5. 0.5, the two sets of product values ​​obtained by the two multiplication operations at the same time step, the same frequency point position, and the same feature dimension are added one by one to obtain the new feature value after fusion. The result of this addition is used to replace the two sets of intersection feature values ​​in the original overlapping window to form a smooth transition feature segment. After the fusion within the window is completed, all frequency domain feature values ​​of the continuous extrapolated frequency domain feature matrix before the preset overlapping window are retained. Then, the smooth transition feature segment just generated is spliced ​​on, and then all frequency domain feature values ​​of the subsequent received frequency domain feature matrix after the preset overlapping window are added. The merging of these three matrices is completed according to the unified timestamp order and frequency point arrangement order to generate a seamless reconstructed frequency domain feature matrix.

[0031] The steps for obtaining recognized text characters are as follows: The seamlessly reconstructed frequency domain feature matrix is ​​input into the preset acoustic model frame by frame in time frame order. The mapping probability of each time frame in all acoustic phonation states is read. The mapping probabilities are arranged according to the acoustic phonation state index to obtain the observation state probability distribution value of each time frame. The built-in reference value of the acoustic phonation state probability is called item by item to correct the candidate probability of the corresponding acoustic phonation state in the current time frame. According to the preset allowed flow relationship between acoustic phonation states, each flow path is expanded frame by frame from the start time frame to the end time frame. The candidate probability of each flow path is accumulated. Flow paths that cannot reach subsequent acoustic phonation states are deleted. The flow path with the largest accumulated probability in each time frame is retained. The acoustic phonation state sequence corresponding to the maximum accumulated probability value is extracted to obtain the recognized acoustic evolution trajectory. Based on the acoustic evolution trajectory, and according to the temporal order of the acoustic pronunciation states, the phoneme characters corresponding to each acoustic pronunciation state in the preset pronunciation dictionary are read. Continuously repeated acoustic pronunciation states are merged, and blank acoustic pronunciation states used to mark pause positions are deleted. The remaining acoustic pronunciation states are sequentially converted into candidate phoneme character sequences. Candidate entries that can cover the candidate phoneme character sequences are retrieved from the preset pronunciation dictionary. Based on the connection probability of adjacent candidate entries recorded by the language model, the candidate entry combinations are expanded one by one at each time position. The connection probability of each candidate entry combination is accumulated, and candidate entry combinations that cannot form complete entry boundaries are deleted. The candidate entry combination with the highest accumulated connection probability is selected, and the characters are connected according to the character arrangement order of the candidate entry combination to generate the recognized text characters.

[0032] Specifically, the seamlessly reconstructed frequency domain feature matrix is ​​input frame by frame into a pre-trained acoustic model, based on a deep neural network structure, such as a Time Delay Neural Network (TDNN) or a Long Short-Term Memory (LSTM) network. For each input time-frame feature vector, the acoustic model calculates its posterior probability across all predefined acoustic articulation states (e.g., phoneme or triphone states). These probability values ​​are then arranged according to the index of the acoustic articulation state to obtain the observed state probability distribution for that time frame. Next, to incorporate prior knowledge for correction, a built-in reference table recording the frequency of each acoustic articulation state is called to adjust the candidate probabilities calculated for the current time frame. Finally, based on a... The predefined state transition matrix defines the allowed transition relationships (e.g., the state of phoneme / a / can transition to the state of / b / , but cannot directly jump to the state of / z / ). The Viterbi algorithm is used to perform a state space search. Starting from the starting time frame, all possible transition paths are expanded frame by frame towards the ending time frame. During the expansion process, the logarithmic probability of all candidate states traversed by each path is accumulated. At each state node in each time frame, only the path with the highest accumulated probability is retained, and other invalid paths that cannot reach subsequent acoustic articulation states or have low probabilities are deleted. When the search is completed and the ending time frame is reached, the acoustic articulation state sequence corresponding to the path is extracted by backtracking the transition path with the highest global accumulated probability, thus obtaining the recognized acoustic evolution trajectory.

[0033] Based on the identified acoustic evolution trajectory, i.e., the optimal acoustic articulation state sequence decoded from the acoustic model, the sequences are processed chronologically. First, a pre-defined pronunciation dictionary is read, which establishes a mapping between acoustic articulation states and standard phoneme characters (e.g., IPA phonetic symbols or Arpabet symbols). Next, the acoustic articulation state sequence undergoes post-processing. Consecutive repeated acoustic articulation states are merged into single states; for example, the sequence "aaabbc" is merged into "abc". Simultaneously, all blank acoustic articulation states used to mark silence or pauses are identified and deleted. The remaining acoustic articulation state sequence after merging and deletion is then converted one by one into a candidate phoneme character sequence using the pronunciation dictionary. Finally, all characters that can cover this candidate phoneme character are retrieved from the pronunciation dictionary. The process of combining candidate word combinations in a sequence involves a pre-trained language model that records the connection probabilities between adjacent words in a large text corpus. This model is typically in the form of an N-gram model (e.g., 3-gram). Starting from the beginning of the sequence, possible candidate word combinations are expanded sequentially at each time position. Based on the connection probabilities provided by the language model, the total probability of each word combination path is calculated cumulatively. During the expansion process, combinations that cannot form complete word boundaries are removed. For example, a phoneme sequence may not correspond to the end of any word. Among all the constructed candidate word combinations, the combination with the highest cumulative connection probability is selected as the optimal solution. The characters are then connected according to the character arrangement order within the optimal candidate word combination to generate the recognized text characters.

[0034] The steps to obtain the trigger command are as follows: Extract continuous character segments according to the character arrangement order, uniformly identify the character encoding, character width, and punctuation form of text characters and characters built into the speech quality inspection dictionary, read the quality inspection entries corresponding to the characters built into the speech quality inspection dictionary one by one, slide the quality inspection entries sequentially according to the character starting position, count the number of characters that are completely consistent at each sliding position, record the number of replacements for characters with inconsistent character arrangement positions, the number of deletions for missing text characters, and the number of insertions for added text characters. Based on the proportion of completely consistent characters to the total number of characters in the quality inspection entries, combined with the number of replacements, deletions, and insertions, determine the degree of matching. Compare the degree of matching for each quality inspection entry with the alarm interception threshold item by item. When any degree of matching exceeds the alarm interception threshold, a trigger command is generated.

[0035] Specifically, following the character arrangement, continuous character segments are extracted from the generated recognized text characters. These characters, along with those built into the speech quality inspection lexicon, are standardized by converting their character encoding to UTF-8, converting full-width characters and numbers to half-width, and standardizing punctuation styles to eliminate matching errors caused by format differences. Then, preset quality inspection terms from the speech quality inspection lexicon, such as "process a refund," are read one by one. This quality inspection term is used as a sliding window, sliding character by character on the recognized text characters from the starting position. At each sliding position, the edit distance between the quality inspection term and the corresponding text segment is calculated, i.e., the number of completely identical characters and the minimum number of operations required for a match, including the number of character replacements, deletions, and insertions. The degree of matching is determined based on these statistics, calculated as (total number of characters in the term - (number of replacements + number of deletions + number of insertions)) / The total number of characters in the term is used to calculate the highest matching degree of each quality inspection term at each position, and then compares it item by item with a preset alarm interception threshold. This threshold is set differently according to the importance and confusion level of the term. For example, the threshold for the keyword "refund" may be set to 0.8, while for "promotional activities" it may be set to 0.7. When the matching degree calculation result of any quality inspection term exceeds its corresponding alarm interception threshold, a trigger command is generated.

Claims

1. A real-time voice quality inspection and risk warning system combining machine learning, characterized in that: The system includes: The voice packet loss masking module is used to extract missing time periods based on the signaling sequence number faults of the real-time voice data stream and extract the preceding received frequency domain feature matrix before the fault. For the missing time periods, zero vectors are filled at the corresponding fault positions of the preceding received frequency domain feature matrix to generate a damaged acoustic frequency domain feature matrix. The frequency domain feature extrapolation module is used to construct acoustic dynamic variables based on the frequency domain amplitude parameters corresponding to the damaged acoustic frequency domain feature matrix, perform state evolution prediction on the acoustic dynamic variables to obtain extrapolated feature sequences, and concatenate the extrapolated feature sequences with the pre-received frequency domain feature matrix in time sequence to generate a continuous extrapolated frequency domain feature matrix. The cross-blinding reconstruction module is used to extract and recover the subsequent received frequency domain feature matrix, and extract the intersection feature values ​​of the matrix and the continuous extrapolated frequency domain feature matrix within a preset overlap window; establish a smooth blanking weight table containing positive weight coefficients and negative weight coefficients, and perform a summation and merging operation on the two sets of intersection feature values ​​in combination with the positive weight coefficients and negative weight coefficients to generate a seamless reconstruction frequency domain feature matrix. The risk speech decoding module is used to input the seamlessly reconstructed frequency domain feature matrix into the acoustic model for mapping, calculate the observed state probability distribution value, extract the acoustic evolution trajectory through state space search, and decode and generate recognized text characters in combination with the language model; compare the recognized text characters with the built-in characters of the speech quality inspection lexicon, and generate a trigger command when the matching degree exceeds the threshold.

2. The real-time voice quality inspection and risk warning system combining machine learning according to claim 1, characterized in that, The steps for obtaining the damaged acoustic frequency domain feature matrix are as follows: Read the transmission protocol signaling sequence numbers arranged in the arrival order within the real-time voice data stream. Select the transmission protocol signaling sequence number of the current signaling and the transmission protocol signaling sequence number of the preceding signaling that arrives immediately before the current signaling. Calculate the sequence number difference between adjacent transmission protocol signaling sequence numbers. Subtract the normal continuous increment interval from the sequence number difference to obtain the sequence number gap span of the corresponding signaling position. Compare the sequence number gap span item by item with a continuity threshold. Filter out signaling positions where the sequence number gap span exceeds the continuity threshold. Read the signaling arrival time before and after the filtered signaling position. Determine the missing voice data boundary according to the end time of the preceding signaling to the start time of the current signaling to obtain the missing time period. Based on the missing time period, the starting time boundary of the real-time voice data stream interruption is determined. The real-time voice data stream is backtracked to before the starting time boundary according to the time frame arrangement order. Continuous voice frames that have been received and whose timestamps are earlier than the starting time boundary are extracted. Voice frames with duplicate timestamps, inconsistent time frame lengths, or inconsistent frequency points are removed. Each voice frame is arranged from earliest to latest according to the timestamp of the retained voice frames. The frequency domain feature values ​​of each voice frame on all frequency points are read. The frequency domain feature values ​​of the same voice frame are arranged into row vectors. Each row vector is combined row by row according to the time frame arrangement order to generate the pre-received frequency domain feature matrix. The number of frequency point dimensions, time frame interval, and end timestamp of the pre-received frequency domain feature matrix are read. The number of time frames to be filled is determined according to the start time boundary, end time boundary, and time frame interval of the missing time period. For each time frame to be filled, a zero vector with a length equal to the number of frequency point dimensions and all dimensions having zero values ​​is created. All zero vectors are arranged in chronological order to the corresponding break position of the pre-received frequency domain feature matrix, so that the timestamps of the zero vectors sequentially cover the missing time period. The timestamp interval of adjacent time frames after expansion is verified to be equal to the time frame interval. The original frequency domain feature values ​​in the pre-received frequency domain feature matrix are retained to generate the damaged acoustic frequency domain feature matrix.

3. The real-time voice quality inspection and risk warning system combining machine learning according to claim 1, characterized in that, The steps for obtaining the acoustic dynamics variables are as follows: Based on the damaged acoustic frequency domain feature matrix, locate the continuous time frame corresponding to the preceding real-received frequency domain feature matrix before filling the zero vector. Read the complex feature values ​​at each frequency point position in each continuous time frame, extract the real and imaginary parts of the complex feature values ​​respectively, sum and take the square root of the squares of the real and imaginary parts, arrange the square root results of each frequency point position according to the time frame order to obtain the frequency domain amplitude parameter, calculate the first-order change value of the frequency domain amplitude parameter adjacent to the same frequency point position, and then calculate the second-order change value of the adjacent first-order change value. Bind the frequency domain amplitude parameter, the first-order change value, and the second-order change value item by item according to the corresponding time frame and frequency point position to generate acoustic dynamic variables.

4. The real-time voice quality inspection and risk warning system combining machine learning according to claim 1, characterized in that, The steps for obtaining the continuous extrapolation frequency domain feature matrix are as follows: Based on the acoustic dynamics variables, continuous historical states are extracted for each frequency point position in time frame order. The state backtracking range is determined based on the number of available time frames for continuous historical states. The frequency domain amplitude parameters, first-level change values, and second-level change values ​​corresponding to each time frame within the state backtracking range are arranged sequentially. The state action coefficients corresponding to each historical time frame are adjusted to minimize the cumulative squared deviation of the historical time frame estimated values ​​relative to the corresponding actual received values. The predicted feature values ​​at the starting position of the missing time period are calculated according to the determined state action coefficients. The predicted feature values ​​obtained each time are added to the continuous historical states. The subsequent predicted feature values ​​are repeatedly calculated along the time step direction until the predicted timestamp reaches the termination boundary of the missing time period. All predicted feature values ​​are arranged according to the predicted timestamps to obtain the inferred feature sequence. Retrieve the feature values ​​corresponding to the preceding actual received frequency domain feature matrix from the damaged acoustic frequency domain feature matrix, retain the original time frame order, frequency point arrangement order, and number of feature dimensions of the preceding actual received frequency domain feature matrix, verify whether the frequency point arrangement order and number of feature dimensions of the extrapolated feature sequence correspond to the preceding actual received frequency domain feature matrix, rearrange the extrapolated feature sequence according to the frequency point index of the preceding actual received frequency domain feature matrix for inconsistent positions, delete the padding zero vectors covering the missing time period in the damaged acoustic frequency domain feature matrix, attach the rearranged extrapolated feature sequence to the end time frame of the preceding actual received frequency domain feature matrix, configure continuous timestamps according to a uniform time frame interval, and generate a continuous extrapolated frequency domain feature matrix.

5. The real-time voice quality inspection and risk warning system combining machine learning according to claim 1, characterized in that, The steps for obtaining the subsequent received frequency domain feature matrix are as follows: Based on the subsequent real-time voice data stream that arrives after the missing time period, the end time boundary of the missing time period is used as the starting point for recovery reception. Voice frames that arrive continuously after the starting point for recovery reception are read. The timestamp, frame length, number of frequency points, and feature dimensions of each voice frame are checked. Voice frames with duplicate timestamps, missing frame lengths, and inconsistent number of frequency points are removed. The retained voice frames are arranged in order of earliest to latest timestamp. The frequency domain feature values ​​of each voice frame at each frequency point are extracted. The frequency domain feature values ​​of the same voice frame are arranged into row vectors. All row vectors are combined row by row according to the order of voice frame arrangement to generate the subsequent received frequency domain feature matrix.

6. The real-time voice quality inspection and risk warning system combining machine learning according to claim 1, characterized in that, The steps for obtaining the seamlessly reconstructed frequency domain feature matrix are as follows: The continuous extrapolated frequency domain feature matrix is ​​invoked, and the starting boundary of the preset overlapping window is determined according to the starting timestamp of the subsequent actual received frequency domain feature matrix. The ending boundary of the preset overlapping window is determined according to the preset overlapping duration. The frequency domain feature values ​​of the continuous extrapolated frequency domain feature matrix and the subsequent actual received frequency domain feature matrix within the preset overlapping window are extracted respectively. The extraction results are aligned item by item according to the timestamp, frequency point index and feature dimension to obtain the handover feature values. According to the relative position of each time step within the preset overlapping window from the starting boundary and the ending boundary, the positive weight coefficient is configured incrementally for each frequency point, and the reverse weight coefficient is configured incrementally for each frequency point. The sum of the positive weight coefficient and the reverse weight coefficient at any time step is verified to be equal to one, and a smoothing blanking weight table is established. Based on the handover feature values ​​and the smoothing and blanking weight table, according to the time step index, frequency point index, and feature dimension, the handover feature values ​​of the subsequent received frequency domain feature matrix are multiplied by the corresponding positive weight coefficients one by one. The handover feature values ​​of the continuous extrapolated frequency domain feature matrix are multiplied by the corresponding negative weight coefficients one by one. The two sets of product values ​​obtained under the same time step, the same frequency point position, and the same feature dimension are added one by one. The addition result replaces the original handover feature values ​​in the preset overlapping window. The frequency domain feature values ​​of the continuous extrapolated frequency domain feature matrix before the preset overlapping window are retained. The frequency domain feature values ​​of the subsequent received frequency domain feature matrix after the preset overlapping window are continued. The matrix merging is completed according to the unified timestamp order and frequency point arrangement order to generate a seamless reconstructed frequency domain feature matrix.

7. The real-time voice quality inspection and risk warning system combining machine learning according to claim 1, characterized in that, The steps for obtaining the identified text characters are as follows: The seamlessly reconstructed frequency domain feature matrix is ​​input into the preset acoustic model frame by frame in time frame order. The mapping probability of each time frame in all acoustic phonation states is read. The mapping probabilities are arranged according to the acoustic phonation state index to obtain the observation state probability distribution value of each time frame. The built-in reference value of the acoustic phonation state probability is called item by item to correct the candidate probability of the corresponding acoustic phonation state in the current time frame. According to the preset allowed flow relationship between acoustic phonation states, each flow path is expanded frame by frame from the start time frame to the end time frame. The candidate probability of each flow path is accumulated. Flow paths that cannot reach subsequent acoustic phonation states are deleted. The flow path with the largest accumulated probability in each time frame is retained. The acoustic phonation state sequence corresponding to the maximum accumulated probability value is extracted to obtain the recognized acoustic evolution trajectory. Based on the identified acoustic evolution trajectory, and according to the temporal arrangement of acoustic pronunciation states, the phoneme characters corresponding to each acoustic pronunciation state in the preset pronunciation dictionary are read. Continuously repeated acoustic pronunciation states are merged, and blank acoustic pronunciation states used to mark pause positions are deleted. The remaining acoustic pronunciation states are sequentially converted into candidate phoneme character sequences. Candidate entries that can cover the candidate phoneme character sequences are retrieved from the preset pronunciation dictionary. Based on the connection probability of adjacent candidate entries recorded by the language model, the candidate entry combinations are expanded one by one at each time position. The connection probability of each candidate entry combination is accumulated. Candidate entry combinations that cannot form complete entry boundaries are deleted. The candidate entry combination with the highest accumulated connection probability is selected, and the characters are connected according to the character arrangement order of the candidate entry combination to generate the recognized text characters.

8. The real-time voice quality inspection and risk warning system combining machine learning according to claim 1, characterized in that, The steps for obtaining the trigger command are as follows: Extract continuous character segments according to the character arrangement order, unify the character encoding, character width, and punctuation form of the recognized text characters and the built-in characters of the speech quality inspection dictionary, read the quality inspection entries corresponding to the built-in characters of the speech quality inspection dictionary one by one, slide the quality inspection entries sequentially according to the character starting position, count the number of characters that are completely consistent at each sliding position, record the number of replacements for characters with inconsistent character arrangement positions, the number of deletions for missing recognized text characters, and the number of insertions for added recognized text characters, determine the matching degree based on the proportion of completely consistent characters to the total number of characters in the quality inspection entries, combined with the number of replacements, deletions, and insertions, and compare the matching degree corresponding to each quality inspection entry item by item with the alarm interception threshold. When any matching degree exceeds the alarm interception threshold, a trigger command is generated.

Images