Functional network-based speech call real-time evaluation method and device, equipment and storage medium

Abstract

The application discloses a speech call real-time evaluation method and device based on a functional network, equipment and a storage medium, relates to the technical field of speech quality monitoring, and comprises the following steps: receiving network delay jitter standard deviation and network packet loss rate in a speech call process; performing normalization processing on the network delay jitter standard deviation and the network packet loss rate and calculating sliding window statistics to obtain time series enhancement features; inputting the time series enhancement features into a function library, performing function stacking processing and topological evolution processing, and obtaining intermediate state results; performing weighted integration processing and activation mapping processing on the intermediate state results to obtain MOS prediction values; by constructing a dynamically evolved functional network, the network delay jitter and the packet loss rate are mapped into MOS values in real time, millisecond-level speech quality evaluation is realized, the calculation complexity is reduced, and the adaptability to non-stationary network environments is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of voice quality monitoring, in particular to a voice call real-time evaluation method and device based on a functional network, equipment and a storage medium. BACKGROUND

[0002] With the rapid expansion of real-time voice communication, operators and users urgently need to know the voice quality in the process of communication in real time, so as to timely adjust the network or terminal strategy and guarantee the consistency and stability of the call experience. Millisecond-level quality perception has become a key requirement to improve service competitiveness.

[0003] The existing objective evaluation technology depends on the comparison of the original voice signal, has long calculation path and high resource occupation, and is difficult to meet the real-time requirement. The static network model has insufficient adaptability to the non-stationary changes of the time delay jitter and the packet loss rate, resulting in obvious drift of the quality evaluation result with the network fluctuation, so that the operation and maintenance personnel cannot intuitively locate the specific influence path of the network parameters on the voice quality.

[0004] The above content is only used to assist in understanding the technical solutions of the present application and does not represent the acknowledgement of the above content as prior art. SUMMARY

[0005] The main purpose of the present application is to provide a voice call real-time evaluation method and device based on a functional network, equipment and a storage medium, which aims to solve the technical problem of realizing real-time, stable and interpretable evaluation of VoIP call voice quality.

[0006] To achieve the above purpose, the present application provides a voice call real-time evaluation method based on a functional network, which comprises the following steps:

[0007] Receiving network time delay jitter standard deviation and network packet loss rate in the process of voice call;

[0008] Normalizing the network time delay jitter standard deviation and the network packet loss rate and calculating the sliding window statistics to obtain time series enhanced features;

[0009] Inputting the time series enhanced features into a function library, performing function stacking processing and topological evolution processing to obtain intermediate state results;

[0010] Performing weighted integration processing and activation mapping processing on the intermediate state results to obtain MOS prediction values.

[0011] In an embodiment, the step of normalizing the network time delay jitter standard deviation and the network packet loss rate and calculating the sliding window statistics to obtain the time series enhanced features comprises:

[0012] obtaining a first target value and a second target value of the network delay jitter standard deviation and calculating a first difference value of the first target value and the second target value;

[0013] subtracting the second target value from the network delay jitter standard deviation of the current frame and dividing by the first difference value to obtain a normalized jitter value;

[0014] obtaining a third target value and a fourth target value of the network packet loss rate and calculating a second difference value of the third target value and the fourth target value;

[0015] subtracting the fourth target value from the network packet loss rate of the current frame and dividing by the second difference value to obtain a normalized packet loss value;

[0016] traversing the normalized jitter value and the normalized packet loss value in a sliding window manner to splice to form a timing enhancement feature.

[0017] In an embodiment, the step of inputting the timing enhancement feature into a function library, performing function stacking processing and topology evolution processing to obtain an intermediate state result comprises:

[0018] inputting the timing enhancement feature into a current layer function unit to obtain a current layer output;

[0019] comparing the current layer output with a corresponding real MOS label to obtain a verification error;

[0020] when the verification error exceeds a preset threshold, selecting an unused function from the function library as a new function unit and inserting it after the current layer, inputting the current layer output as the input of the new function unit to obtain a new layer output;

[0021] when the verification error does not exceed the preset threshold, directly inputting the current layer output as the input of the next layer to obtain a straight-through layer output;

[0022] inputting the new layer output or the straight-through layer output as the input of the next layer function unit until the number of layers reaches a preset number of layers, and inputting the output of the last layer function unit as the intermediate state result.

[0023] In an embodiment, the step of selecting an unused function from the function library as a new function unit and inserting it after the current layer, inputting the current layer output as the input of the new function unit to obtain a new layer output comprises:

[0024] reading the function identifiers of all functions in the function library that are not called by the current layer to obtain a candidate function list;

[0025] randomly selecting a function from the candidate function list and initializing parameters to obtain a function unit to be inserted;

[0026] connecting an input port of the function unit to be inserted to an output port of a current layer, and connecting an output port of the function unit to be inserted to an input port of a next layer, completing the insertion;

[0027] inputting an output of the current layer into the function unit to be inserted to obtain an output of a new layer.

[0028] In an embodiment, the step of performing a weighted integration processing and an activation mapping processing on the intermediate state result to obtain a MOS prediction value comprises:

[0029] expanding the intermediate state result into a plurality of state vectors in a channel dimension;

[0030] allocating a learnable weight to each of the state vectors and performing an element-wise multiplication processing to obtain a weighted vector;

[0031] inputting the weighted vector into an activation function to obtain an activation result;

[0032] performing a linear mapping processing on the activation result to obtain a MOS prediction value.

[0033] In an embodiment, the method further comprises:

[0034] receiving a real-time voice call data stream, extracting a real-time network delay jitter standard deviation and a real-time network packet loss rate;

[0035] performing normalization and sliding window statistics on the real-time network delay jitter standard deviation and the real-time network packet loss rate to obtain a real-time timing enhancement feature;

[0036] inputting the real-time timing enhancement feature into a trained functional network to obtain a real-time intermediate state result;

[0037] inputting the real-time intermediate state result into an output layer to obtain a real-time MOS prediction value;

[0038] packaging the real-time MOS prediction value into a control instruction and sending it to a call terminal, so that the call terminal adjusts the coding and decoding parameters according to the real-time MOS prediction value.

[0039] In an embodiment, the step of packaging the real-time MOS prediction value into a control instruction and sending it to a call terminal, so that the call terminal adjusts the coding and decoding parameters according to the real-time MOS prediction value comprises:

[0040] packaging the real-time MOS prediction value and a timestamp into a control frame;

[0041] sending the control frame to the call terminal through a communication interface;

[0042] Receive the confirmation frame returned by the call terminal and analyze the confirmation code to obtain an analysis result;

[0043] When the analysis result is successful, update the current codec configuration record to the configuration corresponding to the real-time MOS prediction value.

[0044] In addition, to achieve the above object, the application further provides a voice call real-time evaluation device based on a functional network, which comprises:

[0045] A receiving module is configured to receive a network delay jitter standard deviation and a network packet loss rate in a voice call process.

[0046] A preprocessing module is configured to normalize the network delay jitter standard deviation and the network packet loss rate and calculate sliding window statistics to obtain time series enhancement features.

[0047] A functional network module is configured to input the time series enhancement features into a function library, perform function stacking processing and topological evolution processing, and obtain intermediate state results.

[0048] An output module is configured to perform weighted integration processing and activation mapping processing on the intermediate state results to obtain MOS prediction values.

[0049] In addition, to achieve the above object, the application further provides a voice call real-time evaluation device based on a functional network, which comprises a memory, a processor, and a voice call real-time evaluation program based on a functional network stored in the memory and executable on the processor, wherein the voice call real-time evaluation program based on a functional network is configured to implement the steps of the voice call real-time evaluation method based on a functional network as described above.

[0050] In addition, to achieve the above object, the application further provides a storage medium having a voice call real-time evaluation program based on a functional network stored thereon, wherein the voice call real-time evaluation program based on a functional network, when executed by a processor, implements the steps of the voice call real-time evaluation method based on a functional network as described above.

[0051] In addition, to achieve the above object, the application further provides a computer program product comprising a computer program, wherein the computer program, when executed by a processor, implements the steps of the voice call real-time evaluation method based on a functional network as described above.

[0052] The one or more technical solutions provided by the application have at least the following technical effects:

[0053] The application simplifies the complex operation of bidirectional comparison to unidirectional forward propagation by establishing a dynamically evolving functional network between network parameters and MOS values, significantly reduces the calculation amount; the topology increases and decreases nodes adaptively according to errors, so that the model continuously fits the non-stationary network; the forward propagation path is shortened to process the link, realizing millisecond-level real-time prediction; the function unit has an interpretable mapping relationship, directly revealing the influence mechanism of jitter and packet loss on voice quality. BRIEF DESCRIPTION OF DRAWINGS

[0054] The drawings incorporated into the specification and forming a part thereof, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the application.

[0055] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows, and obviously, other drawings can also be obtained by those skilled in the art without creative labor.

[0056] Figure 1 The flowchart provided by the speech call real-time evaluation method based on the functional network of the first embodiment of the present application;

[0057] Figure 2 The system overall architecture diagram provided by the speech call real-time evaluation method based on the functional network of the first embodiment of the present application;

[0058] Figure 3 The data preprocessing flowchart provided by the speech call real-time evaluation method based on the functional network of the first embodiment of the present application;

[0059] Figure 4 The model training flowchart provided by the speech call real-time evaluation method based on the functional network of the first embodiment of the present application;

[0060] Figure 5 The online learning and real-time prediction flowchart provided by the speech call real-time evaluation method based on the functional network of the first embodiment of the present application;

[0061] Figure 6 The flowchart provided by the speech call real-time evaluation method based on the functional network of the second embodiment of the present application;

[0062] Figure 7 The module structure diagram of the speech call real-time evaluation device based on the functional network of the embodiment of the present application;

[0063] Figure 8 The device structure diagram of the hardware running environment involved in the speech call real-time evaluation method based on the functional network in the embodiment of the present application.

[0064] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0065] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0066] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0067] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device capable of performing the above functions, such as a real-time voice call evaluation device based on functional networks. The following description uses a real-time voice call evaluation device based on functional networks as an example to illustrate this embodiment and the subsequent embodiments.

[0068] Based on this, embodiments of this application provide a real-time evaluation method for voice calls based on functional networks, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the real-time evaluation method for voice calls based on functional networks in this application.

[0069] In this embodiment, the real-time evaluation method for voice calls based on functional networks includes steps S10 to S40:

[0070] Step S10: Receive the standard deviation of network latency jitter and network packet loss rate during the voice call;

[0071] It's important to note that network latency jitter standard deviation is a metric for measuring network latency fluctuations. In voice calls, network latency fluctuations can affect call smoothness. For example, large latency fluctuations may cause intermittent audio cuts.

[0072] Packet loss rate refers to the proportion of data packets lost during network transmission. A high packet loss rate can lead to decreased call quality, with incomplete sound or stuttering.

[0073] like Figure 2 As shown, Figure 2 The system architecture diagram shows that the input layer is connected to the functional unit layer, the state storage layer is connected to the output layer, and the real-time evaluation results are connected to the output layer.

[0074] The input layer is connected to each computational unit of the functional unit layer, and transmits the input data to the subsequent processing module for receiving network latency jitter standard deviation (Jitter SD) and network packet loss rate (PLR) data in VoIP calls.

[0075] Data collection and preliminary preprocessing are completed, including feature standardization (normalized to the range [0, 1]) and time series feature enhancement (if the input is a time series, calculate the sliding window statistics).

[0076] The base function library in the functional unit layer includes linear functions (such as weighted summation), nonlinear functions (such as sigmoid, tanh), dynamic system functions (such as integration, differentiation), statistical functions (such as standard deviation, mean), etc.

[0077] The unit combination rule in the functional unit layer constructs a multi-layer mapping through "function stacking", with each layer output serving as the input of the next layer. Linear function units are used to perform linear transformations, such as weighted summation. They are mainly used to handle linear relationships in data, converting input data into output through linear combination. High-order polynomial function units are used to handle polynomial transformations in non-linear relationships. They can capture high-order polynomial features in data, enhancing the model's ability to fit complex patterns. Nonlinear function units are used to introduce nonlinear characteristics, such as nonlinear transformations through activation functions (such as sigmoid, tanh, etc.). Nonlinear function units enable the model to better handle complex nonlinear relationships. The output of the linear function unit can serve as the input of the high-order polynomial function unit, and the output of the high-order polynomial function unit can serve as the input of the nonlinear function unit. This combination can construct a function processing chain with different levels and complexities.

[0078] The dynamic topology adjustment step in the functional unit layer includes:

[0079] The structural evolution algorithm uses genetic algorithms to adaptively adjust the number of layers, unit types, and connection weights.

[0080] Node addition, when the validation set error exceeds the threshold, randomly insert new function units (such as LSTM units to handle time series correlation).

[0081] Node pruning, based on L1 regularization technology to delete redundant connections.

[0082] Parallel-serial hybrid structure, part of the layers use parallel function processing (such as simultaneously calculating the mean and standard deviation of Jitter SD), and the subsequent layers combine the results in series.

[0083] The adaptive function combination mechanism in the functional unit layer includes:

[0084] Conditional function switching dynamically selects the type of activation function based on input data. For example, when Jitter SD fluctuates dramatically, switch to a high-order polynomial function; when PLR approaches the threshold, enable a piecewise linear function.

[0085] The meta-learning module introduces "function meta-parameters" (such as the slope and offset of the activation function) and learns the optimal function combination strategy through the outer loop.

[0086] The state storage layer is connected to the functional unit layer and the output layer, and passes intermediate calculation results. It is used to store the intermediate results of the functional unit layer calculation units for the output layer to call.

[0087] Temporarily store intermediate variables, such as the function mapping results of each computing unit on the input data, so that the output layer can integrate them.

[0088] The output layer is connected to the state storage layer, receives and processes intermediate calculation results, and integrates these intermediate results from the state storage layer to generate the final MOS prediction value.

[0089] By combining intermediate variables through weighted summation or composite functions, the MOS prediction value is output to evaluate the voice quality of VoIP calls.

[0090] Step S20: Normalize the standard deviation of network latency jitter and network packet loss rate and calculate the sliding window statistics to obtain the time-series enhancement features;

[0091] It should be noted that normalization involves scaling the data proportionally to a specific range (such as 0 to 1). This helps with subsequent processing and comparison of data from different ranges.

[0092] Sliding window statistics capture the temporal patterns of data by moving a fixed-length window across the data sequence and calculating the statistical characteristics within that window.

[0093] The temporal enhancement features are feature vectors concatenated by statistically analyzing the network latency jitter standard deviation and the network packet loss rate through sliding windows. For each sliding window, the jitter channel and the packet loss channel each form a three-dimensional vector: the mean represents the average level of the indicator within that window, the standard deviation represents the fluctuation amplitude, and the range represents the extreme offset. These six-dimensional feature vectors are arranged in chronological order to constitute the temporal enhancement features, which are used to simultaneously provide the functional network with information on numerical level, fluctuation, and extreme values ​​to improve the robustness and discriminative power of subsequent mappings.

[0094] like Figure 3 As shown, Figure 3 This is a flowchart of the data preprocessing process. After data collection, the next step is to standardize the features of the collected data to unify the scale and distribution of the data. Based on this, sliding window statistics can be selectively calculated to capture the temporal variation characteristics of the data. After these preprocessing steps, the preprocessed data is finally obtained, preparing it for subsequent analysis and modeling.

[0095] In an implementable embodiment, step S20 comprises steps A11-A15:

[0096] A11: Obtain a first target value and a second target value of the network delay jitter standard deviation and calculate a first difference value of the first target value and the second target value.

[0097] It should be noted that the first target value and the second target value are the maximum value and the minimum value of the network delay jitter standard deviation.

[0098] The first difference value is the difference between the first target value and the second target value, which is used for normalization calculation.

[0099] A12: Subtract the second target value from the network delay jitter standard deviation of the current frame and divide by the first difference value to obtain a normalized jitter value.

[0100] It should be noted that the network delay jitter standard deviation of the current frame is the network delay jitter value at the current time.

[0101] The normalized jitter value is obtained by normalizing the current value to the range of 0 to 1.

[0102] A13: Obtain a third target value and a fourth target value of the network packet loss rate and calculate a second difference value of the third target value and the fourth target value.

[0103] It should be noted that the third target value and the fourth target value are the maximum value and the minimum value of the network packet loss rate.

[0104] The second difference value is the difference between the third target value and the fourth target value, which is used for normalization calculation.

[0105] A14: Subtract the fourth target value from the network packet loss rate of the current frame and divide by the second difference value to obtain a normalized packet loss value.

[0106] It should be noted that the network packet loss rate of the current frame is the network packet loss rate at the current time.

[0107] The normalized packet loss value is obtained by normalizing the current value to the range of 0 to 1.

[0108] A15: Traverse the normalized jitter value and the normalized packet loss value in a sliding window manner to splice a time series enhancement feature.

[0109] It should be noted that the sliding window manner is to move a fixed length window on the data sequence and calculate the statistical features in the window one by one.

[0110] The time series enhancement feature is to splice the normalized jitter value and the packet loss value into a feature vector, which is used to capture the time series change of the data.

[0111] Step S30: Input the temporal enhancement features into the function library, perform function stacking and topological evolution processing, and obtain intermediate state results;

[0112] It should be noted that the function library contains a variety of mathematical functions for processing input data.

[0113] Function stacking involves applying multiple functions sequentially to the data for multi-level processing.

[0114] Topology evolution processing involves dynamically adjusting the connection relationships between functions to adapt to changes in data.

[0115] In its implementation, function stacking starts with temporal enhancement features and calls units from the function library layer by layer in a "linear-nonlinear-statistical" order, abstracting features progressively in depth. Each output directly becomes the input of the next layer, forming a fixed-order deep link. Topology evolution calculates the verification error with the actual MOS after each layer's output. When the error exceeds a preset threshold, a function is randomly selected from those not yet called and inserted after the current layer, and the network depth is immediately incremented. If the error does not exceed the threshold, the original connection is maintained, and the current layer's output directly connects to the next layer, until the maximum number of layers is reached. The final output is the intermediate state result.

[0116] like Figure 4 As shown, Figure 4 The model training flowchart first randomly initializes the function parameters and performs interpolation, generating MOS prediction values ​​based on the initial parameters. Next, the mean squared error (MSE) between the predicted and true values ​​is calculated to evaluate prediction accuracy. Then, the function parameters are updated, and a genetic algorithm is used to optimize the function to improve model performance. The function parameters are then adjusted based on the optimization results to further improve prediction accuracy. Finally, it is determined whether the model meets the conditions for training completion. If it does, training ends; otherwise, it returns to continue training, repeating the above process until the training objective is achieved.

[0117] Step S40: Perform weighted integration and activation mapping on the intermediate state results to obtain the MOS prediction value.

[0118] It should be noted that weighted integration processing involves summing multiple data points with weights to obtain a comprehensive result.

[0119] Activation mapping involves applying a nonlinear transformation to the data using an activation function to obtain the final predicted value.

[0120] The MOS prediction value is a continuous quantity of the model output, and the value interval is usually mapped to 1.0-5.0, corresponding to the subjective listening of human beings to the grading of voice quality: 1 is extremely poor, and 5 is excellent. The numerical value is obtained by mapping the objective indexes such as network delay jitter and packet loss by a functional network, and is used to quantify the intelligibility, continuity and naturalness of the call under the given network condition, and provides an operable quantitative benchmark for the operation and maintenance side or the terminal side.

[0121] In a feasible implementation, step S40 includes steps A21-A24:

[0122] A21: expanding the intermediate state result into a plurality of state vectors in the channel dimension;

[0123] It should be noted that the channel dimension is a dimension of data, which is usually used to represent different feature channels.

[0124] The state vector is to decompose the intermediate result into a plurality of vectors, which is convenient for subsequent processing.

[0125] A22: assigning a learnable weight to each state vector and performing element-wise multiplication processing to obtain a weighted vector;

[0126] It should be noted that the learnable weight is the weight of each state vector, which is used to represent its importance.

[0127] The weighted vector is the result obtained by multiplying the state vector and the weight.

[0128] A23: inputting the weighted vector into an activation function to obtain an activation result;

[0129] It should be noted that the activation function is used to perform nonlinear transformation on the weighted vector to introduce nonlinear characteristics.

[0130] A24: performing linear mapping processing on the activation result to obtain the MOS prediction value.

[0131] It should be noted that the linear mapping processing is to map the activation result to a specific range to obtain the final MOS prediction value.

[0132] Further, the present application also includes:

[0133] Perform normalization and sliding window statistics on the real-time network delay jitter standard deviation and the real-time network packet loss rate to obtain real-time time series enhancement features;

[0134] Input the real-time time series enhancement features into the trained functional network to obtain a real-time intermediate state result;

[0135] Input the real-time intermediate state result into the output layer to obtain a real-time MOS prediction value;

[0136] The real-time MOS prediction value is encapsulated as a control instruction and sent to the call terminal to enable the call terminal to adjust the codec parameters according to the real-time MOS prediction value.

[0137] As shown in Figure 5 , Figure 5 The online learning and real-time prediction flowchart is shown in FIG. 1. After real-time data collection, the collected data is immediately subjected to online feature preprocessing to extract useful feature information. The preprocessed feature data is input into a model with a fixed topology for rapid prediction to obtain a real-time MOS prediction value. At the same time, the system continuously performs incremental training and updates the parameters using a sliding window method to ensure that the model can adapt to changes in the data and maintain the accuracy of the prediction.

[0138] It should be noted that the real-time network delay jitter standard deviation and the real-time network packet loss rate are network delay jitter standard deviation and packet loss rate data obtained in real time during voice communication. These data reflect the actual status of the current network and can be used for real-time monitoring and adjustment of call quality.

[0139] The real-time timing enhancement feature is a feature obtained by normalizing and calculating the sliding window statistics of the real-time network delay jitter standard deviation and the real-time network packet loss rate. These features can reflect the timing variation of the network in real time and provide a basis for subsequent real-time prediction.

[0140] The real-time intermediate state result is the result obtained by inputting the real-time timing enhancement feature into the trained functional network, representing the intermediate representation of the current network state in the functional network.

[0141] The real-time MOS prediction value is a prediction value obtained by weighting and integrating the real-time intermediate state result and performing activation mapping processing, reflecting the quality of the current voice call. MOS (Mean Opinion Score) is a commonly used voice quality evaluation index, with a value range of 1 to 5, and a larger value indicating better voice quality.

[0142] The control instruction is a command form encapsulating the real-time MOS prediction value, used to send to the call terminal to guide the terminal to adjust the codec parameters according to the real-time MOS prediction value. The codec parameters refer to the parameters used by the call terminal in the voice coding and decoding process, such as the coding rate, quantization accuracy, etc., which affect the quality of voice communication and bandwidth occupation, etc.

[0143] In a feasible implementation, the step of encapsulating the real-time MOS prediction value as a control instruction and sending it to the call terminal to enable the call terminal to adjust the codec parameters according to the real-time MOS prediction value includes steps A31-A34:

[0144] A31: encapsulate the real-time MOS prediction value and the timestamp into a control frame;

[0145] It should be noted that the timestamp is the time information of the recorded data.

[0146] The control frame is a data packet containing control instructions.

[0147] A32: send the control frame to the call terminal through the communication interface;

[0148] It should be noted that the communication interface is an interface for data transmission.

[0149] A33: receive the confirmation frame returned by the call terminal and parse the confirmation code to obtain a parsing result;

[0150] It should be noted that the confirmation frame is the confirmation information returned by the call terminal.

[0151] The parsing result is the parsing result of the confirmation frame, indicating whether the instruction is successfully received.

[0152] A34: when the parsing result is successful, update the current codec configuration record to the configuration corresponding to the real-time MOS prediction value.

[0153] It should be noted that the codec configuration record is the codec parameter setting of the call terminal.

[0154] The updated configuration is to adjust the codec parameters according to the real-time MOS prediction value.

[0155] The embodiment provides a voice call real-time evaluation method based on a functional network. Through functional network architecture design, a basic function library containing linear, nonlinear, dynamic system and statistical function is constructed, and a flexible functional network architecture is constructed through "function stacking" and dynamic topology adjustment (structure evolution algorithm realizes node addition and pruning) to adapt to the complex changes of network parameters in VoIP calls. Through the self-adaptive function combination mechanism, the conditional function switching and meta-learning module are designed, the type of the activation function is dynamically selected according to the input data, and the optimal function combination strategy is learned through the outer loop to improve the adaptability of the model to the dynamic network environment. Through hybrid training and optimization algorithm, a supervised-unsupervised joint training paradigm is adopted, and the AdamW algorithm and genetic algorithm are combined for parameter and structure optimization, while function parameter regularization and dynamic stability constraint are introduced to improve the prediction accuracy and stability of the model. Through the real-time evaluation method, online learning and real-time prediction are supported, and through incremental training and fixed topology forward propagation, millisecond-level real-time evaluation of MOS value of each call in VoIP calls is realized, which meets the real-time requirement.

[0156] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as the above embodiment one can refer to the above introduction, and the subsequent will not be described. On this basis, please refer to Figure 6 , step S30, comprising steps S301-S305:

[0157] Step S301, input the timing enhancement feature into the current layer function unit to obtain the current layer output;

[0158] It should be noted that the timing enhancement feature is the feature vector obtained by normalization processing and sliding window statistics, which contains the timing change information of network delay jitter standard deviation and packet loss rate.

[0159] The current layer function unit is a layer in the functional network, which contains various mathematical functions (such as linear function, nonlinear function, etc.), which is used for transforming and feature extraction of input data.

[0160] The current layer output is the result of the timing enhancement feature after being processed by the current layer function unit, which reflects the information after the transformation and extraction of the input feature by this layer.

[0161] Step S302, compare the current layer output with the corresponding real MOS label to obtain the verification error;

[0162] It should be noted that the real MOS label is the pre-labeled speech quality score, which is used as the reference standard for model training and verification.

[0163] The verification error is the difference between the current layer output and the real MOS label, which is used to evaluate the prediction accuracy of the model. The larger the verification error, the greater the gap between the model prediction result and the true value.

[0164] Step S303, when the verification error exceeds the preset threshold, select an unused function from the function library as a new function unit and insert it after the current layer, take the current layer output as the input of the new function unit, and obtain the new layer output;

[0165] It should be noted that when the verification error exceeds the preset threshold, that is, when the verification error is greater than the set threshold, it means that the current model structure is not enough to accurately predict the MOS value, and structure optimization is needed.

[0166] The function library contains various unused mathematical functions, which is used to dynamically expand the network structure.

[0167] The new function unit is an unused function selected from the function library, which is used to enhance the expression ability of the model.

[0168] The new layer output is the result of the current layer output after being processed by the new function unit, which contains new feature extraction information.

[0169] In a feasible implementation, step S303 comprises steps A41-A44:

[0170] A41: read the function identifiers of all functions in the function library that are not called by the current layer to obtain a candidate function list;

[0171] It should be noted that the function identifier is a unique identifier of each function in the function library, which is used to distinguish different functions.

[0172] The candidate function list lists all functions that are not called by the current layer, providing a range for subsequent selection.

[0173] A42: randomly select a function from the candidate function list and initialize the parameters to obtain a function unit to be inserted;

[0174] It should be noted that random selection means randomly selecting a function from the candidate function list to increase the diversity of the model.

[0175] Initializing the parameters means initializing the parameters of the selected function to ensure that its initial state is suitable for subsequent processing.

[0176] The function unit to be inserted is a function unit to be inserted into the network and has new feature extraction capability.

[0177] A43: connect the input port of the function unit to be inserted to the output port of the current layer, and connect the output port of the function unit to be inserted to the input port of the next layer, and complete the insertion;

[0178] It should be noted that the input port connection means connecting the input end of the new function unit to the output end of the current layer to ensure the continuity of data flow.

[0179] The output port connection means connecting the output end of the new function unit to the input end of the next layer to ensure the smooth progress of subsequent processing.

[0180] The completion of the insertion means that the new function unit is successfully inserted into the network structure, and the network topology changes.

[0181] A44: input the output of the current layer into the function unit to be inserted to obtain the output of the new layer.

[0182] Step S304, when the verification error does not exceed the preset threshold, directly taking the output of the current layer as the input of the next layer to obtain the output of the straight-through layer;

[0183] It should be noted that when the verification error is less than or equal to the set threshold, it means that the output of the current layer is already accurate enough and does not need to be further processed.

[0184] The straight-through layer output is the output of the current layer directly passed to the next layer, maintaining the continuity and integrity of the data.

[0185] In step S305, the newly added layer output or the straight-through layer output is taken as the input of the function unit of the next layer, until the number of layers reaches the preset number of layers. The output of the function unit of the last layer is taken as the intermediate state result.

[0186] It should be noted that the newly added layer output or the straight-through layer output is selected as the input of the next layer according to whether the verification error exceeds the threshold.

[0187] The number of layers reaching the preset number of layers means that the processing is performed according to the set network depth, ensuring the balance between the complexity and performance of the model.

[0188] The intermediate state result is the final output after the multi-layer processing, which contains the multi-level feature extraction and transformation results of the network on the input data.

[0189] The embodiment provides a voice call real-time evaluation method based on a functional network, which can significantly improve the adaptability of the model to network state changes. When the verification error is large, by inserting a new function unit, the expression ability of the model can be enhanced, so that the model can better learn the complex relationship between network parameters and voice quality. Conversely, when the verification error is small, direct straight-through can reduce unnecessary calculations and improve processing efficiency. This adaptive network topology adjustment can optimize the use of computing resources while ensuring prediction accuracy, achieving efficient and accurate prediction of the MOS value of VoIP calls.

[0190] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the voice call real-time evaluation method based on the functional network of the present application. More forms of simple transformation based on this technical concept are within the protection scope of the present application.

[0191] The present application also provides a voice call real-time evaluation device based on a functional network, which is described in detail in Figure 7 The voice call real-time evaluation device based on the functional network comprises:

[0192] The receiving module 10 is configured to receive the network delay jitter standard deviation and the network packet loss rate in the voice call process.

[0193] The preprocessing module 20 is configured to perform normalization processing on the network delay jitter standard deviation and the network packet loss rate and calculate sliding window statistics to obtain time series enhancement features.

[0194] The functional network module 30 is configured to input the time series enhancement features into a function library, perform function stacking processing and topology evolution processing, and obtain an intermediate state result.

[0195] The output module 40 is configured to perform weighted integration processing and activation mapping processing on the intermediate state result to obtain the MOS prediction value.

[0196] The speech call real-time evaluation device based on the functional network provided in the application adopts the speech call real-time evaluation method based on the functional network in the above embodiment, and can solve the technical problem of realizing real-time, stable and interpretable evaluation of the VoIP call voice quality. Compared with the prior art, the speech call real-time evaluation device based on the functional network provided in the application has the same beneficial effects as the speech call real-time evaluation method based on the functional network provided in the above embodiment, and other technical features in the speech call real-time evaluation device based on the functional network are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0197] In an embodiment, the preprocessing module 20 is further configured to obtain a first target value and a second target value of the network delay jitter standard deviation, and calculate a first difference value of the first target value and the second target value.

[0198] The normalized jitter value is obtained by subtracting the second target value from the network delay jitter standard deviation of the current frame and dividing the result by the first difference value.

[0199] The third target value and the fourth target value of the network packet loss rate are obtained, and a second difference value of the third target value and the fourth target value is calculated.

[0200] The normalized packet loss value is obtained by subtracting the fourth target value from the network packet loss rate of the current frame and dividing the result by the second difference value.

[0201] The normalized jitter value and the normalized packet loss value are traversed in a sliding window manner to form a time sequence enhanced feature.

[0202] In an embodiment, the functional network module 30 is further configured to input the time sequence enhanced feature into a current layer function unit to obtain a current layer output.

[0203] The current layer output is compared with the corresponding real MOS label to obtain a verification error.

[0204] When the verification error exceeds a preset threshold, an unused function is selected as a new function unit from a function library and inserted after the current layer, the current layer output is taken as the input of the new function unit, and a new layer output is obtained.

[0205] When the verification error does not exceed the preset threshold, the current layer output is directly taken as the input of the next layer to obtain a straight-through layer output.

[0206] The new layer output or the straight-through layer output is taken as the input of the next layer function unit until the number of layers reaches a preset number of layers, and the output of the last layer function unit is taken as the intermediate state result.

[0207] In an embodiment, the functional network module 30 is further configured to read all function identifiers in the function library that are not called by the current layer to obtain a candidate function list;

[0208] randomly select a function from the candidate function list and initialize parameters to obtain a function unit to be inserted;

[0209] connect an input port of the function unit to be inserted to an output port of the current layer, and connect an output port of the function unit to be inserted to an input port of the next layer, to complete the insertion;

[0210] input the output of the current layer into the function unit to be inserted to obtain an output of the new layer.

[0211] In an embodiment, the output module 40 is further configured to expand the intermediate state result into a plurality of state vectors in the channel dimension;

[0212] assign learnable weights to each state vector and perform element-wise multiplication processing to obtain a weighted vector;

[0213] input the weighted vector into an activation function to obtain an activation result;

[0214] perform linear mapping processing on the activation result to obtain a MOS prediction value.

[0215] In an embodiment, the output module 40 is further configured to receive a real-time voice call data stream, extract a real-time network delay jitter standard deviation and a real-time network packet loss rate;

[0216] perform normalization and sliding window statistics on the real-time network delay jitter standard deviation and the real-time network packet loss rate to obtain real-time timing enhancement features;

[0217] input the real-time timing enhancement features into the trained functional network to obtain a real-time intermediate state result;

[0218] input the real-time intermediate state result into the output layer to obtain a real-time MOS prediction value;

[0219] package the real-time MOS prediction value into a control instruction and send it to the call terminal, so that the call terminal adjusts the encoding and decoding parameters according to the real-time MOS prediction value.

[0220] In an embodiment, the output module 40 is further configured to package the real-time MOS prediction value and a timestamp into a control frame;

[0221] send the control frame to the call terminal through a communication interface;

[0222] receive an acknowledgement frame returned by the call terminal and parse the acknowledgement code to obtain a parsing result;

[0223] When the parsing result is successful, the current codec configuration record is updated to the configuration corresponding to the real-time MOS prediction value.

[0224] The application provides a speech call real-time evaluation device based on a functional network. The speech call real-time evaluation device based on the functional network comprises at least one processor and a memory connected in communication with the at least one processor. The memory stores instructions executable by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the speech call real-time evaluation method based on the functional network in the first embodiment.

[0225] Reference is made below Figure 8 which shows a structural schematic diagram of the speech call real-time evaluation device based on the functional network suitable for being used to implement the embodiments of the application. The speech call real-time evaluation device based on the functional network in the embodiments of the application can include but is not limited to mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (Personal Digital Assistant), PADs (Portable Application Description), PMPs (Portable Media Player), vehicle-mounted terminals (for example, vehicle-mounted navigation terminals) and the like and fixed terminals such as digital TVs, desktop computers and the like. Figure 8 The speech call real-time evaluation device shown is only an example and should not bring any limitation to the functions and use range of the embodiments of the application.

[0226] As Figure 8As shown, the functional network-based real-time evaluation device for voice communication can include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to programs stored in a ROM (Read Only Memory) 1002 or programs loaded from a storage device 1003 into a RAM (Random Access Memory) 1004. In the RAM 1004, various programs and data required for the functional network-based real-time evaluation device to operate are also stored. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; the storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the functional network-based real-time evaluation device to communicate wirelessly or wired with other devices to exchange data. Although the functional network-based real-time evaluation device with various systems is shown in the figure, it should be understood that all the shown systems are not required to be implemented or possessed. More or less systems can be alternatively implemented or possessed.

[0227] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network through the communication device, or installed from the storage device 1003, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the methods of the embodiments of the present disclosure are performed.

[0228] The speech call real-time evaluation device based on the functional network provided in the application adopts the speech call real-time evaluation method based on the functional network in the above embodiment, and can solve the technical problem of realizing real-time, stable and interpretable evaluation of the VoIP call voice quality. Compared with the prior art, the speech call real-time evaluation device based on the functional network provided in the application has the same beneficial effects as the speech call real-time evaluation method based on the functional network provided in the above embodiment, and other technical features in the speech call real-time evaluation device based on the functional network are the same as the features disclosed in the above embodiment method, and will not be repeated here.

[0229] It should be understood that parts of the present application can be realized by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0230] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0231] The present application provides a computer readable storage medium having computer readable program instructions (i.e. computer programs) stored thereon, the computer readable program instructions being used to execute the speech call real-time evaluation method based on the functional network in the above embodiment.

[0232] The computer readable storage medium provided in the application may be, for example, a U disk, but is not limited to an electric, magnetic, optical, electromagnetic, infrared, or semiconductor system, system, or device, or any combination of the above. More specific examples of the computer readable storage medium may include, but are not limited to, an electric connection with one or more conductive wires, a portable computer disk, a hard disk, a RAM (Random Access Memory), a ROM (Read Only Memory), an EPROM (Erasable Programmable Read Only Memory or flash memory), an optical fiber, a CD-ROM (CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the embodiment, the computer readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system, or device. The program code contained on the computer readable storage medium can be transmitted by any suitable medium, including but not limited to an electric wire, an optical cable, an RF (Radio Frequency), and the like, or any suitable combination of the above.

[0233] The computer readable storage medium described above may be contained in the functional network-based real-time evaluation device for voice calls, or may exist independently without being assembled into the functional network-based real-time evaluation device for voice calls.

[0234] The computer readable storage medium described above carries one or more programs, which, when executed by the functional network-based real-time evaluation device for voice calls, cause the functional network-based real-time evaluation device for voice calls to: receive a network delay jitter standard deviation and a network packet loss rate in a voice call process; normalize the network delay jitter standard deviation and the network packet loss rate and calculate a sliding window statistic to obtain a time series enhancement feature; input the time series enhancement feature into a function library, perform function stacking processing and topology evolution processing, and obtain an intermediate state result; perform weighted integration processing and activation mapping processing on the intermediate state result to obtain a MOS prediction value.

[0235] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0236] The computer program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0237] The modules involved in the embodiments of the present application can be implemented in software or hardware. In some cases, the names of the modules do not limit the modules themselves.

[0238] The readable storage medium provided by the present application is a computer readable storage medium, which stores computer readable program instructions (i.e., computer programs) for executing the above-mentioned functional network-based real-time evaluation method of voice calls, and can solve the technical problem of realizing real-time, stable and interpretable evaluation of voice quality of VoIP calls. Compared with the prior art, the computer readable storage medium provided by the present application has the same beneficial effects as the functional network-based real-time evaluation method of voice calls provided by the above-mentioned embodiments, and will not be described here.

[0239] The application also provides a computer program product comprising a computer program which, when executed by a processor, implements the steps of the functional network-based real-time evaluation method for voice calls as described above.

[0240] The computer program product provided by the application can solve the technical problem of realizing real-time, stable and interpretable evaluation of the voice quality of VoIP calls. Compared with the prior art, the beneficial effects of the computer program product provided by the application are the same as those of the functional network-based real-time evaluation method for voice calls provided by the above-mentioned embodiments, and are not described here.

[0241] The above only describes some embodiments of the application, and does not limit the patent scope of the application. Any equivalent structural transformation made by using the content of the application specification and drawings, or direct / indirect application in other related technical fields is included in the patent protection scope of the application.

Claims

1. A real-time evaluation method for voice calls based on functional networks, characterized in that, The method includes: The standard deviation of network latency jitter and network packet loss rate during voice calls; The network latency jitter standard deviation and the network packet loss rate are normalized and the sliding window statistics are calculated to obtain the time-series enhancement features; The temporal enhancement features are input into a function library, and function stacking and topological evolution are performed to obtain intermediate state results. The intermediate state results are weighted and integrated, and activation mapping is performed to obtain the MOS prediction value; The step of inputting the temporal enhancement features into a function library, performing function stacking and topological evolution processing to obtain intermediate state results includes: The temporal enhancement features are input into the current layer function unit to obtain the current layer output; The current layer output is compared with the corresponding real MOS label to obtain the verification error; When the verification error exceeds a preset threshold, an unused function is selected from the function library as a new function unit and inserted into the current layer. Then, the output of the current layer is used as the input of the new function unit to obtain the output of the new layer. When the verification error does not exceed the preset threshold, the current layer output is directly used as the input of the next layer to obtain the pass-through layer output; The output of the newly added layer or the output of the pass-through layer is used as the input of the next layer function unit until the number of layers reaches the preset number, and the output of the last layer function unit is used as the intermediate state result.

2. The method as described in claim 1, characterized in that, The step of normalizing the standard deviation of network latency jitter and the network packet loss rate and calculating sliding window statistics to obtain temporal enhancement features includes: Obtain a first target value and a second target value for the standard deviation of network latency jitter, and calculate a first difference between the first target value and the second target value; Subtract the second target value from the standard deviation of network latency jitter in the current frame and divide by the first difference to obtain the normalized jitter value; Obtain the third target value and the fourth target value of the network packet loss rate, and calculate the second difference between the third target value and the fourth target value; Subtract the fourth target value from the network packet loss rate of the current frame and divide by the second difference to obtain the normalized packet loss value; The normalized jitter value and the normalized packet loss value are traversed in a sliding window manner and concatenated to form a temporal enhancement feature.

3. The method as described in claim 1, characterized in that, The step of selecting an unused function from the function library as a new function unit and inserting it into the current layer, and then using the output of the current layer as the input of the new function unit to obtain the output of the new layer, includes: Read all function identifiers in the function library that have not been called by the current layer to obtain a list of candidate functions; Randomly select a function from the candidate function list and initialize its parameters to obtain the function unit to be inserted; Connect the input port of the function unit to be inserted to the output port of the current layer, and connect the output port of the function unit to be inserted to the input port of the next layer to complete the insertion; The output of the current layer is input into the function unit to be inserted to obtain the output of the new layer.

4. The method as described in claim 1, characterized in that, The step of performing weighted integration and activation mapping processing on the intermediate state results to obtain the MOS prediction value includes: The intermediate state results are expanded into multiple state vectors according to the channel dimension; Learnable weights are assigned to each of the state vectors and multiplied element by element to obtain a weighted vector; The weighted vector is input into the activation function to obtain the activation result; The activation results are linearly mapped to obtain the MOS prediction value.

5. The method as described in claim 1, characterized in that, The method further includes: Receive real-time voice call data streams and extract the real-time network latency jitter standard deviation and real-time network packet loss rate; Normalization and sliding window statistics are performed on the standard deviation of the real-time network latency jitter and the real-time network packet loss rate to obtain real-time time-series enhancement features; The real-time temporal enhancement features are input into the trained functional network to obtain real-time intermediate state results; The real-time intermediate state result is input into the output layer to obtain the real-time MOS prediction value; The real-time MOS prediction value is encapsulated into a control command and sent to the call terminal so that the call terminal adjusts the encoding and decoding parameters according to the real-time MOS prediction value.

6. The method as described in claim 5, characterized in that, The step of encapsulating the real-time MOS prediction value into a control command and sending it to the call terminal, so that the call terminal adjusts the encoding and decoding parameters according to the real-time MOS prediction value, includes: The real-time MOS prediction value and timestamp are encapsulated into a control frame; The control frame is sent to the call terminal via the communication interface; Receive the confirmation frame returned by the call terminal and parse the confirmation code to obtain the parsing result; When the parsing result is successful, the current encoding / decoding configuration record is updated to the configuration corresponding to the real-time MOS prediction value.

7. A real-time voice call evaluation device based on functional networks, characterized in that, The device includes: The receiving module is used to receive the standard deviation of network latency jitter and network packet loss rate during voice calls; The preprocessing module is used to normalize the standard deviation of network latency jitter and the network packet loss rate and calculate the sliding window statistics to obtain the temporal enhancement features; The functional network module is used to input the temporal enhancement features into the function library, perform function stacking and topology evolution processing, and obtain intermediate state results. The functional network module is also used to input the temporal enhancement features into the current layer function unit to obtain the current layer output; The current layer output is compared with the corresponding real MOS label to obtain the verification error; When the verification error exceeds a preset threshold, an unused function is selected from the function library as a new function unit and inserted into the current layer. Then, the output of the current layer is used as the input of the new function unit to obtain the output of the new layer. When the verification error does not exceed the preset threshold, the current layer output is directly used as the input of the next layer to obtain the pass-through layer output; The output of the newly added layer or the output of the through layer is used as the input of the next layer function unit until the number of layers reaches the preset number of layers, and the output of the last layer function unit is used as the intermediate state result. The output module is used to perform weighted integration processing and activation mapping processing on the intermediate state results to obtain the MOS prediction value.

8. A real-time voice call evaluation device based on functional networks, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the real-time evaluation method for voice calls based on functional networks as described in any one of claims 1 to 6.

9. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the real-time evaluation method for voice calls based on functional networks as described in any one of claims 1 to 6.

Images