Method and system for evaluating voice quality of mobile perception terminal based on deep learning
Through a five-layer fully connected neural network model based on deep learning, combining time domain, frequency domain and amplitude characteristics, the speech quality expression level is calculated and adjusted, and the complexity problem of speech quality evaluation in mobile perception terminals is solved, and the complexity is achieved Accurate voice quality assessment in the environment.
Patent Information
- Application Number
- CN202411124123.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-16
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-08-16
AI Technical Summary
The prior art cannot effectively evaluate the voice quality in mobile-perceptual terminals, especially in complex noise interference environments, resulting in the speech quality assessment being unable to be accurately performed.
Using a deep learning-based method, a five-layer fully connected neural network model is constructed, and the speech quality expression level is calculated using time domain, frequency domain and amplitude characteristics, and the model is adjusted through reverse guidance until the prediction results reach the expected value.
It effectively improves the generalization ability of the speech evaluation model, allowing it to conduct accurate speech quality evaluation in complex speech acquisition environments, and overcomes the problems of limited data set number and limited environmental coverage.
Smart Images

Figure CN118645085B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of speech analysis, and in particular to a method and system for evaluating speech quality of a mobile perception terminal based on deep learning. Background Art
[0002] Mobile sensing terminals (such as smartphones, smart speakers, etc.) are widely used in voice communication, voice recognition, voice control and other scenarios. Voice quality has a great impact on the use of mobile sensing terminals.
[0003] At the same time, in the prior art, the voice quality evaluation algorithm includes the perceptual evaluation algorithm of voice quality and the perceptual objective voice quality evaluation algorithm, etc. The input voice of the above algorithm generally requires no noise interference, and the application environment of the mobile perception terminal is complex and generally has a lot of other noise interference. Therefore, the actual application environment of the mobile perception terminal is obviously completely different from the application environment required by the prior art. Therefore, the voice quality evaluation of the mobile terminal cannot be evaluated by the above prior art. Therefore, the application of the prior art in the field of mobile perception terminals has limitations. Summary of the invention
[0004] In order to solve the above technical problems, the purpose of the present invention is to provide a method and system for evaluating the voice quality of a mobile perception terminal based on deep learning to solve the above problems. In order to achieve the above purpose, the technical solution adopted by the present invention is as follows:
[0005] In a first aspect, the present application provides a method for evaluating the voice quality of a mobile perception terminal based on deep learning, the method comprising: obtaining voice data to be evaluated, test voice data and a data set, the data set comprising at least two voice data and a quality evaluation result corresponding to each voice data, the quality evaluation result being obtained by manual scoring; constructing a deep learning mathematical model, and using the data set to train the deep learning mathematical model to obtain a voice evaluation model; calculating a quality expression level of the test voice data, the quality expression level being calculated by time domain features, frequency domain features and amplitude of the test voice data; inputting the test voice data into the voice evaluation model for calculation to obtain a test quality evaluation result; comparing the test quality evaluation result with the quality expression level, and modifying the parameters of the data set or the deep learning mathematical model until the difference between the test quality evaluation result and the quality expression level is less than a preset threshold, to obtain an adjusted voice evaluation model; inputting the voice data to be evaluated into the voice evaluation model for calculation to obtain a quality evaluation result of the voice data to be evaluated.
[0006] In combination with the first aspect, in a possible implementation method, the quality expression level of the test speech data is calculated, including: using a Hamming window function to segment the test speech data to obtain a number of short-time frame speech data segments; calculating the differences between the short-time frame speech data segments to obtain an instability evaluation value of the test speech data; Fourier transforming the test speech data to obtain a spectrogram; and calculating the quality expression level based on the spectrogram and the instability evaluation value.
[0007] In combination with the first aspect, in a possible implementation manner, the frame length of each of the short time frames is 25 ms, and the frame shift between the short time frames is set to 10 ms.
[0008] In combination with the first aspect, in a possible implementation method, the differences between the speech data segments of the short time frames are calculated to obtain the instability evaluation value of the test speech data, including: counting the number of times the data curve corresponding to each speech data segment passes through the 0 value line, recorded as the zero-crossing rate; calculating according to the corresponding amplitude and zero-crossing rate of the data curve corresponding to each speech data segment at each moment, to obtain the data validity corresponding to each speech data segment; calculating according to the first preset calculation formula and the data validity corresponding to each speech data segment, to obtain the instability evaluation value of the test speech data.
[0009] In combination with the first aspect, in a possible implementation method, the quality expression level is calculated based on the spectrum diagram and the instability evaluation value, and then further includes: obtaining environmental information when the test voice data is collected, the environmental information including the network connection log and the base station switching frequency; obtaining the number of network connection warnings and the number of log retransmissions according to the network connection log statistics; calculating the packet loss abnormality rate based on the number of network connection warnings, the number of log retransmissions and the base station switching frequency; updating the quality expression level based on the packet loss abnormality rate to obtain an updated quality expression level.
[0010] In combination with the first aspect, in a possible implementation, the deep learning mathematical model is a five-layer fully connected neural network mathematical model.
[0011] In the second aspect, the present application also provides a mobile perception terminal voice quality evaluation system based on deep learning, including: a first acquisition module, used to obtain voice data to be evaluated, test voice data and a data set, wherein the data set includes at least two voice data and a quality evaluation result corresponding to each voice data, and the quality evaluation result is obtained by manual scoring; a model training module, used to construct a deep learning mathematical model, and use the data set to train the deep learning mathematical model to obtain a voice evaluation model; a first calculation module, used to calculate the quality expression level of the test voice data, and the quality expression level is calculated by the time domain characteristics, frequency domain characteristics and amplitude of the test voice data; a preliminary evaluation module, used to input the test voice data into the voice evaluation model for calculation to obtain a test quality evaluation result; a correction module, used to compare the test quality evaluation result with the quality expression level, and modify the parameters of the data set or the deep learning mathematical model until the difference between the test quality evaluation result and the quality expression level is less than a preset threshold, thereby obtaining an adjusted voice evaluation model; a quality evaluation module, used to input the voice data to be evaluated into the voice evaluation model for calculation to obtain the quality evaluation result of the voice data to be evaluated.
[0012] In combination with the second aspect, in a possible implementation, the first calculation module includes: a segmentation module, used to segment the test speech data using a Hamming window function to obtain speech data segments of several short-time frames; a second calculation module, used to calculate the differences between the speech data segments of the short-time frames to obtain an instability evaluation value of the test speech data; a Fourier transform module, used to Fourier transform the test speech data to obtain a spectrogram; and a third calculation module, used to calculate the quality expression level based on the spectrogram and the instability evaluation value.
[0013] In combination with the second aspect, in a possible implementation, the second calculation module includes: a first statistical module, used to count the number of times the data curve corresponding to each speech data segment passes through the 0 value line, recorded as the zero-crossing rate; a first sub-calculation module, used to calculate the data validity corresponding to each speech data segment based on the corresponding amplitude and zero-crossing rate of the data curve corresponding to each speech data segment at each moment; a second sub-calculation module, used to calculate the instability evaluation value of the test speech data based on the first preset calculation formula and the data validity corresponding to each speech data segment.
[0014] In combination with the second aspect, in a possible implementation method, the first calculation module includes: a second acquisition module, used to obtain environmental information when the test voice data is collected, and the environmental information includes a network connection log and a base station switching frequency; a second statistical module, used to obtain the number of network connection warnings and the number of log retransmissions based on the network connection log statistics; a third sub-calculation module, used to calculate the packet loss abnormality rate based on the number of network connection warnings, the number of log retransmissions and the base station switching frequency; an update module, used to update the quality expression level based on the packet loss abnormality rate to obtain an updated quality expression level.
[0015] The present invention has the following beneficial effects:
[0016] In the present invention, a speech evaluation model based on deep learning is predicted for an additional test speech at the same time, and the speech quality expression level is calculated based on time domain, frequency domain, amplitude and other data. The speech quality expression level is used to reversely guide the speech evaluation model based on deep learning to be modified until the prediction result of the speech evaluation model based on deep learning reaches the expected value. The present invention overcomes the problem that the number of data sets obtained through mobile sensing terminals is limited and the environmental coverage is limited in the above manner, and effectively improves the generalization level of the speech evaluation model based on deep learning, so that the speech evaluation model is more competent for complex speech collection environments. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0018] Figure 1 A flowchart of a method for evaluating voice quality of a mobile perception terminal based on deep learning provided by an embodiment 1 of the present invention;
[0019] Figure 2 A schematic flow chart of step S3 provided in Example 1 of the present invention;
[0020] Figure 3 This is a schematic diagram of the test voice data after being segmented in Embodiment 1 of the present invention;
[0021] Figure 4 This is a schematic diagram of the structure of the deep learning-based mobile perception terminal voice quality evaluation system described in Example 2 of the present invention;
[0022] Figure 5This is a schematic diagram of the structure of the first computing module described in Example 2 of the present invention. DETAILED DESCRIPTION
[0023] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the specific implementation method, structure, features and effects of a method and system for evaluating the voice quality of a mobile perception terminal based on deep learning proposed by the present invention in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.
[0024] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
[0025] Embodiment 1:
[0026] The following is a detailed description of a method and system for evaluating voice quality of a mobile perception terminal based on deep learning provided by the present invention in conjunction with the accompanying drawings.
[0027] See also Figure 1 , which shows a flowchart of a method for evaluating voice quality of a mobile perception terminal based on deep learning provided by an embodiment of the present invention, including step S1, step S2, step S3, step S4, step S5 and step S6.
[0028] S1. Acquire speech data to be evaluated, test speech data and a data set, wherein the data set includes at least two pieces of speech data and a quality evaluation result corresponding to each piece of speech data, and the quality evaluation result is obtained by manual scoring.
[0029] It should be noted that, in this embodiment, the value range of the quality evaluation result is between 0 and 1. The speech data to be evaluated, the test speech data and each speech data in the data set are in the data format of amplitude accompanied by time movement, and include at least two speech packets.
[0030] S2. Construct a deep learning mathematical model, and use the data set to train the deep learning mathematical model to obtain a speech evaluation model.
[0031] It should be noted that the deep learning mathematical model in this application is a five-layer fully connected neural network mathematical model.
[0032] At the same time, regarding the data set mentioned in this embodiment, those skilled in the art may choose to calculate the relevant features such as time domain performance features, frequency domain performance features, signal-to-noise ratio and network connection status for each voice data for subsequent calculation of the deep learning mathematical model. The calculation of the relevant features is a technical means well known to those skilled in the art and will not be repeated in the present invention.
[0033] Furthermore, in the present embodiment, the training process of the deep learning mathematical model is described. In the present embodiment, the data set is divided into a training set and a validation set in a ratio of 6:3, and the training set is used as the training basis of the deep learning mathematical model. It should also be noted that the loss function used in the training process is the cross entropy function. For those skilled in the art, the division ratio of the training set and the validation set and the selection of the loss function can be modified according to actual needs, and no specific restrictions are made in the present invention.
[0034] S3. Calculate the quality expression level of the test speech data, where the quality expression level is calculated based on the time domain features, frequency domain features and amplitude of the test speech data.
[0035] In the present invention, it is considered that the number of data sets obtained by mobile sensing terminals is limited and the environmental coverage is limited. Therefore, relying solely on a data set to train a deep learning mathematical model is not enough to improve its generalization ability and is not competent for complex voice collection environments. Therefore, the present application also includes verifying the generalization ability of the voice evaluation model through an additional test voice.
[0036] Therefore, in the present invention, the test voice data is collected by the mobile sensing terminal in a complex voice environment, such as music, vendors hawking or people talking around. Similarly, the network status during the test voice data collection can be in a more complex situation, such as the network is disconnected multiple times.
[0037] Specifically, see Figure 2 , this step S3 also includes step S31, step S32, step S33 and step S34.
[0038] S31. Segment the test speech data using a Hamming window function to obtain a number of short-time frame speech data segments.
[0039] That is, see Figure 3, in this step, the test voice data obtained in the mobile perception terminal is segmented into short-time frames by using a Hamming window function, that is, a test voice signal is segmented into r short-time frames, where r is a positive integer. In the segmentation, each short-time frame is defined to be 25 ms long, and the frame shift between the short-time frames is set to 10 ms. Assuming that the duration of the obtained test voice data is 3 min, then a total of r = 7200 short-time frames can be obtained through the above segmentation. Setting an overlap rate of 50% helps to improve the continuity and stability of subsequent feature extraction.
[0040] S32. Calculate the difference between the voice data segments of the short-time frames to obtain the instability evaluation value of the test voice data.
[0041] In this embodiment, considering the test voice data obtained through the mobile perception terminal, a clear voice will have the following characteristics at the voice data level: relatively high short-time energy and a relatively low zero-crossing rate. At the same time, for a continuous voice data, it will also have the following characteristics: the changes between the voice data segments of consecutive short-time frames should be relatively close, without significant changes. If there are significant changes between the voice data segments of consecutive short-time frames, it may indicate that the voice data is unstable or interfered.
[0042] Therefore, in this embodiment, first, the quality of the voice is judged whether there is an abnormal situation in combination with the above characteristics. Specifically, this step includes step S321, step S322, and step S323.
[0043] S321. Count the number of times the data curve corresponding to each voice data segment passes through the 0 value line, and record it as the zero-crossing rate .
[0044] S322. Calculate according to the amplitude value and zero-crossing rate corresponding to each moment of the data curve corresponding to each voice data segment to obtain the data validity corresponding to each voice data segment.
[0045] Among them, the data validity calculation formula in this step is as follows:
[0046]
[0047] In the above formula, represents the data validity corresponding to the e-th short-time frame, where e < r, and r is the total number of short-time frames obtained by segmenting a test voice signal; represents the amplitude value of the m-th data point in the voice data within this short-time frame; n represents that there are n data points in the voice data segment corresponding to the e-th short-time frame; represents the zero-crossing rate of the voice data within this short-time frame.
[0048] In the above calculation formula, It represents the short-time energy in the speech data within the short-time frame. The smaller the average value, the lower the short-time energy. The lower the energy, the weaker the speech data is, so that the real speech data is covered. A high zero-crossing rate may indicate that there is too much silence or invalid information in the surface speech data. The product indicates the short-term data validity of the voice data. The larger the product, the poorer the voice data quality in the short-term frame and the poorer the data validity. The above calculation formula can better meet the validity of the voice data and provide data support for the subsequent calculation of the difference between the short-term frames of the voice data.
[0049] S323: Calculate the data validity corresponding to each voice data segment according to the first preset calculation formula to obtain the instability evaluation value of the test voice data. .
[0050]
[0051] In the above formula, represents the instability evaluation value; r represents the total number of short time frames; represents the difference between the data validity corresponding to the e+1th short time frame and the data validity corresponding to the eth short time frame; It represents the sum of the absolute values of the differences between all adjacent short-time frames.
[0052] In the above calculation formula, the larger the Q is, the more significant changes may exist between consecutive short time frames, which further indicates that the test speech data is unstable or disturbed.
[0053] Furthermore, in this embodiment, considering that the data characteristics of the test voice data are essentially monitoring data in a time series, the data changes in the time series should have certain manifestations in the time domain and the frequency domain, and they are mutually verified; if the manifestation in the time series is not good, the voice data manifestation in the frequency domain is also not good for the abnormal data manifestation. Therefore, in this application, steps S33 and S34 are also included to further provide the data basis for voice quality analysis through the time domain and the frequency domain.
[0054] S33, performing Fourier transform on the test speech data to obtain a spectrogram.
[0055] S34, calculating the quality expression level based on the spectrum graph and the instability evaluation value .
[0056] The calculation formula of quality expression level is as follows:
[0057]
[0058] in, Indicates the quality expression level; represents the instability assessment value; Represents the slope value between the adjacent mth data point and m+1th data point in the spectrum of the test speech data; similarly, It represents the slope value between the adjacent m+1th data point and the m+2th data point in the spectrum diagram of the test speech data; there are n-1 groups of difference values in total; the 0.01 in the denominator is a constant to prevent the denominator from being zero.
[0059] In the above calculation formula, It indicates the flatness of the spectrum of the test speech data. The smaller the average value, the flatter the spectrum graph of the test speech data. If the spectrum graph is too flat, it may mean that the test speech data loses some spectrum information, resulting in poor listening experience. It can also be called the authenticity of the abnormal expression of the test voice data collected by the mobile perception terminal. The larger the product is, the worse the expression of the voice data in the time domain and the frequency domain, that is, the quality evaluation result of the voice data is poor.
[0060] Therefore, the quality expression level of the test speech collected by the mobile sensing terminal is obtained by comprehensively judging from the time domain and frequency domain in the process of step S31 to step S34. However, in this application, considering that the application environment of the mobile sensing terminal is often not a fixed location, it may be a constantly moving state. Therefore, the present invention also considers correcting the quality expression level of the test speech collected by the mobile sensing terminal from the perspective of the network state. Specifically, see Figure 2 Step S3 also includes step S35, step S36, step S37 and step S38.
[0061] S35. Acquire the environment information when the test voice data is collected, wherein the environment information includes a network connection log and a base station switching frequency D.
[0062] S36: Obtain the number of network connection warnings based on the network connection log statistics and log retransmission times.
[0063] S37. Calculate the packet loss abnormality rate according to the number of network connection warnings, the number of log retransmissions and the base station switching frequency.
[0064] The calculation formula of the packet loss abnormality rate is as follows:
[0065]
[0066] in, Indicates the abnormal packet loss rate; Indicates the number of warnings recorded in the network connection log of the mobile sensing terminal when the test voice data is collected. Assuming that there are B voice packets in the test voice data, It represents the number of retransmissions of the b-th type voice packet in the test voice data; D represents the base station switching frequency of the mobile perception terminal when collecting the test voice data.
[0067] In the above calculation formula, It indicates the abnormal packet loss rate of the mobile sensing terminal. The larger the product, the more serious the abnormal packet loss during voice data transmission during the process of the mobile sensing terminal acquiring voice data, and the mobile sensing terminal switches base stations frequently. Frequent base station switching may indicate that the signal quality of the currently connected base station is unstable, which may be caused by long distance, signal attenuation or network congestion. That is, it means that there is an abnormality in the network connection of the mobile sensing terminal. The network connection problem may cause poor quality of voice data and affect the evaluation of voice data quality.
[0068] S38. Update the quality expression level based on the abnormal packet loss rate to obtain an updated quality expression level.
[0069] Among them, the update formula of the quality expression level is as follows:
[0070]
[0071] in, represents the updated quality expression level; represents the quality expression level before updating; Indicates the abnormal packet loss rate; represents the hyperbolic tangent function.
[0072] In the above calculation formula, It represents the updated expression level of the voice quality of the mobile perception terminal. The larger the product is, the more realistic the expression level of the voice quality of the mobile perception terminal is, and the more it can reflect the quality problem of the voice data; that is, the more obvious the abnormal behavior of the voice data of the mobile perception terminal is.
[0073] S4. Input the test speech data into the speech evaluation model for calculation to obtain a test quality evaluation result.
[0074] S5. Compare the test quality evaluation result with the quality expression level, and modify the parameters of the data set or deep learning mathematical model until the difference between the test quality evaluation result and the quality expression level is less than a preset threshold, thereby obtaining an adjusted speech evaluation model.
[0075] It should be noted that the modification of the parameters of the data set or deep learning mathematical model mentioned in this step includes but is not limited to increasing the data set collection scenario, improving the collection method of training data, increasing the amount of complex data collection, modifying the model parameters of the deep learning mathematical model, modifying the loss function of the deep learning mathematical model, etc. The preset threshold is 0.01.
[0076] S6. Input the speech data to be evaluated into the speech evaluation model for calculation to obtain a quality evaluation result of the speech data to be evaluated.
[0077] In this application, the quality expression level is specifically calculated by combining the voice quality evaluation requirements of mobile perception terminals in different networks and under different conditions. Then, the output error of the voice evaluation model is measured by the quality expression level, and the voice evaluation model is modified according to the quality expression level and the output of the voice evaluation model, until the error between the quality expression level and the output of the voice evaluation model is small, the voice evaluation model is modified. In this way, the voice evaluation model can be better generalized, and it can be used in the use environment of mobile perception terminals and can make more accurate voice data quality evaluation.
[0078] Embodiment 2:
[0079] like Figure 4 and Figure 5 As shown, this embodiment provides a mobile perception terminal voice quality evaluation system based on deep learning, and the system includes:
[0080] The first acquisition module is used to acquire the speech data to be evaluated, the test speech data and the data set, wherein the data set includes at least two speech data and the quality evaluation result corresponding to each speech data, and the quality evaluation result is obtained by manual scoring.
[0081] The model training module is used to construct a deep learning mathematical model and use the data set to train the deep learning mathematical model to obtain a speech evaluation model.
[0082] The first calculation module is used to calculate the quality expression level of the test speech data, where the quality expression level is calculated based on the time domain features, frequency domain features and amplitude of the test speech data.
[0083] The preliminary evaluation module is used to input the test speech data into the speech evaluation model for calculation to obtain a test quality evaluation result.
[0084] A correction module is used to compare the test quality evaluation result with the quality expression level, and to modify the parameters of the data set or the deep learning mathematical model until the difference between the test quality evaluation result and the quality expression level is less than a preset threshold, thereby obtaining an adjusted speech evaluation model.
[0085] The quality evaluation module is used to input the speech data to be evaluated into the speech evaluation model for calculation to obtain the quality evaluation result of the speech data to be evaluated.
[0086] In some specific embodiments, the first calculation module includes:
[0087] The segmentation module is used to segment the test speech data using a Hamming window function to obtain a plurality of short-time frame speech data segments.
[0088] The second calculation module is used to calculate the difference between the speech data segments of the short time frame to obtain the instability evaluation value of the test speech data.
[0089] The Fourier transform module is used to perform Fourier transform on the test speech data to obtain a spectrogram.
[0090] A third calculation module is used to calculate the quality expression level based on the frequency spectrum and the instability evaluation value.
[0091] In some specific embodiments, the second calculation module includes:
[0092] The first statistical module is used to count the number of times that the data curve corresponding to each speech data segment passes through the zero value line, which is recorded as the zero crossing rate.
[0093] The first sub-calculation module is used to calculate the data validity corresponding to each speech data segment according to the corresponding amplitude and zero-crossing rate of the data curve corresponding to each speech data segment at each moment.
[0094] The second sub-calculation module is used to obtain the instability evaluation value of the test voice data according to the first preset calculation formula and the data validity calculation corresponding to each voice data segment.
[0095] In some specific embodiments, the first calculation module includes:
[0096] The second acquisition module is used to acquire the environmental information when the test voice data is collected, and the environmental information includes a network connection log and a base station switching frequency.
[0097] The second statistical module is used to obtain the number of network connection warnings and the number of log retransmissions according to the network connection log statistics.
[0098] The third sub-calculation module is used to calculate the packet loss abnormality rate according to the number of network connection warnings, the number of log retransmissions and the base station switching frequency.
[0099] An updating module is used to update the quality expression level based on the abnormal packet loss rate to obtain an updated quality expression level.
[0100] It should be noted that, regarding the system in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.
[0101] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0102] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.
Claims
1. A method for evaluating the voice quality of a mobile perception terminal based on deep learning, characterized in that: The method comprises: Acquire speech data to be evaluated, test speech data, and a data set, wherein the data set includes at least two pieces of speech data and a quality evaluation result corresponding to each piece of speech data, wherein the quality evaluation result is obtained by manual scoring; Constructing a deep learning mathematical model, and using the data set to train the deep learning mathematical model to obtain a speech evaluation model; Calculating a quality expression level of the test speech data, wherein the quality expression level is calculated based on time domain features, frequency domain features and amplitude of the test speech data; Inputting the test speech data into a speech evaluation model for calculation to obtain a test quality evaluation result; Comparing the test quality evaluation result with the quality expression level, and modifying the parameters of the data set or the deep learning mathematical model until the difference between the test quality evaluation result and the quality expression level is less than a preset threshold, thereby obtaining an adjusted speech evaluation model; Inputting the speech data to be evaluated into the adjusted speech evaluation model for calculation to obtain a quality evaluation result of the speech data to be evaluated; Calculating the quality expression level of the test speech data includes: Using a Hamming window function to segment the test speech data to obtain a plurality of short-time frame speech data segments; Calculating the difference between the speech data segments of the short time frames to obtain an instability evaluation value of the test speech data; Performing Fourier transformation on the test speech data to obtain a spectrogram; Calculating the quality expression level based on the frequency spectrum and the instability evaluation value; Calculating the difference between the speech data segments of the short time frames to obtain the instability evaluation value of the test speech data includes: Count the number of times the data curve corresponding to each speech data segment passes through the 0 value line, which is recorded as the zero-crossing rate; Calculate the data validity corresponding to each voice data segment according to the corresponding amplitude and zero-crossing rate of the data curve corresponding to each voice data segment at each time; According to the first preset calculation formula and the data validity calculation corresponding to each voice data segment, an instability evaluation value of the test voice data is obtained.
2. The method for evaluating the voice quality of a mobile perception terminal based on deep learning according to claim 1, characterized in that: The frame length of each of the short time frames is 25 ms, and the frame shift between the short time frames is set to 10 ms.
3. The method for evaluating the voice quality of a mobile perception terminal based on deep learning according to claim 1, characterized in that: The quality expression level is calculated based on the frequency spectrum and the instability evaluation value, and then further includes: Acquire environmental information when the test voice data is collected, the environmental information including network connection logs and base station switching frequency; Obtaining the number of network connection warnings and the number of log retransmissions according to the network connection log statistics; The packet loss abnormality rate is calculated according to the number of network connection warnings, the number of log retransmissions and the base station switching frequency; The quality expression level is updated based on the abnormal packet loss rate to obtain an updated quality expression level.
4. The method for evaluating the voice quality of a mobile perception terminal based on deep learning according to claim 1, characterized in that: The deep learning mathematical model is a five-layer fully connected neural network mathematical model.
5. A mobile perception terminal voice quality evaluation system based on deep learning, executing the mobile perception terminal voice quality evaluation method based on deep learning according to any one of claims 1 to 4, characterized in that: include: A first acquisition module is used to acquire speech data to be evaluated, test speech data and a data set, wherein the data set includes at least two speech data and a quality evaluation result corresponding to each speech data, wherein the quality evaluation result is obtained by manual scoring; A model training module, used to construct a deep learning mathematical model, and use the data set to train the deep learning mathematical model to obtain a speech evaluation model; A first calculation module, used for calculating the quality expression level of the test speech data, wherein the quality expression level is calculated by the time domain characteristics, frequency domain characteristics and amplitude of the test speech data; A preliminary evaluation module, used for inputting the test speech data into a speech evaluation model for calculation to obtain a test quality evaluation result; A correction module, used for comparing the test quality evaluation result with the quality expression level, and modifying the parameters of the data set or the deep learning mathematical model until the difference between the test quality evaluation result and the quality expression level is less than a preset threshold, thereby obtaining an adjusted speech evaluation model; The quality evaluation module is used to input the speech data to be evaluated into the adjusted speech evaluation model for calculation to obtain the quality evaluation result of the speech data to be evaluated.
6. The deep learning-based mobile perception terminal voice quality evaluation system according to claim 5, characterized in that: The first calculation module includes: A segmentation module, used for segmenting the test voice data using a Hamming window function to obtain a plurality of short-time frame voice data segments; A second calculation module is used to calculate the difference between the speech data segments of the short time frame to obtain the instability evaluation value of the test speech data; A Fourier transform module, used for Fourier transforming the test speech data to obtain a spectrogram; A third calculation module is used to calculate the quality expression level based on the frequency spectrum and the instability evaluation value.
7. The deep learning-based mobile perception terminal voice quality evaluation system according to claim 6, characterized in that: The second calculation module includes: The first statistical module is used to count the number of times the data curve corresponding to each voice data segment passes through the zero value line, which is recorded as the zero crossing rate; A first sub-calculation module, configured to calculate the data validity corresponding to each speech data segment according to the corresponding amplitude and zero-crossing rate of the data curve corresponding to each speech data segment at each time; The second sub-calculation module is used to obtain the instability evaluation value of the test voice data according to the first preset calculation formula and the data validity calculation corresponding to each voice data segment.
8. The deep learning-based mobile perception terminal voice quality evaluation system according to claim 5, characterized in that: The first calculation module includes: A second acquisition module is used to acquire the environmental information when the test voice data is collected, wherein the environmental information includes a network connection log and a base station switching frequency; A second statistical module is used to obtain the number of network connection warnings and the number of log retransmissions according to the network connection log statistics; A third sub-calculation module is used to calculate the packet loss abnormality rate according to the number of network connection warnings, the number of log retransmissions and the base station switching frequency; An updating module is used to update the quality expression level based on the packet loss abnormality rate to obtain an updated quality expression level.
