Processing method, device and equipment

Abstract

The embodiment of the invention provides a processing method, device and equipment. The processing method comprises the following steps: acquiring first voice data; analyzing the first voice data to obtain an audio parameter of the first voice data, wherein the audio parameter is used for representing the performance of the sound signal of the first voice data at each time point in each frequency band; if the audio parameter represents that the first voice data does not meet the quality condition, processing the first voice data based on a target model to generate second voice data, the audio parameter of the second voice data meeting the quality condition; wherein the second voice data is used for output, and sound generated by output of the second voice data is continuous and clear.

Description

Technical Field

[0001] The present application relates to but is not limited to the field of audio processing technology, and in particular to a processing method, device and equipment. Background Art

[0002] In the related art, for the solution of speech repair, a big data model is directly used to compensate for the missing and poor-quality speech signals in the original speech signals. However, there is still the problem that the compensated speech is contrary to the user's original intention, thereby reducing the user experience. Summary of the invention

[0003] In view of this, the embodiments of the present application at least provide a processing method, device and equipment.

[0004] The technical solution of the embodiment of the present application is implemented as follows:

[0005] In a first aspect, an embodiment of the present application provides a processing method, including obtaining first voice data; analyzing the first voice data to obtain audio parameters of the first voice data, the audio parameters being used to characterize the performance of the first voice data at each time point in each frequency band on the sound signal; if the audio parameters characterize that the first voice data does not meet the quality condition, processing the first voice data based on a target model to generate second voice data, the audio parameters of the second voice data meeting the quality condition; wherein the second voice data is used for output, and the sound generated by the second voice data output is continuous and clear.

[0006] In a second aspect, an embodiment of the present application provides a processing device, comprising: an acquisition module for obtaining first voice data; an analysis module for analyzing the first voice data to obtain audio parameters of the first voice data, wherein the audio parameters are used to characterize the performance of the first voice data in each frequency band on the sound signal at each time point; a generation module for processing the first voice data based on a target model to generate second voice data if the audio parameters characterize that the first voice data does not meet the quality conditions, and the audio parameters of the second voice data meet the quality conditions; wherein the second voice data is used for output, and the sound generated by the second voice data output is continuous and clear.

[0007] In a third aspect, an embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that can be executed on the processor, and when the processor executes the program, some or all of the steps in the above method are implemented.

[0008] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which implements some or all of the steps in the above method when executed by a processor.

[0009] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program or instructions, which, when executed by a processor, implement some or all of the steps in the above method.

[0010] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.

[0012] Figure 1 A schematic diagram of an implementation flow of a processing method provided in an embodiment of the present application;

[0013] Figure 2 A schematic diagram of an implementation flow of a processing method provided in an embodiment of the present application;

[0014] Figure 3 A schematic diagram of an implementation flow of a processing method provided in an embodiment of the present application;

[0015] Figure 4 A schematic diagram of an implementation flow of a processing method provided in an embodiment of the present application;

[0016] Figure 5 A schematic diagram of an implementation flow of a processing method provided in an embodiment of the present application;

[0017] Figure 6 A schematic diagram of an implementation flow of a processing method provided in an embodiment of the present application;

[0018] Figure 7 A schematic diagram of an implementation flow of a processing method provided in an embodiment of the present application;

[0019] Figure 8 A schematic diagram of an implementation flow of a processing method provided in an embodiment of the present application;

[0020] Fig. 9 A schematic diagram of an implementation flow of a processing method provided in an embodiment of the present application;

[0021] Fig.10 A schematic diagram of an implementation flow of a processing method provided in an embodiment of the present application;

[0022] Fig.11 A schematic diagram of an implementation flow of a processing method provided in an embodiment of the present application;

[0023] Fig.12A schematic diagram of the structure of a processing device provided in an embodiment of the present application;

[0024] Fig.13 A hardware entity schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions of the present application are further elaborated in detail below in conjunction with the drawings and embodiments. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0026] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict. The terms "first / second / third" involved are merely used to distinguish similar objects and do not represent a specific order for the objects. It is understood that "first / second / third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0027] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing this application and are not intended to limit this application.

[0028] In the scenario of voice interaction, due to the influence of environmental noise and signals, the original voice signal input by the user may be missing or of poor quality. The solution adopted in the relevant technology is to directly use the big data model to compensate for the missing and poor quality voice signals in the original voice signal. However, there is still the problem that the compensated voice is contrary to the user's original intention, thereby reducing the user experience.

[0029] In view of the above technical problems, an embodiment of the present application provides a processing method, which can be executed by a processor of an electronic device. The electronic device may refer to a server, a laptop, a tablet computer, a desktop computer, a smart TV, a set-top box, a mobile device (such as a mobile phone, a portable video player, a personal digital assistant, a dedicated messaging device, a portable gaming device) and other devices with data processing capabilities.

[0030] Figure 1 A schematic diagram of an implementation flow of a processing method provided in an embodiment of the present application is shown in FIG. Figure 1As shown, the method includes the following steps S101 to S103, combining Figure 1 The steps shown are explained.

[0031] Step S101: Obtain first voice data.

[0032] In some embodiments, the first voice data refers to unprocessed voice data; it may be voice data input by a user through voice in a voice interaction scenario, it may be voice data saved after being recorded by a recording device, it may be voice data downloaded from the Internet, etc., and the embodiments of the present application do not make specific limitations on this.

[0033] Step S102: Analyze the first voice data to obtain audio parameters of the first voice data.

[0034] The audio parameters are used to characterize the performance of the sound signal of the first voice data at each time point in each frequency band.

[0035] In some embodiments, the audio parameters may include energy proportion, amplitude, phase, etc.

[0036] In some embodiments, it is understood that the first voice data is first preprocessed, including sampling and quantizing the first voice data; it is understood that the first voice data is converted into a digital signal, and the first voice data is sampled based on a preset sampling rate to obtain n sampling points in the time domain. Secondly, the time domain characteristics and frequency domain characteristics of each sampling point after sampling are extracted, including calculating the short-time energy, short-time average zero-crossing rate and other parameters of the voice data at each sampling point to characterize the time domain variation characteristics of the voice data; the voice data at each sampling point is Fourier transformed or fast Fourier transformed to obtain spectrum information to characterize the performance of the signal in the frequency domain; the audio parameters of the first voice data are determined based on the time domain characteristics and frequency domain characteristics of each sampling point; finally, the spectrum of the first voice data is divided into multiple frequency bands, so as to determine the performance of the sound signal at each time point in each frequency band based on the audio parameters.

[0037] Exemplarily, the audio parameters take energy proportion, amplitude, and multiple frequency bands as low frequency band, medium frequency band, and high frequency band as examples to illustrate the embodiments of the present application. Among them, energy proportion refers to the proportion of energy in a certain frequency band to the total energy of the entire audio signal. The size of this proportion can reflect the richness of the audio signal in the low frequency band; the size of the average amplitude represents the strength of the sound signal in different frequency bands. For example, the low frequency band is 20Hz-500Hz, the medium frequency band is 500Hz-2kHz, and the high frequency band is 2kHz-20kHz. The energy proportion of the voice data at the target time point in each frequency band is 30%, and the average amplitude is 0.5V; it can be understood that the energy proportion of 45% and the average amplitude of 0.5V indicate that the voice data at the target time point has a richer energy distribution in the low frequency band, and the energy proportion of 45% in the medium frequency band and the high frequency band target time point The voice data is sharp and harsh.

[0038] Step S103: If the audio parameter indicates that the first speech data does not meet the quality condition, the first speech data is processed based on a target model to generate second speech data.

[0039] The audio parameters of the second voice data meet the quality condition.

[0040] The second voice data is used for output, and the sound generated by the output of the second voice data is continuous and clear.

[0041] In some embodiments, whether the audio parameter satisfies the quality condition represents whether the first voice data lacks the user's voice information at the said time point.

[0042] In some embodiments, whether the first voice data meets the quality condition can be determined based on whether the voice data at each time point meets the quality condition; that is, whether the first voice data lacks the user's voice information can be determined based on whether the voice data at each time point lacks the user's voice information.

[0043] In some embodiments, the target model is a trained audio repair model, which can be trained in the following manner: Obtain a plurality of training samples, each training sample including a plurality of speech data satisfying quality conditions, one speech data not satisfying quality conditions, and the semantic content of each speech data satisfying quality conditions and the speech data not satisfying quality conditions, as well as the target speech data corresponding to the training sample and the corresponding semantic content; Input the plurality of speech data satisfying quality conditions, one target speech data not satisfying quality conditions, and the semantic content of each speech data satisfying quality conditions into the audio repair model to be trained, obtain the repaired speech data for the speech data not satisfying quality conditions, and adjust the model parameters of the audio repair model to be trained based on the loss value between the semantic content of the repaired speech data and the semantic content of the target speech data until the convergence condition is satisfied, and output the trained audio repair model.

[0044] In some embodiments, the target model may include a recurrent neural network model, a convolutional neural network, a long short-term memory network model, a gated recurrent unit model, a Transformer and its self-attention mechanism model, etc.

[0045] In some embodiments, it can be understood that the first speech data not satisfying the conditions is input into the trained audio repair model to obtain continuous and clear second speech data. Exemplarily, the text content corresponding to the first speech data is "How about today *", where the speech data corresponding to the missing "qi" in "weather" results in the discontinuity of the first speech data. Then, the first speech data is input into the trained audio repair model to obtain continuous and complete second speech data, and the text content corresponding to the second speech data is "How about today's weather", and the second speech data is used as the output speech of the user.

[0046] In some embodiments, if the audio parameters indicate that the first speech data does not satisfy the quality conditions, the first speech data can also be denoised, and the denoised first speech data is processed based on the target model to generate the second speech data.

[0047] In some embodiments, if the audio parameters indicate that the first speech data does not satisfy the quality conditions, the first speech data is processed based on the target model to generate the second speech data, the second speech data is denoised, and the denoised second speech data is used as the output speech data.

[0048] In an embodiment of the present application, time domain and frequency domain analysis is performed on the voice data input by the user to determine whether the input voice data meets the conditions, that is, whether there are any missing or unclear voice data. If the input voice data does not meet the conditions, the input voice data is repaired by a trained audio repair model to obtain continuous and clear voice data, and the continuous and clear voice data is used as the user's output voice data, thereby improving the user experience.

[0049] Figure 2 The present invention provides a processing method for implementing a process flow diagram, which can be executed by a processor of an electronic device. Figure 1 , Figure 1 Step S102 in can be updated to step S201 to step S203, combining Figure 2 The steps shown are explained.

[0050] Step S201: Sample the first voice data from multiple time points to obtain voice data corresponding to each of the time points.

[0051] In some embodiments, a sampling rate for sampling the first voice data is determined; a time interval between each sampling point is determined based on the determined sampling rate; a sampling time point is set based on the time interval; and sampling is performed based on the set sampling time point to obtain voice data corresponding to each sampling time point.

[0052] Exemplary sampling rates may include 8kHz, 16kHz, 44.1kHz or 48kHz, etc.; determining the sampling time point, it can be understood that the time interval between each sampling point is determined based on the determined sampling rate, starting from the starting point of the first voice data, and setting the sampling time point according to the determined time interval. For example, if the sampling rate is 16kHz, the time interval between each sampling time point is 1 / 16000 seconds, that is, about 0.0625 milliseconds; sampling based on the set sampling time point, it can be understood that at each set sampling time point, a sample is collected from the first voice data to obtain voice data corresponding to each sampling time point; for example, if the first voice data is 6.5 milliseconds, the 0th millisecond of the first voice data is used as the first sampling time point for sampling, and the time interval is 0.0625 milliseconds, then 101 samples are obtained, that is, voice data corresponding to 101 time points are obtained.

[0053] Step S202: Perform spectrum analysis on the speech data corresponding to each of the time points to determine the loss parameter of each of the time points.

[0054] The loss parameter at each time point represents whether the voice data corresponding to the time point lacks the user's voice information.

[0055] In some embodiments, the loss parameter at each of the time points may also represent whether the speech data corresponding to the time point is clear.

[0056] In some embodiments, a spectral analysis is performed on the speech data corresponding to each of the time points to obtain the time domain characteristics and frequency domain characteristics of the speech data at each time point, and a loss parameter for each time point is obtained based on the time domain characteristics and frequency domain characteristics of the speech data at each time point; illustratively, the loss parameter and the energy ratio are taken as an example; Fourier transform or fast Fourier transform is performed on the speech data at each sampling time point to obtain the spectral characteristics of the speech data at each sampling time point that change with time; based on the spectral characteristics of the speech data at each sampling time point, its energy at each frequency component is calculated.

[0057] Step S203: Determine the audio parameters of the first speech data based on the loss parameters at each of the time points.

[0058] In some embodiments, whether the voice data at each time point lacks the user's voice information is determined based on the loss parameter of each time point; and the audio parameters of the first voice data are determined based on the ratio of the number of time points lacking the user's voice information.

[0059] Exemplarily, the loss parameter of each time point is taken as an example of energy ratio to illustrate the embodiment of the present application. It is judged whether the energy of each sampling time point on each frequency component reaches the preset energy threshold. If the energy on a certain frequency component reaches the energy threshold, it indicates that the user voice exists and is not missing on the frequency component at the sampling time point; otherwise, it is determined that the voice data at the sampling time point lacks the user's voice information; if the number of time points less than the energy threshold is greater than the first preset number, an audio parameter is generated indicating that the first voice data does not meet the conditions; that is, the proportion of missing user voice data in the first voice data is greater than the preset proportion; if the number of time points less than the energy threshold is less than the first preset number, an audio parameter is generated indicating that the first voice data meets the conditions; if the number of time points less than the energy threshold is greater than the second preset number, an audio parameter is generated indicating that the voice data needs to be re-recorded or acquired. For example, if the number of time points is 100, of which 30 are time points where the user's voice information is missing, and the first preset number is 20, then an audio parameter is generated that represents that the first voice data does not meet the conditions; if the number of time points where the user's voice information is missing is 5, then an audio parameter is generated that represents that the first voice data meets the conditions; if the second preset number is 40, and the number of time points where the user's voice information is missing is 50, then an audio parameter is generated that represents that the voice data needs to be re-recorded or acquired.

[0060] In an embodiment of the present application, the first voice data is sampled from multiple time points to obtain voice data at each time point, and spectral analysis is performed on the voice data at each time point to determine whether the voice data at each time point lacks the user's voice information, thereby determining the audio parameters of the first voice data based on whether the voice data at each time point lacks the user's voice information to improve the accuracy of repairing the first voice data.

[0061] Figure 3 The present invention provides a processing method for implementing a process flow diagram, which can be executed by a processor of an electronic device. Figure 2 , Figure 2 Step S203 in can be updated to step S301 and step S302, combining Figure 3 The steps shown are explained.

[0062] Step S301, counting the loss parameters at each time point to obtain statistical results.

[0063] The statistical result represents the ratio of the number of time points at which the first voice data lacks the user's voice information at the time point to the multiple time points.

[0064] In some embodiments, based on whether the energy of each frequency component at each sampling time point reaches a preset energy threshold, it is determined whether the loss parameter of each time point is missing the user's voice information, and based on whether the loss parameter of each time point is missing the user's voice information, a histogram or line graph is constructed to obtain the number of time points where the user's voice information is missing and the number of time points where the user's voice information is not missing.

[0065] Step S302: Generate audio parameters of the first speech data based on the statistical results.

[0066] Based on the number of time points at which the user's voice information is missing and the number threshold or the ratio of the number of time points at which the user's voice information is missing to the total number, the audio parameters of the first voice data are determined. Wherein, when the number of time points at which the user's voice information is missing reaches the first number threshold, an audio parameter is generated indicating that the first voice data does not meet the condition; when the number of time points at which the user's voice information is missing is less than the number threshold, an audio parameter is generated indicating that the first voice data meets the condition.

[0067] Exemplarily, the quantity threshold is 30, and the number of time points at which the user's voice information is missing is 30, then an audio parameter is generated that represents that the first voice data does not meet the conditions; if the number of time points at which the user's voice information is missing is 10, then an audio parameter is generated that represents that the first voice data meets the conditions.

[0068] In an embodiment of the present application, whether the energy of each frequency component at each sampling time point reaches a preset energy threshold is used to determine whether the loss parameter at each time point is missing the user's voice information, the number of time points at which the user's voice information is missing is counted, and audio parameters of the first voice data are generated to characterize whether the conditions are met, thereby improving the accuracy of the repaired voice data and improving the user experience.

[0069] Figure 4 The present invention provides a processing method for implementing a process flow diagram, which can be executed by a processor of an electronic device. Figure 3 ; Figure 3 Step 302 in the above example can also be updated to step S401, step S402 or step 403. Figure 4 The steps shown are explained.

[0070] Step S401: Generate a first audio parameter when the statistical result indicates that the proportion of the number of time points in the multiple time points is less than or equal to a first proportion threshold; the first audio parameter indicates that the first speech data meets a quality condition.

[0071] In some embodiments, it can be understood that when the number of time points where the user's voice information is missing is less than a first quantity threshold or the ratio of the number of time points where the user's voice information is missing is less than a first ratio threshold, an audio parameter characterizing that the first voice data meets the condition is generated.

[0072] Exemplarily, the number of time points where the user's voice information is missing is 5, and if the first quantity threshold is 30; or, if the first ratio threshold is 30%, the total number of time points is 100, then an audio parameter is generated that characterizes that the first voice data meets the conditions.

[0073] Step S402: When the proportion of the number of time points represented by the statistical result among the multiple time points is greater than a first proportion threshold and less than a second proportion threshold, generate a second audio parameter; the second audio parameter represents that the first voice data does not meet the quality condition.

[0074] In some embodiments, it can be understood that when the number of time points where the user's voice information is missing is greater than the first quantity threshold and less than the second quantity threshold or the ratio of the number of time points where the user's voice information is missing is less than the first ratio threshold and greater than the second ratio threshold, an audio parameter characterizing that the first voice data does not meet the condition is generated.

[0075] Exemplarily, the number of time points in which the user's voice information is missing is 30. If the first quantity threshold is 30 and the second quantity threshold is 50; or, if the first ratio threshold is 30% and the second ratio threshold is 50%, and the total number of time points is 100, then audio parameters are generated that characterize that the first voice data meets the conditions.

[0076] Step S403: generating a third audio parameter when the statistical result indicates that the proportion of the number of time points in the multiple time points is greater than or equal to a second proportion threshold; the third audio parameter indicates that the first voice data needs to be re-recorded.

[0077] In some embodiments, it can be understood that when the number of time points where the user's voice information is missing is greater than a second quantity threshold or the ratio of the number of time points where the user's voice information is missing is greater than a second ratio threshold, audio parameters representing the first voice data that needs to be re-recorded are generated.

[0078] Exemplarily, the number of time points where the user's voice information is missing is 50, the second number threshold is 40, or the second ratio threshold is 50%, and the total number of time points is 100, then the audio parameters of the first voice data need to be re-recorded.

[0079] In an embodiment of the present application, whether the first voice data meets the conditions is determined by comparing the ratio of the number of time points at which the user's voice information is missing with a ratio threshold, thereby determining whether the first voice data needs to be repaired, thereby improving the user experience.

[0080] Figure 5 The present invention provides a processing method for implementing a process flow diagram, which can be executed by a processor of an electronic device. Figure 1 , Figure 1 Step S103 in can be updated to step S501 and step S502, combining Figure 5 The steps shown are explained.

[0081] Step S501: Acquire speech data of at least one adjacent time point of the speech data to be repaired in the first speech data.

[0082] The voice data to be repaired is voice data in which the voice information of the missing user is missing.

[0083] In some embodiments, it can be understood that the first voice data is sampled from multiple time points to obtain voice data at each time point; the voice data at each time point is spectrally analyzed to obtain the energy of the voice data at each time point in each frequency band; to determine whether the voice data at each time point is missing the user's voice information, the voice data corresponding to at least one time point where the user's voice information is missing is used as the voice data to be repaired in the first voice data; and the voice data corresponding to at least one time point where the user's voice information is not missing that is immediately before and after the time point where the at least one user's voice information is missing is obtained, wherein the at least one time point where the user's voice information is not missing is a continuous time point.

[0084] Exemplarily, the first voice data is set to 60 sampling time points, where the interval time between each sampling time point is 0.01 milliseconds, then the sampling time of the first sampling time point is 0th second, the sampling time of the second sampling time point is 0.001 second, and so on; the time points where the user's voice information is missing are 0.034th second to 0.040th second, including 7 sampling time points, then the time points where the user's voice information is not missing before and after 0.034th second to 0.040th second are 0th second to 0.033th second and 0.040th second to 0.059th second; the voice data corresponding to 0th second to 0.033th second and 0.040th second to 0.059th second are obtained.

[0085] Step S502: repair the speech data to be repaired based on the speech data at at least one adjacent time point using the target model to obtain second speech data.

[0086] In some embodiments, semantic content corresponding to speech data of at least one adjacent time point is obtained, and the semantic content corresponding to the first speech data and the speech data of at least one adjacent time point is input into a target model, and speech data whose semantic content similarity with the speech data of at least one adjacent time point reaches a similarity threshold is obtained through the target model, and the speech data to be repaired in the first speech data is repaired based on the speech data that reaches the similarity threshold to obtain the repaired speech data to be repaired; the repaired speech data to be repaired is spliced ​​with the speech data of at least one adjacent time point to obtain the second speech data.

[0087] Exemplarily, if the first speech data is "What's the weather like today*", then the speech data to be repaired is "*", and the speech data at at least one adjacent time point is "What's the weather like today" and "today". Through the target model, the speech data with a similarity to the speech data at at least one adjacent time point reaching the similarity threshold is "What's the weather like today", thus, the speech data to be repaired is repaired through the target model, and the repaired speech data to be repaired is "weather", then the speech data "What's the weather like today" and "today" at at least one adjacent time point are concatenated with the repaired speech data "weather" to obtain the second speech data "What's the weather like today".

[0088] In the embodiments of the present application, through the target model, the speech data to be repaired is repaired based on the speech data at at least one adjacent time point of the speech data to be repaired, and the repaired speech data is obtained. The repaired speech data and the speech data at at least one adjacent time point are concatenated to obtain continuous and clear second speech data, thus, the continuous and clear second speech data is used as the output to improve the user experience.

[0089] Figure 6 Schematic diagram of the implementation process of a processing method provided by the embodiments of the present application. This method can be executed by the processor of an electronic device. Based on Figure 1 , Figure 1 Step S103 in can be updated to steps S601 to S603, and will be described in combination with Figure 6 the steps shown.

[0090] Step S601: Obtain the first text data of the first speech data.

[0091] In some embodiments, the first speech data is converted into the corresponding first text data through an existing translation tool or conversion tool; exemplarily, the first text data is "What's the weather like today*".

[0092] Step S602: Based on the target model, perform correction processing on the first text data to obtain the second text data.

[0093] In some embodiments, the target model refers to a trained text repair model, which can be trained in the following manner: obtaining multiple training samples, each training sample including multiple continuous and semantically complete text data and one missing and semantically incomplete text data; and target text data corresponding to the training samples; inputting multiple continuous and semantically complete text data and one missing and semantically incomplete text data into the text repair model to be trained to obtain predicted text data for the missing and semantically incomplete text data, adjusting the model parameters of the text repair model to be trained based on the loss value between the predicted text data and the target text data until the convergence condition is met, and outputting the trained text repair model.

[0094] In some embodiments, through the target model, reference text data that reaches a similarity threshold with the first text data is obtained, and the second text data is obtained by repairing the first text data through the reference text data; exemplarily, the first text data is "Do you feel unwell?", and the reference text data obtained based on the target model is "Does your body feel unwell?", so that "Do you feel unwell?" is repaired based on "Does your body feel unwell?" to obtain the second text data "Do you feel unwell?"

[0095] Step S603: Obtain the second voice data based on the second text data.

[0096] In some embodiments, the second text data is converted into second voice data through existing text tools, conversion tools, etc.

[0097] In an embodiment of the present application, in an embodiment of the present application, the text data of the first voice data is repaired by the target model to obtain semantically complete second text data, and the second text data is converted into clear and continuous second voice data, so that the continuous and clear second voice data is used as output to improve the user experience.

[0098] Figure 7 1 is a schematic diagram of an implementation flow of a processing method provided in an embodiment of the present application, and the method can be executed by a processor of an electronic device. Figure 6 , Figure 6 The second speech data includes the compensated text modified by the correction process on the first text data and the unchanged original text; after step S603, step S701 is also included to combine Figure 7 The steps shown are explained.

[0099] Step S701: display the second text data.

[0100] The display effect of the compensated text is different from that of the original text.

[0101] In some embodiments, in response to generating the second text data, the compensated text and the original text in the second text data are displayed with different display effects; illustratively, the compensated text and the original text are displayed in different colors, so as to prompt the user which text is the text generated by the target model. illustratively, the compensated text is displayed in red font and the original text is displayed in black font.

[0102] In some embodiments, the compensation text and the original text may be displayed in different display states, for example, the compensation text may be displayed in a flashing manner and the original text may be displayed normally; or the font of the compensation text may be larger than that of the original text.

[0103] In some embodiments, after obtaining the second voice data in the above step S502 or step S603, the method further includes:

[0104] Display the prompt information of the second voice data.

[0105] The prompt information is used to indicate that the second voice data is repaired voice data.

[0106] In some embodiments, the prompt information may be text, pattern, voice, etc.; if the prompt information is text or pattern, the prompt information will be displayed in a blank position near the second voice data; wherein the prompt information and the second voice data may also be connected by indication information, and the indication information may be an arrow, etc.

[0107] In an embodiment of the present application, prompt information of the second voice data is displayed to prompt the user that the second voice data is repaired voice data, thereby improving the user experience.

[0108] Figure 8 is a schematic diagram of an implementation flow of a processing method provided in an embodiment of the present application, which can be executed by a processor of an electronic device. The process includes steps S801 to S803, which will be combined with Figure 8 The steps shown are explained.

[0109] Step S801: Acquire second text data of second voice data.

[0110] In some embodiments, the second voice data is converted into corresponding second text data through an existing translation tool or conversion tool; illustratively, the second text data is “How is the weather today?”.

[0111] Step S802: display the second text data.

[0112] In some embodiments, in response to obtaining the second text data, the second text data is displayed in a blank area near the second voice data, wherein the second text data and the second voice data are connected by indication information. For example, the second text data is displayed in a blank area below the second voice data, and the second text data and the second voice data are connected by an arrow to indicate the association relationship between the second text data and the second voice data.

[0113] Step S803: receiving an interactive operation for the second text data.

[0114] In some embodiments, the interaction operation may be input by the user through click input, voice input, touch input, etc. Exemplarily, an interaction operation of the user inputted by at least one of click input, voice input, touch input, etc. for the second text data is received.

[0115] Step S804: In response to the interactive operation, generate third voice data corresponding to the third text data.

[0116] The third text data is generated based on the interactive operation and the second text data.

[0117] In some embodiments, in response to an interactive operation for inputting at least one of click input, voice input, touch input, etc. into the second text data, third text data is generated; and the third text data is converted into third voice data through existing text tools, audio editing tools, dubbing tools, etc.

[0118] In an embodiment of the present application, after the first voice data is repaired by the target model, the second voice data is obtained, and the third voice data is obtained through the user's interactive operation on the second text data of the second voice data, and the third voice data determined by the user is output, thereby improving the user experience.

[0119] Fig. 9 It is a schematic diagram of an implementation flow of a processing method provided in an embodiment of the present application, and the method can be executed by a processor of an electronic device. Figure 8 The interactive operation includes at least one of the following: a confirmation operation and a text correction operation; step S804 includes step S901 or step S902, which is combined with Fig. 9 The steps shown are explained.

[0120] Step S901: In response to the confirmation operation, use the second voice data as the third voice data.

[0121] In some embodiments, the confirmation operation indicates that the user considers the second voice data to be voice data that is identical to the information that the user wants to express.

[0122] In some embodiments, in response to a user's confirmation operation on the second text data through one of click input, voice input, touch input, etc., the second voice data corresponding to the second text data is used as third voice data with the same meaning as the user wants to express.

[0123] Step S902: In response to the text correction operation, perform text correction on the second text data to obtain the third text data, and generate a third voice signal corresponding to the third text data.

[0124] In some embodiments, the text correction operation may be input by the user through input methods such as click input, keyboard input, voice input, touch input, etc.

[0125] In some embodiments, the text correction operation indicates that the second voice data is different from the information the user wants to express.

[0126] In some embodiments, when the second text content corresponding to the second voice data is different from the information the user wants to express, the first text data is repaired based on a text correction operation to obtain third text data, and the third text data is converted into third voice data.

[0127] Exemplarily, the second text data is "Did you take your medicine today?", and the message the user wants to express is "Have you eaten today?", so the second text data is different from the information the user wants to express. Based on the user's operation of deleting "medicine" by clicking on the second text data and inputting "meal" through the keyboard, the third text data is obtained, and the third text data is converted into third voice data.

[0128] In an embodiment of the present application, after the first voice data is repaired through the target model, the second voice data is obtained. The second text data corresponding to the second voice data is displayed to allow the user to determine whether the repaired data is accurate. If it is inaccurate, the second text data is corrected through the user's text correction operation to obtain the third text data that is the same as the information the user wants to express. The third text data is converted into third voice data as output voice data, thereby improving the user experience.

[0129] An exemplary application of a processing method provided in an embodiment of the present application in an actual scenario is described below.

[0130] At present, voice interaction has become an increasingly important part of people's lives. Voice interaction plays an increasingly important role in daily life, business communication, education and training, health care and other fields. In the scene of voice interaction, sometimes the voice quality is poor due to the influence of the environment and signals. At present, mainstream intelligent platforms are equipped with artificial intelligence algorithms to reduce the noise of voice signals and restore the missing signals through big data models to make the data coherent and continuous. However, if the intermittent words happen to be keywords in the sentence, individual words will affect the meaning of the entire sentence, such as "yes" and "no". When the AI ​​algorithm compensates, it compensates according to the probability of the big data model, which may sometimes be completely contrary to the speaker's own meaning, resulting in misunderstandings between the two parties.

[0131] In response to the above technical problems, the present application first determines the voice signal quality of the voice sentence and determines whether there is a missing voice signal. If there is, the missing voice signal is repaired through the trained model, or the voice signal is converted into text, and the missing voice part generated by the trained model is displayed in red font to allow the user to confirm whether it is the meaning of the original sentence.

[0132] Fig.10 The following is a schematic diagram of a processing method provided in an embodiment of the present application. The method can be executed by a processor of an electronic device. The process includes steps S1001 to S1007. Fig.10 The steps shown are explained.

[0133] Step S1001: Detect that the user is sending a voice message.

[0134] Step S1002: Determine the quality of the voice signal.

[0135] Step S1003: If the voice signal loss is greater than or equal to 30%, it is recommended that the user re-record.

[0136] Step S1004: If the voice signal is not missing, send it directly.

[0137] Step S1005: If the speech signal loss is less than 30%, the signal is repaired and compensated through an artificial intelligence model.

[0138] Step S1006: convert the voice signal into text information, and correct the text information corresponding to the missing part of the voice signal through the artificial intelligence model.

[0139] Step S1007: If the user confirms that there is no problem, it is sent normally. If there is a problem, the user modifies the content, and the modified content is converted into voice and sent.

[0140] If the missing voice signal is detected to be greater than 30%, the user will be prompted and advised to re-record. If there is no missing or discontinuous signal, the signal is sent normally. If the missing voice signal is less than 30%, the AI ​​model is first used to repair and compensate the signal, and the voice signal is converted into text. The text compensated / corrected by the AI ​​model is marked in red font, allowing the user to check whether it is the original meaning of the user's sentence. The user can modify the marked font, and after the modification, the model will re-repair the voice according to the modified font and send it.

[0141] Fig.11 The following is a schematic diagram of a processing method provided in an embodiment of the present application. The method can be executed by a processor of an electronic device. The process includes steps S1101 to S1105. Fig.11 The steps shown are explained

[0142] Step S1101: input feature preparation.

[0143] In some embodiments, the impaired speech signal is converted into a feature form that the model can process; including: Audio reading: Use an audio processing library (such as librosa or pydub) to read the audio file. Make sure to use the same sampling rate when loading as when training. Feature extraction: Apply short-time Fourier transform (STFT) or Mel spectrum transform to obtain a time-frequency representation. Gain normalization or other preprocessing is performed on the spectrum to suit the requirements of the model.

[0144] Step S1102: Input features into the model.

[0145] In some embodiments, the prepared features are input into the trained AI model for repair; including: batch processing, batch processing of feature data, adapting the input shape of the model. Ensure that the input dimension is consistent with the input dimension expected by the model; obtain context information: context frame (for example, using the information of the previous and next frames in the LSTM model) to improve the repair effect.

[0146] Step S1103: model reasoning.

[0147] In some embodiments, audio restoration using a trained model includes: passing data: inputting the prepared features into the model (calling model.predict() or related interfaces). Generating output: obtaining the model output features, which are usually the restored spectrogram or other feature representations. The shape and format of the output should be consistent with the input features for further processing.

[0148] Step S1104: feature post-processing.

[0149] In some embodiments, the repaired features output by the model are reversely processed to restore the time domain signal; including: inverse transformation, using the inverse short-time Fourier transform (Inverse STFT) to convert the repaired spectrogram back to a time domain waveform, using the Mel spectrum to apply an appropriate vocoder (Griffin-Lim algorithm, etc.) to reconstruct the audio; splicing audio segments, the input audio is sliced, and the generated audio segments need to be merged into a complete audio.

[0150] Step S1105: post-process the merged audio.

[0151] In some embodiments, the merged audio is denoised to improve the auditory quality of the generated audio and make it closer to natural speech.

[0152] In some embodiments, in social software, when the host detects that the user is sending a voice message, it first determines whether the original data of the voice signal (not processed by the AI ​​noise reduction algorithm) is missing or discontinuous. If there is a missing or discontinuity, the voice signal is repaired through the trained model to obtain a clear and complete voice signal, thereby improving the user experience.

[0153] Based on the foregoing embodiments, the embodiments of the present application provide a processing device, which includes the units included and the modules included in the units, and can be implemented by a processor in an electronic device; of course, it can also be implemented by a specific logic circuit; in the implementation process, the processor can be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP) or a field programmable gate array (FPGA), etc.

[0154] Fig.12 A schematic diagram of the structure of a processing device provided in an embodiment of the present application is shown in FIG. Fig.12As shown, the processing device 1200 includes: an acquisition module 1201, an analysis module 1202, and a generation module 1203, wherein: the acquisition module 1201 is used to obtain first voice data; the analysis module 1202 is used to analyze the first voice data to obtain audio parameters of the first voice data, and the audio parameters are used to characterize the performance of the first voice data in each frequency band on the sound signal at each time point; the generation module 1203 is used to process the first voice data based on the target model to generate second voice data if the audio parameters characterize that the first voice data does not meet the quality condition; the audio parameters of the second voice data meet the quality condition; wherein the second voice data is used for output, and the sound generated by the second voice data output is continuous and clear.

[0155] In some embodiments, the analysis module 1202 is also used to sample the first voice data from multiple time points to obtain voice data corresponding to each of the time points; perform spectral analysis on the voice data corresponding to each of the time points to determine the loss parameters of each of the time points; the loss parameters of each of the time points characterize whether the voice data corresponding to the time point lacks the user's voice information; and determine the audio parameters of the first voice data based on the loss parameters of each of the time points.

[0156] In some embodiments, the analysis module 1202 is also used to count the loss parameters of each of the time points to obtain statistical results; the statistical results represent the proportion of the number of time points at which the first voice data lacks the user's voice information at the time points among the multiple time points; based on the statistical results, the audio parameters of the first voice data are generated.

[0157] In some embodiments, the analysis module 1202 is also used to generate a first audio parameter when the proportion of the number of time points represented by the statistical result among the multiple time points is less than or equal to a first proportion threshold; the first audio parameter represents that the first voice data meets the quality condition; generate a second audio parameter when the proportion of the number of time points represented by the statistical result among the multiple time points is greater than the first proportion threshold and less than the second proportion threshold; the second audio parameter represents that the first voice data does not meet the quality condition; generate a third audio parameter when the proportion of the number of time points represented by the statistical result among the multiple time points is greater than or equal to the second proportion threshold; the third audio parameter represents that the first voice data needs to be re-recorded.

[0158] In some embodiments, the generation module 1203 is also used to obtain speech data of at least one adjacent time point of the speech data to be repaired in the first speech data; the speech data to be repaired is speech data of the missing user's voice information; through the target model, the speech data to be repaired is repaired based on the speech data of at least one adjacent time point to obtain second speech data.

[0159] In some embodiments, the generation module 1203 is further used to obtain first text data of the first voice data; modify the first text data based on a target model to obtain second text data; and obtain the second voice data based on the second text data.

[0160] In some embodiments, the second text data includes compensatory text changed by the correction processing on the first text data and the unchanged original text; the processing device 1200 also includes a display module (not shown in the figure) for displaying the second text data; the display effect of the compensatory text is different from that of the original text.

[0161] In some embodiments, the display module (not shown in the figure) is also used to display prompt information of the second voice data; the prompt information is used to indicate that the second voice data is repaired voice data.

[0162] In some embodiments, the generation module 1203 is also used to obtain second text data of second voice data; display the second text data; receive an interactive operation for the second text data; and generate third voice data corresponding to third text data in response to the interactive operation; the third text data is generated based on the interactive operation and the second text data.

[0163] In some embodiments, the interactive operation includes at least one of the following: a confirmation operation and a text correction operation; the generation module 1203 is also used to respond to the confirmation operation and use the second voice data as the third voice data; respond to the text correction operation, perform text correction on the second text data to obtain the third text data, and generate a third voice signal corresponding to the third text data.

[0164] The description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. In some embodiments, the functions or modules included in the device provided in the embodiment of the present application can be used to execute the method described in the above method embodiment. For technical details not disclosed in the device embodiment of the present application, please refer to the description of the method embodiment of the present application for understanding.

[0165] It should be noted that in the embodiment of the present application, if the above method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, which is stored in a storage medium, including a number of instructions to enable an electronic device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a disk or an optical disk. In this way, the embodiment of the present application is not limited to any specific hardware, software or firmware, or any combination of hardware, software, and firmware.

[0166] An embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, some or all of the steps in the above method are implemented.

[0167] The embodiment of the present application provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, some or all of the steps in the above method are implemented. The computer-readable storage medium can be transient or non-transient.

[0168] An embodiment of the present application provides a computer program, including a computer-readable code. When the computer-readable code is run in an electronic device, a processor in the electronic device executes some or all of the steps for implementing the above method.

[0169] The embodiment of the present application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and when the computer program is read and executed by a computer, some or all of the steps in the above method are implemented. The computer program product can be implemented specifically by hardware, software or a combination thereof. In some embodiments, the computer program product is specifically embodied as a computer storage medium, and in other embodiments, the computer program product is specifically embodied as a software product, such as a software development kit (SDK) and the like.

[0170] It should be noted here that the description of the various embodiments above tends to emphasize the differences between the various embodiments, and the same or similar aspects can be referenced to each other. The description of the above device, storage medium, computer program and computer program product embodiments is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the embodiments of the device, storage medium, computer program and computer program product of this application, please refer to the description of the method embodiment of this application for understanding.

[0171] Fig.13 A hardware entity diagram of an electronic device provided in an embodiment of the present application, such as Fig.13 As shown, the hardware entity of the electronic device 1300 includes: a processor 1301 and a memory 1302, wherein the memory 1302 stores a computer program that can be run on the processor 1301, and the processor 1301 implements the steps in the method of any of the above embodiments when executing the program.

[0172] The memory 1302 stores computer programs that can be run on the processor. The memory 1302 is configured to store instructions and applications executable by the processor 1301. It can also cache data to be processed or processed by the processor 1301 and various modules in the electronic device 1300 (for example, image data, voice data, voice communication data, and video communication data). This can be achieved through flash memory (FLASH) or random access memory (Random Access Memory, RAM).

[0173] When the processor 1301 executes the program, the steps of any of the above methods are implemented. The processor 1301 generally controls the overall operation of the electronic device 1300.

[0174] An embodiment of the present application provides a computer storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the steps of the method of any of the above embodiments.

[0175] It should be noted here that the description of the above storage medium and device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0176] The processor may be at least one of an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a Programmable Logic Device (PLD), a Field Programmable Gate Array (FPGA), a Central Processing Unit (CPU), a controller, a microcontroller, and a microprocessor. It is understandable that the electronic device that implements the functions of the processor may also be other, and the embodiments of the present application are not specifically limited.

[0177] The above-mentioned computer storage medium / memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory (Flash Memory), a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM) and the like; it can also be various terminals including one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.

[0178] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the serial number of each step / process mentioned above does not mean the order of execution, and the execution order of each step / process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application. The serial numbers of the embodiments of the present application mentioned above are for description only and do not represent the advantages and disadvantages of the embodiments.

[0179] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0180] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0181] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0182] In addition, all functional units in the embodiments of the present application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the above integrated unit can be implemented in the form of hardware or in the form of hardware plus software functional units. A person of ordinary skill in the art can understand that all or part of the steps of the above method embodiments can be completed by hardware related to program instructions, and the above program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above method embodiments are executed; and the above storage medium includes: mobile storage devices, read-only memory (ROM), disks or optical disks, etc. Various media that can store program codes.

[0183] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can essentially or in other words, the part that contributes to the relevant technology can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for an electronic device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0184] The above is only an implementation method of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A processing method comprising: obtaining first voice data; Analyze the first voice data to obtain audio parameters of the first voice data, where the audio parameters are used to characterize the performance of the sound signal of the first voice data at each time point in each frequency band; If the audio parameters characterize that the first voice data does not meet the quality conditions, the first voice data is processed based on the target model to generate second voice data, and the audio parameters of the second voice data meet the quality conditions; wherein the second voice data is used for output, and the sound generated by the second voice data output is continuous and clear.

2. The method according to claim 1, wherein analyzing the first voice data to obtain audio parameters of the first voice data comprises: Sampling the first voice data from multiple time points to obtain voice data corresponding to each of the time points; Performing spectrum analysis on the voice data corresponding to each of the time points to determine the loss parameter of each of the time points; the loss parameter of each of the time points indicates whether the voice data corresponding to the time point lacks the voice information of the user; Based on the loss parameter at each of the time points, an audio parameter of the first speech data is determined.

3. The method according to claim 2, wherein determining the audio parameters of the first speech data based on the loss parameters at each of the time points comprises: Counting the loss parameters at each time point to obtain statistical results; The statistical result represents the ratio of the number of time points at which the first voice data lacks the user's voice information at the time point to the multiple time points; Based on the statistical result, audio parameters of the first speech data are generated.

4. The method according to claim 3, wherein generating the audio parameters of the first speech data based on the statistical result comprises: generating a first audio parameter when the proportion of the number of time points represented by the statistical result among the multiple time points is less than or equal to a first proportion threshold; The first audio parameter indicates that the first voice data meets the quality condition; When the statistical result indicates that the proportion of the number of time points in the multiple time points is greater than a first proportion threshold and less than a second proportion threshold, generating a second audio parameter; the second audio parameter indicates that the first voice data does not meet the quality condition; When the statistical result indicates that the ratio of the number of time points in the multiple time points is greater than or equal to a second ratio threshold, a third audio parameter is generated; the third audio parameter indicates that the first voice data needs to be re-recorded.

5. The method according to claim 1, wherein the processing the first speech data based on the target model to generate the second speech data comprises: Acquire speech data of at least one adjacent time point of the speech data to be repaired in the first speech data; The voice data to be repaired is voice data in which the voice information of the missing user is missing; The speech data to be repaired is repaired based on the speech data of at least one adjacent time point through the target model to obtain second speech data.

6. The method according to claim 1, wherein the processing the first speech data based on the target model to generate the second speech data comprises: Acquire first text data of the first voice data; Performing correction processing on the first text data based on the target model to obtain second text data; Obtaining the second voice data based on the second text data; The second text data includes the compensated text modified by the correction process on the first text data and the original text which is not modified; the method further includes: The second text data is displayed; the display effect of the compensated text is different from the display effect of the original text.

7. The method according to any one of claims 1 to 6, further comprising: Displaying prompt information of the second voice data; The prompt information is used to indicate that the second voice data is restored voice data; The method further comprises: Acquire second text data of the second voice data; displaying the second text data; receiving an interactive operation for the second text data; A third text data is generated in response to the interaction operation according to the corresponding third voice data; the third text data is generated based on the interaction operation and the second text data.

8. The method according to claim 7, wherein the interactive operation comprises at least one of the following: a confirmation operation and a text correction operation; wherein in response to the interactive operation, generating a third voice signal corresponding to the third text data comprises one of the following: In response to the confirmation operation, using the second voice data as the third voice data; In response to the text correction operation, the second text data is subjected to text correction to obtain the third text data, and a third speech signal corresponding to the third text data is generated.

9. A processing device comprising: An obtaining module, used for obtaining first voice data; An analysis module, configured to analyze the first voice data to obtain audio parameters of the first voice data, wherein the audio parameters are used to characterize the performance of the sound signal of the first voice data at each time point in each frequency band; A generation module is used to process the first voice data based on a target model to generate second voice data if the audio parameters characterize that the first voice data does not meet the quality conditions, and the audio parameters of the second voice data meet the quality conditions; wherein the second voice data is used for output, and the sound generated by the second voice data output is continuous and clear.

10. An electronic device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: The processor executes to obtain first voice data; analyzes the first voice data to obtain audio parameters of the first voice data, wherein the audio parameters are used to characterize the performance of the sound signal of the first voice data at each time point in each frequency band; If the audio parameters characterize that the first voice data does not meet the quality conditions, the first voice data is processed based on the target model to generate second voice data, and the audio parameters of the second voice data meet the quality conditions; wherein the second voice data is used for output, and the sound generated by the second voice data output is continuous and clear.

Images