Audio processing method and device

Abstract

The invention provides an audio processing method and device, and the method comprises the steps: obtaining a first sound signal in real time based on collection equipment; converting the first sound signal into a first electric signal; amplifying the first electric signal into a second electric signal based on a first gain, wherein the first gain is a first gain value corresponding to the acquisition equipment; processing the second electric signal based on a target model to obtain an output electric signal, the change of the output electric signal relative to the first electric signal being a second gain value; the output electric signal comprises a third electric signal, and the second gain of the third electric signal relative to the first electric signal is smaller than the first gain value.

Description

Technical Field

[0001] The present application relates to the field of audio technology, and in particular to an audio processing method and device. Background Art

[0002] During the recording process, due to the performance limitations of the recording device, if the input volume is large, the amplitude of the electrical signal of the sound after the recording device amplifies it may exceed the upper limit of the amplitude of the recording device. In this case, the part of the electrical signal that exceeds the upper limit will be directly deleted, resulting in the loss of audio data obtained based on the electrical signal conversion, causing the collected audio data to be distorted. Summary of the invention

[0003] To this end, this application discloses the following technical solutions:

[0004] The first aspect of the present application provides an audio processing method, comprising:

[0005] Obtaining a first sound signal in real time based on an acquisition device;

[0006] converting the first sound signal into a first electrical signal;

[0007] amplifying the first electrical signal into a second electrical signal based on a first gain, wherein the first gain is a first gain value corresponding to the acquisition device;

[0008] The second electrical signal is processed based on the target model to obtain an output electrical signal, wherein the change of the output electrical signal relative to the first electrical signal is a second gain value; the output electrical signal includes a third electrical signal, and the second gain of the third electrical signal relative to the first electrical signal is less than the first gain value.

[0009] Optionally, the processing the second electrical signal based on the target model to obtain an output electrical signal includes:

[0010] Acquiring a plurality of second electrical signals that are continuous in time;

[0011] Processing the plurality of second electrical signals based on the target model to obtain a plurality of output electrical signals, wherein the i-1th, i-th and i+1th of the plurality of output electrical signals are three third electrical signals that are continuous in time;

[0012] The second gain of the i-th third electrical signal relative to the first electrical signal is greater than the second gain of the (i-1)-th third electrical signal relative to the first electrical signal;

[0013] The second gain of the i-th third electrical signal relative to the first electrical signal is greater than the second gain of the (i+1)-th third electrical signal relative to the first electrical signal.

[0014] Optionally, the output electrical signal includes a fourth electrical signal, and a second gain of the fourth electrical signal relative to the first electrical signal is equal to the first gain value.

[0015] Optionally, the processing the second electrical signal based on the target model to obtain an output electrical signal includes:

[0016] Processing N second electrical signals that are consecutive in time based on the target model to determine that the N+1th second electrical signal meets the target condition;

[0017] The N+1th second electrical signal is processed based on the target model to obtain a third electrical signal.

[0018] Optionally, the processing, based on the target model, N second electrical signals that are consecutive in time to determine that the N+1th second electrical signal meets the target condition includes:

[0019] Extracting time domain features and frequency domain features of N temporally continuous second electrical signals;

[0020] The time domain features and frequency domain features of the N second electrical signals are processed based on the target model to determine that the N+1th second electrical signal meets the target condition.

[0021] Optionally, the processing the N+1th second electrical signal based on the target model to obtain a third electrical signal includes:

[0022] The N+1th second electrical signal is processed according to the target model and a plurality of second electrical signals before and after the N+1th second electrical signal to obtain a third electrical signal.

[0023] Optionally, also include:

[0024] Processing a plurality of the second electrical signals before the N+1th second electrical signal based on the target model to obtain a plurality of third electrical signals;

[0025] A plurality of the second electrical signals after the N+1th second electrical signal is processed based on the target model to obtain a plurality of third electrical signals.

[0026] Optionally, the processing, based on the target model, N second electrical signals that are consecutive in time to determine that the N+1th second electrical signal meets the target condition includes:

[0027] Based on the target model, N second electrical signals that are consecutive in time are processed to determine that the actual signal amplitude of the N+1th second electrical signal is greater than the signal amplitude upper limit of the acquisition device.

[0028] Optionally, the target model is constructed based on a sample sound signal and a target sound signal collected from the same sound source, the sample sound signal is collected by the collection device, and the target sound signal is collected by a target device different from the collection device.

[0029] A second aspect of the present application provides an audio processing device, comprising:

[0030] An acquisition unit, configured to acquire a first sound signal in real time based on a collection device;

[0031] A conversion unit, configured to convert the first sound signal into a first electrical signal;

[0032] an amplifying unit, configured to amplify the first electrical signal into a second electrical signal based on a first gain, wherein the first gain is a first gain value corresponding to the acquisition device;

[0033] A processing unit is used to process the second electrical signal based on a target model to obtain an output electrical signal, wherein the change of the output electrical signal relative to the first electrical signal is a second gain value; the output electrical signal includes a third electrical signal, and the second gain of the third electrical signal relative to the first electrical signal is less than the first gain value. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0035] Figure 1 A method flow chart of an audio processing method provided in an embodiment of the present application;

[0036] Figure 2 A signal schematic diagram provided for an embodiment of the present application;

[0037] Figure 3 A schematic diagram of electrical signals input and output of a target model provided in an embodiment of the present application;

[0038] Figure 4 A schematic diagram of electrical signals input and output of another target model provided in an embodiment of the present application;

[0039] Figure 5 A schematic diagram of time domain characteristics and frequency domain characteristics of a second electrical signal provided in an embodiment of the present application;

[0040] Figure 6A schematic diagram of the structure of an audio processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0042] This application embodiment provides an audio processing method, see Figure 1 , the method may include the following steps.

[0043] S101, obtaining a first sound signal in real time based on a collection device.

[0044] The acquisition device may be any recording device, such as a microphone.

[0045] After the acquisition device is started, it can collect multiple first sound signals that are continuous in time and generated by sound sources in the surrounding environment in real time. Each first sound signal can include the sound collected by the acquisition device within a certain period of time.

[0046] For example, a first sound signal may include the sound collected by the collection device within 10 milliseconds (ms). Correspondingly, obtaining the first sound signal in real time based on the collection device is equivalent to the collection device collecting the sound of the sound source in real time, and every 10 ms of sound is regarded as a first sound signal.

[0047] S102: Convert the first sound signal into a first electrical signal.

[0048] The acquisition device may include a sensor that can convert a sound signal into an electrical signal. The diaphragm of the sensor can vibrate when stimulated by the obtained first sound signal. As the diaphragm vibrates, the sensor can output electrical parameters that continuously change over time and characterize the sound intensity at the corresponding moment. These electrical parameters that continuously change over time form a first electrical signal corresponding to the first sound signal.

[0049] Among them, each first sound signal can be converted into a first electrical signal of corresponding duration. For example, each first sound signal includes the sound collected by the collection device within 10ms. Then, the first electrical signal converted from a first sound signal can include electrical parameters characterizing the corresponding sound intensity within 10ms.

[0050] The acquisition device can convert each first sound signal into a first electrical signal in real time. For example, the acquisition device can obtain a first sound signal every 10 ms, and each time a first sound signal is obtained, the first sound signal is converted into a corresponding first electrical signal, thereby obtaining multiple first electrical signals that are continuous in time.

[0051] S103, amplifying the first electrical signal into a second electrical signal based on a first gain, where the first gain is a first gain value corresponding to the acquisition device.

[0052] The above-mentioned first gain can be an analog gain of the acquisition device. The acquisition device can amplify each converted first electrical signal in real time according to the set analog gain, thereby converting the multiple temporally continuous first electrical signals obtained in step S102 into multiple temporally continuous second electrical signals after amplification.

[0053] For the same first electrical signal, the greater the gain value of the analog gain (ie, the first gain value), the greater the signal amplitude of the amplified second electrical signal.

[0054] The first gain value of the acquisition device may be preset by a user using the acquisition device. For example, the acquisition device may include components for setting analog gain, and the user may operate these components to increase or decrease the first gain value of the acquisition device.

[0055] Generally, when executing S103, the first gain value may be a fixed gain value.

[0056] S104, processing the second electrical signal based on the target model to obtain an output electrical signal, wherein a change of the output electrical signal relative to the first electrical signal is a second gain value; the output electrical signal includes a third electrical signal, and a second gain of the third electrical signal relative to the first electrical signal is less than the first gain value.

[0057] Each time the acquisition device converts a second electrical signal, the second electrical signal can be input into the target model. For each second electrical signal, the target model can output an output electrical signal corresponding to the second electrical signal, thereby processing the continuous multiple second electrical signals converted in S103 into continuous multiple output electrical signals.

[0058] Each output electrical signal corresponds to a first electrical signal. For example, after the acquisition device obtains the i-th first electrical signal, it amplifies the i-th first electrical signal into the i-th second electrical signal according to the first gain value, and then processes the i-th second electrical signal with the target model to obtain the i-th third electrical signal. Then, the i-th third electrical signal corresponds to the i-th first electrical signal. i is an arbitrary positive integer.

[0059] The output electrical signal may be regarded as an electrical signal obtained by amplifying the corresponding first electrical signal based on the second gain value.

[0060] Different output electrical signals may correspond to different second gain values. The second gain value of a portion of the output electrical signals relative to the corresponding first electrical signal may be smaller than the first gain value in S103, and such output electrical signals are the aforementioned third electrical signals.

[0061] The target model can determine the processing method for each second electrical signal according to the input continuous plurality of second electrical signals, that is, determine which second electrical signals are processed into third electrical signals.

[0062] The target model may be a pre-built neural network model for processing electrical signals converted from sound signals.

[0063] The beneficial effects of this embodiment are:

[0064] In the related art, some schemes can set a lower analog gain value for the acquisition device to ensure that the amplified electrical signal does not exceed the upper limit of the amplitude allowed by the acquisition device, and then perform digital gain processing on the audio data obtained by converting the electrical signal through digital gain. This method limits the performance of the acquisition device on the one hand, and increases the noise in the audio data. Compared with the solution of the related art, this embodiment can directly adjust the analog gain value of the partially amplified electrical signal through the target model. This processing method can allow the electrical signal that does not exceed the upper limit of the acquisition device after amplification to have a larger analog gain value (i.e., the first gain value), thereby maximizing the performance of the acquisition device and reducing the noise in the audio data. On the other hand, the amplitude of the electrical signal that exceeds the upper limit after amplification can be adjusted to below the upper limit by reducing the gain of the output electrical signal relative to the first electrical signal, thereby avoiding the part of this electrical signal that exceeds the upper limit being directly deleted and causing the corresponding audio data to be distorted. Therefore, this scheme can fully utilize the performance of the acquisition device while at least partially solving the problem of distortion caused by excessive signal amplitude in the electrical signal obtained by recording.

[0065] In order to obtain multiple output electrical signals with good continuity, the second electrical signal can be processed as follows:

[0066] Acquiring a plurality of second electrical signals that are continuous in time;

[0067] Processing the plurality of second electrical signals based on the target model to obtain a plurality of output electrical signals, wherein the i-1th, i-th and i+1th of the plurality of output electrical signals are three third electrical signals that are continuous in time;

[0068] The second gain of the i-th third electrical signal relative to the first electrical signal is greater than the second gain of the (i-1)-th third electrical signal relative to the first electrical signal;

[0069] The second gain of the i-th third electrical signal relative to the first electrical signal is greater than the second gain of the (i+1)-th third electrical signal relative to the first electrical signal.

[0070] That is, when processing a plurality of continuous second electrical signals, the target model may process at least three continuous second electrical signals into at least three continuous third electrical signals, for example Figure 2 In Figure 2 The i-1th second electrical signal, the i-th second electrical signal and the i+1th second electrical signal shown in (1) are processed into Figure 2 As shown in (2), the i-1th third electrical signal, the i-th third electrical signal and the i+1th third electrical signal.

[0071] The second gain value corresponding to the i-th third electrical signal is greater than the second gain value corresponding to the i-1-th third electrical signal, and greater than the second gain value corresponding to the i+1-th third electrical signal. In this way, the target model can control the amplitude corresponding to the i-th third electrical signal to be greater than the amplitude corresponding to the i-1-th third electrical signal, and greater than the amplitude corresponding to the i+1-th third electrical signal, so that the three consecutive third electrical signals are closer to the actual change law of the sound signal, thereby reducing the distortion of the collected audio.

[0072] In some embodiments, among the multiple output electrical signals obtained by the target model, a portion of the output electrical signals may be a fourth electrical signal, and a second gain of the fourth electrical signal relative to the first electrical signal is equal to the first gain value.

[0073] Exemplarily, the acquisition device amplifies a first electrical signal into a second electrical signal according to a first gain value of 5 dB, and the target model processes the second electrical signal and outputs a corresponding fourth electrical signal, and the second gain of the fourth electrical signal relative to the first electrical signal is still 5 dB.

[0074] The fourth electrical signal can be obtained by, after the target model obtains a second electrical signal, if it is determined that the second electrical signal does not need to be processed into a third electrical signal, the second electrical signal can be directly output as the corresponding fourth electrical signal, thereby obtaining a fourth electrical signal with the same second gain value and first gain value.

[0075] For example, after obtaining the i-th second electrical signal, the target model determines whether to process the i-th second electrical signal into a third electrical signal. If the i-th second electrical signal does not need to be processed into a third electrical signal, the i-th second electrical signal is output as a fourth electrical signal. If the i-th second electrical signal needs to be processed into a third electrical signal, the gain value of the i-th second electrical signal relative to the corresponding i-th first electrical signal is adjusted, and an adjusted third electrical signal with a second gain value less than the first gain value is output.

[0076] The target model can determine which second electrical signals are output as the fourth electrical signal based on a plurality of continuous second electrical signals that are input.

[0077] The method of obtaining the third electrical signal based on the target model processing may include:

[0078] Processing N second electrical signals that are continuous in time based on the target model to determine that the N+1th second electrical signal meets the target condition;

[0079] The N+1th second electrical signal is processed based on the target model to obtain a third electrical signal.

[0080] N can be a preset integer, and its value can be set as needed. If the target model needs to have a higher processing efficiency, a smaller N can be set. If the accuracy of the processing result of the target model needs to be improved, a larger N can be output. For example, N can be set to 5, 8 or other integers.

[0081] After the acquisition device starts to output the second electrical signal, each second electrical signal output by the acquisition device can be input into the target model in real time. Each time a second electrical signal is input, the target model can process the second electrical signal and output a corresponding output electrical signal.

[0082] Before the cumulative number of second electrical signals input into the target model reaches N, the target model may process the second electrical signal in the following manner:

[0083] The second electrical signal is directly output as an output electrical signal. In this case, the output electrical signal obtained is equivalent to the aforementioned fourth electrical signal.

[0084] like Figure 3 As shown in (1) and (2), before the Nth second electrical signal is input to the target model, the target model can determine the 1st to Nth second electrical signals as output electrical signals and directly output the 1st to Nth output electrical signals.

[0085] After the cumulative number of second electrical signals input into the target model reaches N, starting from the N+1th second signal input into the target model, each time a second electrical signal is input, the target model can determine whether the second electrical signal meets the target condition based on the N second electrical signals before the second electrical signal; if the second electrical signal meets the target condition, the target model can predict the processing result corresponding to the second electrical signal based on the second electrical signal and the previous N second electrical signals, that is, predict what kind of third electrical signal the second electrical signal will be processed into, and then output the predicted processing result as the output electrical signal.

[0086] like Figure 3As shown in (3), after receiving N second electrical signals cumulatively, when receiving the N+kth second electrical signal, the target model determines whether the N+kth second electrical signal meets the target condition based on the kth to N+k-1th second electrical signals, and after determining that the N+kth second electrical signal meets the target condition, the target model predicts the corresponding processing result based on the kth to N+k-1th second electrical signals, that is, predicts the third electrical signal obtained after processing the N+kth second electrical signal, and outputs the predicted third electrical signal as the N+kth output electrical signal. k can be greater than or equal to 1.

[0087] The beneficial effects of this embodiment are:

[0088] For the N+1th and subsequent second electrical signals, the target model can determine whether to process the second electrical signal into a third electrical signal based on the N second electrical signals before the second electrical signal, and determine the third electrical signal obtained after processing the second electrical signal based on the second electrical signal and the previous N second electrical signals. This can improve the accuracy of the processing results of the target model, so that the output of multiple consecutive output electrical signals is more consistent with the real first sound signal.

[0089] In some embodiments, processing N second electrical signals that are consecutive in time based on the target model to determine that the N+1th second electrical signal meets the target condition may include:

[0090] Based on the target model, N second electrical signals that are consecutive in time are processed to determine that the actual signal amplitude of the N+1th second electrical signal is greater than the signal amplitude upper limit of the acquisition device.

[0091] Similarly, for any N+kth second electrical signal, the N second electrical signals that are temporally consecutive before the second electrical signal, i.e., the kth to N+k-1th second electrical signals, can be processed based on the target model to determine whether the actual signal amplitude of the N+kth second electrical signal is greater than the signal amplitude upper limit of the acquisition device; if so, it is determined that the N+kth second electrical signal meets the target condition; if not, it is determined that the N+kth second electrical signal does not meet the target condition.

[0092] That is to say, the target condition in this embodiment may include that the actual signal amplitude corresponding to the second electrical signal is greater than the signal amplitude upper limit of the acquisition device.

[0093] Due to the limitations of the hardware performance of the acquisition device, there is an upper limit to the signal amplitude that the acquisition device can output. If the signal amplitude of an electrical signal exceeds this upper limit, then when the acquisition device outputs the electrical signal, the acquisition device will directly filter out the electrical signal that exceeds the signal amplitude upper limit, and ultimately output an electrical signal with a signal amplitude that reaches the upper limit of the device.

[0094] If the signal amplitude of a second electrical signal output by the acquisition device reaches the upper limit of the acquisition device, the actual signal amplitude of the second electrical signal may be exactly equal to the upper limit of the acquisition device, or the signal amplitude of the electrical signal obtained after amplification by the first gain value may exceed the upper limit of the acquisition device, causing the acquisition device to filter out the part of the electrical signal that exceeds the upper limit of the signal amplitude and only output the electrical signal whose signal amplitude reaches the upper limit of the device.

[0095] Therefore, for the N+kth second electrical signal input into the target model, if the signal amplitude of the second electrical signal reaches the upper limit of the acquisition device, the target model can predict the actual signal amplitude of the second electrical signal without being filtered by the acquisition device based on the change law of the N consecutive second electrical signals before the second electrical signal in the time domain, thereby determining whether the second electrical signal meets the target condition based on the actual signal amplitude of the second electrical signal.

[0096] In some optional embodiments, the target conditions may also include:

[0097] The actual signal amplitude of the second electrical signal is greater than the upper limit of the signal amplitude of the acquisition device, or the second electrical signal belongs to M second electrical signals before or after the second electrical signal whose actual signal amplitude is greater than the upper limit of the signal amplitude of the acquisition device.

[0098] That is to say, in this embodiment, the second electrical signal whose actual signal amplitude is greater than the signal amplitude upper limit of the acquisition device, and the first M second electrical signals and the last M second electrical signals of the second electrical signal whose actual signal amplitude is greater than the signal amplitude upper limit of the acquisition device can be processed into a third electrical signal based on the target model.

[0099] M may be a fixed value, such as 5, 10, or other integers, or a dynamic value determined based on a target model.

[0100] In combination with the above example, if the target model determines that the actual signal amplitude of the N+1th second electrical signal is greater than the upper limit of the acquisition device, the N+1th second electrical signal can be processed based on the target model to output the corresponding third electrical signal, and it is also possible to:

[0101] Processing a plurality of second electrical signals before the N+1th second electrical signal based on the target model to obtain a plurality of third electrical signals;

[0102] A plurality of second electrical signals after the N+1th second electrical signal are processed based on the target model to obtain a plurality of third electrical signals.

[0103] For any N+kth second electrical signal, if the target model determines that the actual signal amplitude of the N+kth second electrical signal is greater than the upper limit of the acquisition device, the N+kth second electrical signal, the first M second electrical signals of the N+kth second electrical signal, and the last M second electrical signals of the N+kth second electrical signal can also be processed in the above manner to obtain corresponding multiple third electrical signals.

[0104] The beneficial effects of this embodiment are:

[0105] Not only is the gain of the second electrical signal whose actual signal amplitude is greater than the upper limit of the acquisition device adjusted, but the gains of multiple second electrical signals before and after it are also adjusted. This can make the signal envelopes of multiple consecutive third electrical signals obtained after adjustment closer to the signal envelope of the first electrical signal, thereby reducing the distortion of the third electrical signal and making the third electrical signal more realistically reflect the collected sound.

[0106] Optional, such as Figure 4 As shown in (1), before the number of second electrical signals input to the target model reaches N+1, the target model may not output the above-mentioned output electrical signal.

[0107] like Figure 4 As shown in (2), starting from the input of the N+1th second electrical signal into the target model, the target model, on the one hand, predicts whether the N+1th second electrical signal meets the target condition based on the first N second electrical signals, and in this way predicts whether each second electrical signal after the N+1th second electrical signal meets the target condition. On the other hand, the target model can process each second electrical signal one by one starting from the first input second electrical signal, and output an output electrical signal (which can be a third electrical signal or a fourth electrical signal) corresponding to each second electrical signal. That is to say, Figure 4 As shown in (3), the output electrical signal output by the target model can be delayed by N electrical signal times compared to the second electrical signal input to the target model, that is, when the N+1th second electrical signal is input, the target model outputs the first output electrical signal, when the N+2th second electrical signal is input, the target model outputs the second output electrical signal, and when the N+kth second electrical signal is input, the target model outputs the kth output electrical signal.

[0108] Optionally, when the N+1th second electrical signal is processed based on the target model to obtain the third electrical signal, the N+1th second electrical signal may be processed in combination with multiple second electrical signals before and after the N+1th second electrical signal. That is, the N+1th second electrical signal may be processed to obtain the third electrical signal in the following manner:

[0109] The N+1th second electrical signal is processed according to the target model and a plurality of second electrical signals before and after the N+1th second electrical signal to obtain a third electrical signal.

[0110] Among them, the target model can process the N+1th second electrical signal according to the S second electrical signals before and the S second electrical signals after the N+1th second electrical signal to obtain the corresponding third electrical signal.

[0111] Similarly, for any N+kth second electrical signal, if the N+kth second electrical signal meets the target condition, the target model can also process the N+kth second electrical signal based on the S second electrical signals before and the S second electrical signals after the N+kth second electrical signal to obtain the corresponding third electrical signal.

[0112] S is a preset positive integer, and S may be equal to N or not equal to N.

[0113] When the N+1th second electrical signal that meets the target condition is processed in the above manner, the target model can predict the third electrical signal corresponding to the N+1th second electrical signal based on the S second electrical signals before and the S second electrical signals after the N+1th second electrical signal, and then output the predicted electrical signal as the third electrical signal corresponding to the N+1th second electrical signal. The method for processing the N+kth second electrical signal is the same and will not be repeated here.

[0114] Optionally, for any second electrical signal that meets the target condition, a method of predicting a third electrical signal corresponding to the second electrical signal based on the target model may be:

[0115] First, the respective second electrical signals used for prediction are processed by an algorithm for extracting time domain features and frequency domain features of electrical signals in related technologies to obtain time domain features and frequency domain features of the second electrical signals.

[0116] Exemplarily, if it is necessary to predict the third electrical signal corresponding to the N+1th second electrical signal based on the first N second electrical signals, the time domain features and frequency domain features of the first N second electrical signals may be first extracted to obtain N time domain features and N frequency domain features;

[0117] If the third electrical signal corresponding to the N+1th second electrical signal is predicted based on the first S electrical signals and the last S second electrical signals of the N+1th second electrical signal, the time domain features and frequency domain features of the first S second electrical signals can be extracted, and the time domain features and frequency domain features of the last S second electrical signals can be extracted to obtain 2S time domain features and 2S frequency domain features.

[0118] After obtaining the above time domain features and frequency domain features, the time domain features and frequency domain features are input into the target model for processing, and the target model predicts the time domain features and frequency domain features of the third electrical signal based on the input time domain features and frequency domain features.

[0119] Finally, the time domain characteristics and frequency domain characteristics of the third electrical signal output by the target model can be restored by an algorithm for restoring the corresponding signal based on the characteristics of the signal in the relevant technical field, so as to obtain the third electrical signal predicted by the target model.

[0120] Exemplarily, the target model can predict the time domain characteristics and frequency domain characteristics of the third electrical signal corresponding to the N+1th second electrical signal to be processed based on the time domain characteristics and frequency domain characteristics of the first N second electrical signals, and then restore the predicted time domain characteristics and frequency domain characteristics to obtain the third electrical signal corresponding to the N+1th second electrical signal.

[0121] The above example of processing the N+1th second electrical signal can be applied to any N+kth second electrical signal that meets the target condition, and will not be described in detail.

[0122] The beneficial effects of this embodiment are:

[0123] When processing the N+kth second electrical signal, the target model can obtain the signal amplitudes of the multiple preceding and succeeding second electrical signals based on the time domain characteristics of the multiple preceding and succeeding second electrical signals, understand the semantics represented by the multiple consecutive second electrical signals based on the frequency domain characteristics of the multiple preceding and succeeding second electrical signals, and then more accurately predict the changing law of the signal amplitude of the second electrical signal based on the semantics. Thus, the target model can combine the signal amplitudes of the multiple preceding and succeeding second electrical signals and the changing law of the signal amplitudes of the multiple consecutive second electrical signals to predict a more accurate third electrical signal.

[0124] Optionally, the target model may combine the first N second electrical signals to determine whether the N+1th second electrical signal meets the target condition in the following manner:

[0125] Extracting time domain features and frequency domain features of N temporally continuous second electrical signals;

[0126] The time domain features and frequency domain features of the N second electrical signals are processed based on the target model to determine that the N+1th second electrical signal meets the target condition.

[0127] For any N+kth second electrical signal, the target model may also determine whether the N+1th second electrical signal meets the target condition according to the N second electrical signals before the N+kth second electrical signal in the above manner.

[0128] As an example, the time domain features of the N second electrical signals which are continuous in time can be extracted as follows: Figure 5 As described in (1), the frequency domain characteristics of the N second electrical signals can be expressed as follows: Figure 5As shown in (2), the time domain characteristics of the second electrical signal reflect the overall signal amplitude of the second electrical signal in the corresponding time period, and the frequency domain characteristics of the second electrical signal reflect the strength of the signal components of the second electrical signal at each frequency.

[0129] The horizontal axes 1 to N in the figure represent the 1st to Nth second electrical signals.

[0130] It can be seen that due to the different semantics represented by different sound signals, the tones of different sound signals are also different, and the difference in tone can be reflected in the frequency domain characteristics of the corresponding electrical signal. For example, for a sound signal with a higher pitch, the high-frequency signal component of the corresponding second electrical signal is stronger, and the low-frequency signal component is weaker. For a sound signal with a lower pitch, the high-frequency signal component of the corresponding second electrical signal is weaker, and the low-frequency signal component is stronger.

[0131] According to the above properties, on the one hand, after the target model obtains the frequency domain features of N second electrical signals, it can understand the semantics of the sound signals corresponding to the N second electrical signals based on these frequency domain features, and then infer the semantics of the sound signal corresponding to the N+1th second electrical signal based on the semantics corresponding to the N second electrical signals; on the other hand, the target model can understand the law of how the signal amplitude of the second electrical signal changes over time based on the time domain features of the first N second electrical signals.

[0132] Therefore, the target model can determine the actual signal amplitude of the N+1th second electrical signal by combining the semantics corresponding to the N+1th second electrical signal and the signal amplitude change law of the second electrical signal, thereby determining whether the actual signal amplitude of the N+1th second electrical signal is greater than the signal amplitude upper limit of the acquisition device.

[0133] In addition, when it is determined that the N+1th second electrical signal needs to be processed into a third electrical signal, the target model can predict the frequency domain characteristics of the third electrical signal obtained by processing based on the semantics corresponding to the inferred N+1th second electrical signal, predict the time domain characteristics of the third electrical signal processed based on the law of change of the signal amplitude, and finally generate the processed third electrical signal by combining the frequency domain characteristics and the time domain characteristics.

[0134] The beneficial effects of this embodiment are that, through the time domain characteristics and frequency domain characteristics of the first N second electrical signals, the target model can determine the signal amplitudes of the first N second electrical signals, and can predict the changing law of the signal amplitudes of the second electrical signals based on the semantics represented by multiple consecutive second electrical signals. Thus, the target model can combine the signal amplitudes of the first N second electrical signals and the changing law of the signal amplitudes to more accurately determine whether the N+1th second electrical signal meets the target conditions.

[0135] Optionally, the target model may be constructed based on a sample sound signal and a target sound signal collected from the same sound source, the sample sound signal being collected by a collection device, and the target sound signal being collected by a target device different from the collection device.

[0136] The sample sound signal and the target sound signal can be collected as follows:

[0137] The sound source device, the acquisition device, and the target device that generate sound are placed separately, wherein the acquisition device and the target device can be placed in the same position. The acquisition device can be a sound acquisition device with low precision integrated on the terminal device (such as a mobile phone), such as a mobile phone microphone, and the target device can be a standard recording device with high precision, such as a standard microphone. The sound source device can be a speaker or other device that can emit sound to the environment.

[0138] After placing the above devices, start the sound source device, the acquisition device and the target device, so that the sound source device sends out a sound signal, and the acquisition device and the target device simultaneously collect the sound signal generated by the sound source device. At this time, the sound signal obtained by the acquisition device can be used as a sample sound signal, and the sound signal obtained by the target device can be used as a target sound signal.

[0139] Among them, the sound signal emitted by the sound source device can be obtained as follows:

[0140] The process of the user speaking is recorded to obtain a voice message emitted by the user, and then random dynamic gain processing is performed on the voice message to simulate the random fluctuation of the sound source volume. The sound signal obtained after the random dynamic gain processing is used as the sound signal emitted by the sound source device.

[0141] The method of constructing a target model based on the sample sound signal and the target sound signal can be:

[0142] First, a randomly initialized neural network model is obtained as the initial model;

[0143] Converting target sound signals into target electrical signals;

[0144] Based on the initial model, the sample sound signal is processed according to the method of the above embodiment to obtain a continuous output electrical signal;

[0145] Comparing the output electrical signal and the target electrical signal, the model loss of the initial model, wherein the model loss can be determined according to the Euclidean distance and the similarity, the Euclidean distance can be the Euclidean distance between the frequency domain features of the output electrical signal and the frequency domain features of the target electrical signal, and the similarity can be the similarity between the time domain features of the output electrical signal and the time series features of the target electrical signal;

[0146] If the model loss does not meet the convergence condition, the model parameters of the initial model can be updated according to the model loss, and after the update is completed, the step of processing the sample sound signal according to the method of the above embodiment based on the initial model to obtain a continuous output electrical signal is returned to execute;

[0147] If the model loss meets the convergence condition, it can be determined that the initial model at this time is the constructed target model;

[0148] The convergence condition may be that the model loss is less than or equal to a specific convergence threshold.

[0149] The present application also provides an audio processing device. Figure 6 , is a schematic diagram of the structure of the device, and the device may include the following units.

[0150] The obtaining unit 601 is used to obtain a first sound signal in real time based on a collection device.

[0151] The conversion unit 602 is configured to convert the first sound signal into a first electrical signal.

[0152] The amplifying unit 603 is used to amplify the first electrical signal into a second electrical signal based on a first gain, where the first gain is a first gain value corresponding to the acquisition device.

[0153] The processing unit 604 is used to process the second electrical signal based on the target model to obtain an output electrical signal, wherein the change of the output electrical signal relative to the first electrical signal is a second gain value; the output electrical signal includes a third electrical signal, and the second gain of the third electrical signal relative to the first electrical signal is less than the first gain value.

[0154] Optionally, when the processing unit 604 processes the second electrical signal based on the target model to obtain an output electrical signal, it can be used to:

[0155] Acquiring a plurality of second electrical signals that are continuous in time;

[0156] Processing the plurality of second electrical signals based on the target model to obtain a plurality of output electrical signals, wherein the i-1th, i-th and i+1th of the plurality of output electrical signals are three third electrical signals that are continuous in time;

[0157] The second gain of the i-th third electrical signal relative to the first electrical signal is greater than the second gain of the (i-1)-th third electrical signal relative to the first electrical signal;

[0158] The second gain of the i-th third electrical signal relative to the first electrical signal is greater than the second gain of the (i+1)-th third electrical signal relative to the first electrical signal.

[0159] Optionally, the output electrical signal includes a fourth electrical signal, and a second gain of the fourth electrical signal relative to the first electrical signal is equal to the first gain value.

[0160] Optionally, when the processing unit 604 processes the second electrical signal based on the target model to obtain an output electrical signal, it can be used to:

[0161] Processing N second electrical signals that are continuous in time based on the target model to determine that the N+1th second electrical signal meets the target condition;

[0162] The N+1th second electrical signal is processed based on the target model to obtain a third electrical signal.

[0163] Optionally, when the processing unit 604 processes N second electrical signals that are continuous in time based on the target model and determines that the N+1th second electrical signal meets the target condition, it can be used to:

[0164] Extracting time domain features and frequency domain features of N temporally continuous second electrical signals;

[0165] The time domain features and frequency domain features of the N second electrical signals are processed based on the target model to determine that the N+1th second electrical signal meets the target condition.

[0166] Optionally, when the processing unit 604 processes the N+1th second electrical signal based on the target model to obtain the third electrical signal, it can be used to:

[0167] The N+1th second electrical signal is processed according to the target model and a plurality of second electrical signals before and after the N+1th second electrical signal to obtain a third electrical signal.

[0168] Optionally, the processing unit 604 is further configured to:

[0169] Processing a plurality of second electrical signals before the N+1th second electrical signal based on the target model to obtain a plurality of third electrical signals;

[0170] A plurality of second electrical signals after the N+1th second electrical signal are processed based on the target model to obtain a plurality of third electrical signals.

[0171] Optionally, when the processing unit 604 processes N second electrical signals that are continuous in time based on the target model and determines that the N+1th second electrical signal meets the target condition, it can be used to:

[0172] Based on the target model, N second electrical signals that are consecutive in time are processed to determine that the actual signal amplitude of the N+1th second electrical signal is greater than the signal amplitude upper limit of the acquisition device.

[0173] Optionally, the target model is constructed based on a sample sound signal and a target sound signal collected from the same sound source, the sample sound signal is collected by a collection device, and the target sound signal is collected by a target device different from the collection device.

[0174] The working principle of the audio processing device of this embodiment can refer to the relevant steps of the audio processing method provided in any embodiment of the present application, and will not be repeated here.

[0175] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without creative work.

[0176] Those skilled in the art may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein may be implemented by electronic hardware, computer software, or a combination of both.

[0177] In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0178] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An audio processing method, include: Obtaining a first sound signal in real time based on an acquisition device; converting the first sound signal into a first electrical signal; amplifying the first electrical signal into a second electrical signal based on a first gain, wherein the first gain is a first gain value corresponding to the acquisition device; The second electrical signal is processed based on the target model to obtain an output electrical signal, wherein the change of the output electrical signal relative to the first electrical signal is a second gain value; the output electrical signal includes a third electrical signal, and the second gain of the third electrical signal relative to the first electrical signal is less than the first gain value.

2. The method according to claim 1, wherein the second electrical signal is processed based on the target model to obtain an output electrical signal, include: Acquiring a plurality of second electrical signals that are continuous in time; Processing the plurality of second electrical signals based on the target model to obtain a plurality of output electrical signals, wherein the i-1th, i-th and i+1th of the plurality of output electrical signals are three third electrical signals that are continuous in time; The second gain of the i-th third electrical signal relative to the first electrical signal is greater than the second gain of the (i-1)-th third electrical signal relative to the first electrical signal; The second gain of the i-th third electrical signal relative to the first electrical signal is greater than the second gain of the (i+1)-th third electrical signal relative to the first electrical signal. 3 . The method according to claim 1 , wherein the output electrical signal comprises a fourth electrical signal, and a second gain of the fourth electrical signal relative to the first electrical signal is equal to the first gain value.

4. The method according to claim 1, wherein the second electrical signal is processed based on the target model to obtain an output electrical signal, include: Processing N second electrical signals that are consecutive in time based on the target model to determine that the N+1th second electrical signal meets the target condition; The N+1th second electrical signal is processed based on the target model to obtain a third electrical signal.

5. The method according to claim 4, wherein the target model is used to process N second electrical signals that are consecutive in time, and determine that the N+1th second electrical signal meets the target condition, include: Extracting time domain features and frequency domain features of N temporally continuous second electrical signals; The time domain features and frequency domain features of the N second electrical signals are processed based on the target model to determine that the N+1th second electrical signal meets the target condition.

6. The method according to claim 4, wherein the N+1th second electrical signal is processed based on the target model to obtain a third electrical signal, include: The N+1th second electrical signal is processed according to the target model and a plurality of second electrical signals before and after the N+1th second electrical signal to obtain a third electrical signal.

7. The method according to claim 4, further comprising: include: Processing a plurality of the second electrical signals before the N+1th second electrical signal based on the target model to obtain a plurality of third electrical signals; A plurality of the second electrical signals after the N+1th second electrical signal is processed based on the target model to obtain a plurality of third electrical signals.

8. The method according to claim 4, wherein the target model is used to process N second electrical signals that are consecutive in time, and determine that the N+1th second electrical signal meets the target condition, include: Based on the target model, N second electrical signals that are consecutive in time are processed to determine that the actual signal amplitude of the N+1th second electrical signal is greater than the signal amplitude upper limit of the acquisition device.

9. According to the method of claim 1, the target model is constructed based on a sample sound signal and a target sound signal collected from the same sound source, the sample sound signal is collected by the collection device, and the target sound signal is collected by a target device different from the collection device.

10. An audio processing device, include: An acquisition unit, configured to acquire a first sound signal in real time based on a collection device; A conversion unit, configured to convert the first sound signal into a first electrical signal; an amplifying unit, configured to amplify the first electrical signal into a second electrical signal based on a first gain, wherein the first gain is a first gain value corresponding to the acquisition device; A processing unit is used to process the second electrical signal based on a target model to obtain an output electrical signal, wherein the change of the output electrical signal relative to the first electrical signal is a second gain value; the output electrical signal includes a third electrical signal, and the second gain of the third electrical signal relative to the first electrical signal is less than the first gain value.

Images