Speech signal processing method and apparatus, computing device, and storage medium

Abstract

The present disclosure provides a speech signal processing method, which comprises: obtaining at least one representative speech frame of a speech signal; obtaining power spectrum information and a fundamental frequency corresponding to each representative speech frame in the at least one representative speech frame; determining that the speech signal is blocked by a barrier when it is determined that the voice quality of the speech signal is impaired based on the fundamental frequency and the power spectrum information corresponding to each representative speech frame; and compensating the speech signal when it is determined that the speech signal is blocked by the barrier. In addition, the present disclosure also relates to a speech signal processing device, a computing device and a computer readable storage medium.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, and in particular, to a speech signal processing method, a speech signal processing device, a computing device and a computer readable storage medium for speech signal blocking situation. BACKGROUND

[0002] At present, speech communication applications are increasingly common. However, in some application scenarios, speech signals can be blocked by a blocking object, which can cause the sound quality of the speech signals to be damaged. For example, during the epidemic period, public health and safety issues are increasingly valued. In order to prevent the spread of viruses and avoid virus infection, wearing a mask has become a necessary behavior in daily life. However, in the case of wearing a mask, the mouth is blocked by the mask, and the sound is blocked by the mask when speaking. Therefore, compared with not wearing a mask, the sound when speaking will be stuffy. If speech communication is carried out in the case of wearing a mask, the sound after speech coding and network transmission will cause the intelligibility of the speech to be greatly affected, so that the user receiving the speech signal cannot clearly hear and understand the content expressed by the speech signal, which seriously affects the experience and effect of speech communication. The existing speech communication application does not consider the situation that the speech signal is blocked by a blocking object (for example, the scenario of a user wearing a mask), so the conventional speech enhancement methods, such as echo cancellation, noise suppression, volume enhancement, etc., cannot play a role in this scenario. SUMMARY

[0003] According to a first aspect of the present disclosure, a speech signal processing method is provided, which comprises: obtaining at least one representative speech frame of a speech signal; obtaining power spectrum information and a fundamental frequency corresponding to each representative speech frame in the at least one representative speech frame; and determining that the speech signal is blocked by a blocking object when it is determined that the sound quality of the speech signal is damaged based on the fundamental frequency and the power spectrum information corresponding to each representative speech frame.

[0004] According to some example embodiments, the obtaining of the at least one representative speech frame of the speech signal comprises: obtaining a plurality of active speech frames of the speech signal; performing Fourier transform on the plurality of active speech frames to obtain power spectrum information corresponding to each active speech frame in the plurality of active speech frames; obtaining a mid-low frequency band energy value corresponding to each active speech frame based on the power spectrum information corresponding to each active speech frame; and determining the at least one representative speech frame from the plurality of active speech frames based on the mid-low frequency band energy value corresponding to each active speech frame.

[0005] According to some example embodiments, the obtaining of the plurality of active speech frames of the speech signal comprises: performing speech activity detection on the speech signal to obtain a plurality of active speech frames of the speech signal, wherein each active speech frame includes human voice and has a preset time period length.

[0006] According to some example embodiments, the determining the at least one representative speech frame from the plurality of active speech frames based on the mid-low band energy value corresponding to each active speech frame comprises: determining an active speech frame corresponding to a mid-low band energy value greater than a preset mid-low band energy threshold as a representative speech frame.

[0007] According to some example embodiments, the determining the at least one representative speech frame from the plurality of active speech frames based on the mid-low band energy value corresponding to each active speech frame comprises: sorting the plurality of active speech frames based on the mid-low band energy value corresponding to each active speech frame; and determining the at least one representative speech frame from the plurality of active speech frames based on the sorting result.

[0008] According to some example embodiments, the determining the at least one representative speech frame from the plurality of active speech frames based on the sorting result comprises: selecting representative speech frames from the plurality of active speech frames according to a preset number in a decreasing order of the mid-low band energy value starting from an active speech frame corresponding to a greatest mid-low band energy value.

[0009] According to some example embodiments, the determining the at least one representative speech frame from the plurality of active speech frames based on the sorting result comprises: selecting representative speech frames from the plurality of active speech frames according to a preset percentage in a decreasing order of the mid-low band energy value starting from an active speech frame corresponding to a greatest mid-low band energy value.

[0010] According to some example embodiments, the obtaining the pitch frequency corresponding to the at least one representative speech frame comprises: performing pitch frequency detection on the plurality of active speech frames to obtain the pitch frequency corresponding to each active speech frame.

[0011] According to some example embodiments, the speech signal processing method further comprises: compensating the speech signal when it is determined that the speech signal is blocked by a blocking object.

[0012] According to some example embodiments, the compensating the speech signal when it is determined that the speech signal is blocked by a blocking object comprises: determining a compensation gain corresponding to each frequency band based on the pitch frequency; and performing speech enhancement processing on the speech signal using the compensation gain.

[0013] According to some example embodiments, the determining the compensation gain based on the pitch frequency comprises: dividing a pitch frequency range into a plurality of pitch intervals, and determining an interval gain of each pitch interval of the plurality of pitch intervals; determining a pitch interval in which the pitch frequency is located; and determining the interval gain of the pitch interval in which the pitch frequency is located as the compensation gain.

[0014] According to some example embodiments, the dividing the pitch frequency range into a plurality of pitch intervals, and determining an interval gain of each pitch interval of the plurality of pitch intervals comprises: for each pitch interval, respectively calculating a frequency band power spectrum average of a plurality of unoccluded speech signal samples and a frequency band power spectrum average of a plurality of occluded speech signal samples; and determining the interval gain of each pitch interval based on a ratio of the frequency band power spectrum average of the plurality of unoccluded speech samples and the frequency band power spectrum average of the plurality of occluded speech samples.

[0015] According to some example embodiments, the plurality of intervals are obtained by equally dividing the pitch frequency range.

[0016] According to some example embodiments, the determining that the speech signal is occluded by an occlusion object based on the pitch frequency and the power spectrum information corresponding to each representative speech frame when it is determined that the voice quality of the speech signal is impaired comprises: detecting the voice quality of the speech signal using a speech signal occlusion detection model based on the pitch frequency and the power spectrum information corresponding to each representative speech frame to generate a speech signal occlusion probability; and determining that the speech signal is occluded by an occlusion object when the speech signal occlusion probability is greater than a preset speech signal occlusion threshold; wherein the speech signal occlusion detection model is obtained by training a neural network using pitch frequencies and power spectrum information of a plurality of training speech samples.

[0017] According to a second aspect of the present disclosure, a speech signal processing apparatus is provided, comprising: a representative speech frame obtaining module configured to obtain at least one representative speech frame of a speech signal; a power spectrum information obtaining module configured to obtain power spectrum information corresponding to each representative speech frame of the at least one representative speech frame; a pitch frequency obtaining module configured to obtain a pitch frequency corresponding to each representative speech frame of the at least one representative speech frame; and a speech signal occlusion condition determining module configured to: determine that the speech signal is occluded by an occlusion object when it is determined that the voice quality of the speech signal is impaired based on the pitch frequency and the power spectrum information corresponding to each representative speech frame.

[0018] According to some example embodiments, the speech signal processing apparatus further comprises: a compensation module configured to compensate the speech signal when it is determined that the speech signal is occluded by an occlusion object.

[0019] According to some example embodiments, the compensation module is further configured to: determine compensation gains corresponding to each frequency band based on the pitch frequencies; and perform speech enhancement processing on the speech signal using the compensation gains.

[0020] According to some example embodiments, the speech signal obstruction determination module is further configured to: detect the quality of the speech signal using a speech signal obstruction detection model based on the pitch frequencies and the power spectrum information of the representative speech frames, to generate a speech signal obstruction probability; and determine that the speech signal is obstructed by an obstruction when the speech signal obstruction probability is greater than a preset speech signal obstruction threshold.

[0021] According to a third aspect of the present disclosure, a computing device is provided, comprising a processor and a memory configured to store computer-executable instructions configured to, when executed on the processor, cause the processor to perform the speech signal processing method according to the above aspects and example embodiments.

[0022] According to a third aspect of the present disclosure, a computer-readable storage medium is provided, configured to store computer-executable instructions configured to, when executed on a processor, cause the processor to perform the speech signal processing method according to the above aspects and example embodiments.

[0023] The speech signal processing method for speech signal obstruction according to the present disclosure at least achieves the following beneficial technical effects: first, it can determine whether the speech signal is obstructed by an obstruction (for example, due to the user wearing a mask) during a voice call; second, it can grade the degree of damage to the quality of the speech signal caused by the obstruction, and use different speech enhancement parameters for processing under different levels, thereby improving the quality of the speech signal and solving the problem of unclear speech content recognition by the other party in the case of calling while wearing a mask. BRIEF DESCRIPTION OF DRAWINGS

[0024] The specific embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings, so that more details, features and advantages of the present disclosure can be fully understood and appreciated; in the drawings:

[0025] FIG. 1A and FIG. 1B The quality of the speech signal in an application scenario in which the speech signal is obstructed by an obstruction is schematically shown;

[0026] FIG. 2 A method of processing a speech signal is schematically illustrated in the form of a flow chart according to some example embodiments of the present disclosure;

[0027] FIG. 3 Details of respective steps in the method of processing a speech signal are further schematically illustrated in the form of a flow chart according to some example embodiments of the present disclosure; FIG. 2

[0028] FIG. 4 Details of respective steps in the method of processing a speech signal are further schematically illustrated in the form of a flow chart according to some example embodiments of the present disclosure; FIG. 3

[0029] FIG. 5 Details of respective steps in the method of processing a speech signal are further schematically illustrated in the form of a flow chart according to some example embodiments of the present disclosure; FIG. 2

[0030] FIG. 6 A method of processing a speech signal is schematically illustrated in the form of a flow chart according to some other example embodiments of the present disclosure;

[0031] FIG. 7 Details of respective steps in the method of processing a speech signal are further schematically illustrated in the form of a flow chart according to some example embodiments of the present disclosure; FIG. 6

[0032] FIG. 8 Details of respective steps in the method of processing a speech signal are further schematically illustrated in the form of a flow chart according to some example embodiments of the present disclosure; FIG. 7

[0033] FIG. 9 Details of respective steps in the method of processing a speech signal are further schematically illustrated in the form of a flow chart according to some example embodiments of the present disclosure; FIG. 8

[0034] FIG. 10 A method of processing a speech signal is schematically illustrated in the form of a flow chart according to some other example embodiments of the present disclosure;

[0035] FIG. 11 Effects of speech enhancement processing on a speech signal in a case where the speech signal is obstructed by an obstruction are schematically illustrated in the form of a spectrogram;

[0036] FIG. 12 A structure of a speech signal processing apparatus according to some example embodiments of the present disclosure is schematically illustrated;

[0037] ​​​​​​FIG. 13 schematically illustrates a structure of a voice signal processing apparatus according to another example embodiment of the present disclosure;

[0038] FIG. 14 schematically illustrates FIG. 12 、 FIG. 13 schematically illustrates a structure of a neural network included in respective modules of the voice signal processing apparatus;

[0039] FIG. 15 schematically illustrates a structure of an example computing device according to an embodiment of the present disclosure.

[0040] It should be understood that all matters displayed in the drawings are merely schematic, and thus they are not necessarily drawn to scale. Also, in all the drawings, the same or similar features are indicated by the same or similar reference numerals. DETAILED DESCRIPTION

[0041] The following description provides specific details of various embodiments of the present disclosure in order to provide a sufficient understanding of and enable a full appreciation of the various embodiments of the present disclosure.

[0042] First, some terms involved in the embodiments of the present disclosure are explained in order to facilitate understanding by those skilled in the art:

[0043] Fundamental frequency: refers to the frequency of vibration of the vocal cords when a person speaks. Generally, the fundamental frequency is related to the length, thickness, toughness, stiffness, and pronunciation habits of the vocal cords of an individual, and largely reflects the characteristics of the individual. In addition, the fundamental frequency also varies with the gender and age of the person. Generally speaking, the fundamental frequency of male speakers is lower, while the fundamental frequency of female speakers and children is relatively higher.

[0044] Long Short-Term Memory network (LSTM network): is a special RNN neural network structure improved from the traditional recurrent neural network (i.e., RNN network), which can learn long-term dependencies. Each LSTM structure includes an input gate, a forget gate, and an output gate, which respectively control the input value, the memory value, and the output value. The LSTM network not only solves the problem of long-distance dependence that the traditional RNN network cannot handle, but also solves the problem of gradient explosion or gradient disappearance commonly seen in neural networks, and is very effective in processing sequence data.

[0045] Gated recurrent unit network (GRU network): a variant of LSTM network obtained by simplifying the LSTM network. Unlike the LSTM structure, each GRU unit has only an update gate and a reset gate. The update gate is used to control the degree to which the state information at the previous moment is brought into the current state. The greater the value of the update gate, the more the state information at the previous moment is brought in. The reset gate controls how much information of the previous state is written to the current candidate set. The smaller the reset gate, the less the previous state is written. Therefore, the GRU model is simpler than the LSTM model, so it has higher training efficiency and lower requirements for hardware.

[0046] As mentioned earlier, in some application scenarios of voice communication, the voice signal can be blocked by a barrier, which can cause the voice quality to be damaged. One possible application scenario is that there are users wearing masks among the parties of voice communication. In the case of wearing a mask, since the mouth is blocked by the mask, the sound of the user speaking is blocked by the mask, so compared with not wearing a mask, the user's voice will sound dull, resulting in a significant decrease in speech intelligibility.

[0047] Referring to FIG. 1A and FIG. 1B , which schematically show the case of voice quality damage of a voice signal in a scenario of wearing a mask. FIG. 1A The voice quality of the same voice signal after recording without wearing a mask and with wearing a mask is schematically shown in the form of a spectrogram. FIG. 1B The voice spectrum of the same voice signal after recording without wearing a mask and with wearing a mask is schematically shown in the form of a spectrogram. As can be seen from the figure, in the case of wearing a mask, since the sound is blocked by the mask after wearing the mask, the voice signal in different frequency bands will suffer different degrees of damage, among which the low-frequency part is slightly damaged, but the high-frequency part is more obviously damaged, which is manifested as more brightness in the high-frequency (above 2000 Hz) range (as shown in FIG. 1A ) or more amplitude reduction of the spectrum line in the high-frequency (above 2000 Hz) range (as shown in FIG. 1B ). The high-frequency voice signal has a great influence on speech intelligibility, and if this part of the signal is severely damaged, the result is that the user receiving the voice signal may not be able to clearly distinguish the content of the voice signal, which will seriously affect the effect and experience of voice communication.

[0048] Referring to FIG. 2The document schematically illustrates, in flowchart form, a speech signal processing method according to some exemplary embodiments of the present disclosure, which can be applied to situations where the speech signal is blocked by an obstruction. This speech signal processing method can detect the sound quality of the speech signal based on the fundamental frequency and power spectrum information of each frequency point within the speech frequency band to determine whether the sound quality of the speech signal is impaired, and when it is determined that the sound quality of the speech signal is impaired, it determines that the speech signal is blocked by an obstruction. Therefore, this speech signal processing method can be applied to, for example... FIG. 1A and FIG. 1B In the application scenario shown, this is to detect whether the person is wearing a mask during the current voice call. FIG. 2 As shown, the speech signal processing method 100 may include steps 110, 120, 130, and 140:

[0049] In step 110, at least one representative speech frame of the speech signal is acquired;

[0050] In step 120, the power spectrum information corresponding to each representative speech frame in the at least one representative speech frame is obtained;

[0051] In step 130, the fundamental frequency corresponding to each representative speech frame in the at least one representative speech frame is obtained;

[0052] In step 140, when it is determined that the sound quality of the speech signal is impaired based on the fundamental frequency and power spectrum information corresponding to each representative speech frame, it is determined that the speech signal is blocked by an obstruction.

[0053] In step 110, a predetermined number of active speech frames can be selected from multiple detected active speech frames of the speech signal (as a non-limiting example, detection can be performed using a speech activity algorithm) according to a preset standard as representative speech frames for use in detecting the sound quality of the speech signal. An active speech frame is a speech frame in the speech signal that contains a human voice signal and has a preset time period length. A representative speech frame is an active speech frame selected from each active speech frame of the speech signal that reflects the characteristics of the speech signal and can be used to evaluate the sound quality of the speech signal. During a voice call, the energy of a human voice is mainly concentrated in the mid-to-low frequency band of the audio frequency range. Therefore, representative speech frames can usually be selected based on the mid-to-low frequency energy value corresponding to the active speech frame in the mid-to-low frequency band of the audio frequency range, which will be explained in further detail below. However, it should be understood that, depending on actual needs, any suitable active speech frame of the speech signal can be used as a representative speech frame for the speech signal.

[0054] Thus, the voice signal processing method 100 can determine whether the voice signal is blocked by a blocking object during the voice call process, for example, due to the user wearing a mask. The existing voice call process does not consider the case where the voice signal is blocked by a blocking object, and thus cannot identify whether the user wears a mask in an application scenario such as the user wearing a mask.

[0055] Referring to FIG. 3 According to some example embodiments of the present disclosure, the details of step 110 in the voice signal processing method 100 are schematically shown in the form of a flowchart as shown in FIG. 2 As shown in FIG. 3 In this example embodiment, step 110 can further include steps 111, 112, 113, and 114 as shown in the following.

[0056] In step 111, a plurality of active voice frames of the voice signal are obtained.

[0057] In step 112, Fourier transform is performed on the plurality of active voice frames to obtain power spectrum information corresponding to each active voice frame in the plurality of active voice frames.

[0058] In step 113, based on the power spectrum information corresponding to each active voice frame, a mid-low frequency band energy value corresponding to each active voice frame is obtained.

[0059] In step 114, based on the mid-low frequency band energy value corresponding to each active voice frame, at least one representative voice frame is determined from the plurality of active voice frames.

[0060] In step 111, the voice signal obtained by recording, for example, can be detected by any suitable means (e.g., a voice activity algorithm) to detect a voice signal containing human voice signals and having a predetermined time period length as an active voice frame. The part of the voice signal that does not contain human voice, such as the time period when no one speaks during the recording process, can be discarded. Therefore, all active voice frames contain human voice signals, which can be used to evaluate the quality of the voice signal.

[0061] In step 112, Fourier transform can be performed on the obtained plurality of active voice frames to obtain the corresponding power spectrum information of each frequency point in the sound frequency band range of each active voice frame. It should be understood that the sound frequency band range is known in the art, which refers to the frequency range of the sound that can be heard by humans, i.e., the frequency range of 20 Hz to 20000 Hz. Therefore, the power spectrum information of each active voice frame obtained in step 112 is the corresponding power spectrum of the human voice signal contained in each active voice frame at each frequency point in the sound frequency band range.

[0062] In step 113, the mid-low frequency band energy value corresponding to each active speech frame can be obtained by integrating the power spectrum information of each active speech frame in the mid-low frequency band range of the sound frequency band range. It should be understood that the mid-low frequency band range of the sound frequency band range is also known in the art, which generally refers to the frequency range of 20 Hz to 2000 Hz. In general voice communication application scenarios, the main frequency components of human voice signals are mostly in this mid-low frequency band range. It should be understood that in other exemplary embodiments, a narrower frequency range can be selected from the above-mentioned mid-low frequency band range to calculate the mid-low frequency band energy value corresponding to each active speech frame, for example, a frequency range of 300 Hz to 1200 Hz can be selected to calculate the mid-low frequency band energy value corresponding to each active speech frame.

[0063] In step 114, after obtaining the mid-low frequency band energy value corresponding to each active speech frame, at least one representative speech frame can be determined from the plurality of active speech frames in different ways according to actual needs. In one non-limiting embodiment, the mid-low frequency band energy value corresponding to each active speech frame can be compared with a pre-set mid-low frequency band energy threshold, and when the mid-low frequency band energy value is greater than the mid-low frequency band energy threshold, the active speech frame corresponding to the mid-low frequency band energy value can be determined as the representative speech frame.

[0064] In addition, referring to FIG. 4 , which further schematically shows the details of step 114 in the method shown in the flowchart according to some exemplary embodiments of the present disclosure. FIG. 3 As shown in FIG. 4 , in this exemplary embodiment, step 114 can further include steps 1141 and 1142:

[0065] In step 1141, the plurality of active speech frames are sorted based on the mid-low frequency band energy values corresponding to each active speech frame.

[0066] In step 1142, the at least one representative speech frame is determined from the plurality of active speech frames based on the sorting result.

[0067] In step 1141, a suitable sorting method can be selected according to actual needs. As a non-limiting example, the mid-low frequency band energy values can be sorted from large to small. However, any other suitable sorting method is also possible, and the present disclosure does not limit

[0068] In step 1142, the representative speech frames can be determined from the plurality of active speech frames according to actual needs and different criteria, as long as the selected representative speech frames contain sufficient voice signals that can be used to evaluate the voice quality of the voice signal. As a non-limiting example, a preset number of active speech frames can be selected as the representative speech frames from the plurality of active speech frames in order of decreasing mid-low frequency band energy values, starting from the active speech frame corresponding to the largest mid-low frequency band energy value. In another non-limiting example, a preset percentage of active speech frames corresponding to a number of active speech frames can be selected as the representative speech frames from the plurality of active speech frames in order of decreasing mid-low frequency band energy values, starting from the active speech frame corresponding to the largest mid-low frequency band energy value. For example, the percentage can be 20%. In this case, the top 20% of active speech frames in order can be selected as the representative speech frames from the plurality of active speech frames, starting from the active speech frame corresponding to the largest mid-low frequency band energy value.

[0069] Selecting the representative speech frames in a preset number is relatively easy to implement, while selecting the representative speech frames in a preset percentage is more flexible as it can select the representative speech frames in a variable number according to the number of active speech frames.

[0070] With reference to the voice processing method 100, FIG. 2 In step 120 of the voice processing method 100, after obtaining the representative speech frames, the corresponding power spectrum information of each frequency point in the sound frequency band range of each representative speech frame can be obtained by performing Fourier transform on the representative speech frames. However, it should be understood that when the method shown in FIG. 3 When the representative speech frames are obtained by the method shown in

[0071] In step 130 of the voice processing method 100, the pitch frequency of each active speech frame can be obtained by performing pitch frequency detection on at least one representative speech frame. It should be understood that any suitable pitch frequency detection method can be used in the voice processing method 100 to detect the pitch frequency of each active speech frame. The pitch frequency reflects the difference in the sound characteristics of different human voices. Generally speaking, the larger the pitch frequency value of a sound, the higher the proportion of mid-high frequency energy is usually. For example, the pitch frequency value of a female voice signal is usually large, so the proportion of mid-high frequency energy is usually higher than that of a male voice signal. Obtaining the pitch frequency of each active speech frame of the voice signal is beneficial to more accurately detecting the voice quality of the voice signal, and is also beneficial to giving the corresponding equalization parameters according to the pitch frequency value range for equalization adjustment.

[0072] In step 140 of the speech signal processing method 100, any suitable manner can be selected to determine whether the quality of the speech signal is impaired according to actual needs. As a non-limiting example, the pitch frequency and the power spectrum information corresponding to each representative speech frame can be compared with the pitch frequency and the power spectrum information of the corresponding unobstructed speech signal. If the intensity of the speech signal contained in each representative speech frame is reduced at the corresponding frequency point, it can be determined that the speech signal contained in each representative speech frame is impaired in quality compared with the unobstructed speech signal.

[0073] In addition, referring to FIG. 5 which further schematically shows the details of step 140 in the speech signal processing method 100 according to some exemplary embodiments of the present disclosure in the form of a flow chart. FIG. 2 As shown in the exemplary embodiment FIG. 5 , step 140 can further include steps 141, 142:

[0074] In step 141, the quality of the speech signal is detected based on the pitch frequency and the power spectrum information corresponding to each representative speech frame using a speech signal obstruction detection model to generate a speech signal obstruction probability.

[0075] In step 142, when the speech signal obstruction probability is greater than a preset speech signal obstruction threshold, it is determined that the speech signal is obstructed by an obstruction.

[0076] In step 141, the speech signal obstruction detection model can be obtained by training a corresponding neural network using the pitch frequency and the power spectrum information of a plurality of training speech samples. As a non-limiting example, the pitch frequency and the power spectrum information of a large number of training speech samples can be used as the input of a neural network that can be deep-learned, and the output expected value of the training of the neural network corresponds to, for example, "1" for a speech signal obstruction sample and "0" for a speech signal non-obstruction sample. It should be understood that for the application scenario of wearing a mask, the output expected value of the training of the neural network can correspond to, for example, "1" for a mask-wearing sample and "0" for a non-mask-wearing sample. In addition, it should also be understood that any neural network with any suitable structure is possible as long as it can be trained using the pitch frequency and the power spectrum information of a plurality of training speech samples so as to be able to give an output probability value for whether the speech signal is obstructed. The exemplary structure of the neural network that can be deep-learned will be described in more detail below.

[0077] In step 142, the voice signal obstruction threshold can be preset, and when the voice signal obstruction probability output by the voice signal obstruction detection model is greater than the preset voice signal obstruction threshold, it is determined that the voice signal is obstructed by an obstruction. As a non-limiting example, the voice signal obstruction high threshold can be set to 0.7, and the voice signal obstruction low threshold can be set to 0.3, when the voice signal obstruction probability output by the voice signal obstruction detection model is greater than 0.7, it is determined that the voice signal is in an obstructed state (e.g., in a state of wearing a mask), when the voice signal obstruction probability is less than 0.3, it is determined that the voice signal is in a non-obstructed state (e.g., in a state of not wearing a mask), and when the voice signal obstruction probability is between 0.3 and 0.7 (e.g., greater than or equal to 0.3 and less than or equal to 0.7), it is determined that the voice signal is in an uncertain obstructed state. In the above non-limiting example, only when it is determined that the voice signal is in an obstructed state, corresponding compensation measures can be taken.

[0078] Referring to FIG. 6 , which schematically shows a voice signal processing method according to some other exemplary embodiments of the present disclosure in the form of a flow chart, which can be applied to the case where the voice signal is obstructed by an obstruction. FIG. 6 The voice signal processing method 200 shown is basically the same as the voice signal processing method 100 shown in FIG. 2 , and the only difference is that the voice signal processing method 200 further comprises step 150: when it is determined that the voice signal is obstructed by an obstruction, compensating the voice signal.

[0079] The compensation of the voice signal in step 150 is a voice enhancement processing of the voice signal to improve the sound quality. As a non-limiting example, the voice enhancement processing can be an equalizer processing, i.e., acting on the corresponding frequency band according to the gain configuration parameters of each frequency band, so that different frequency bands of the voice signal are enhanced to different degrees. However, it should be understood that any other compensation method of the voice signal is also possible, and the present disclosure does not limit this.

[0080] Referring to FIG. 7 , which further schematically shows the details of step 150 in the voice signal processing method 200 according to some exemplary embodiments of the present disclosure in the form of a flow chart. As FIG. 6 shown, in this exemplary embodiment, step 150 can further comprise steps 151, 152: FIG. 7

[0081] In step 151, based on the pitch frequency, the compensation gain corresponding to each frequency band is determined;

[0082] In step 152, the voice signal is subjected to voice enhancement processing using the compensation gain. ​

[0083] As already mentioned in the foregoing of the present disclosure, the pitch frequency reflects the difference in the sound characteristics of different human voices. Generally speaking, the greater the pitch frequency value of a sound, the higher the proportion of mid-high frequency energy is usually. For example, the pitch frequency value of a female voice signal is usually greater, so the proportion of high frequency energy therein is usually higher than that of a male voice signal. The speech signal processing method 200 can provide corresponding gain parameters for targeted compensation according to the frequency range of the pitch.

[0084] Referring to FIG. 8 According to some example embodiments of the present disclosure, the details of step 151 in the method shown in the flowchart are further illustrated in FIG. 7 As shown in the flowchart, in this example embodiment, step 151 can further include steps 1511, 1512, 1513: FIG. 8 As shown in the flowchart, in this example embodiment, step 1511 can further include steps 1511a, 1511b:

[0085] In step 1511, the pitch frequency range is divided into a plurality of pitch intervals, and the interval gain of each pitch interval in the plurality of pitch intervals is determined;

[0086] In step 1512, the pitch interval in which the pitch frequency is located is determined;

[0087] In step 1513, the interval gain of the pitch interval in which the pitch frequency is located is determined as the compensation gain.

[0088] In step 1511, the pitch frequency range can be divided into a plurality of pitch intervals according to a certain frequency range. As a non-limiting example, the pitch frequency range can be 50 Hz to 500 Hz, and therefore, the pitch frequency range can be divided into 9 pitch intervals according to every 50 Hz. It should be understood that in this example, the pitch frequency range is divided into a plurality of pitch intervals in an equidistant manner, however, the pitch frequency range can also be divided into a plurality of pitch intervals in different manners, for example, the range of some pitch intervals can be different from that of other pitch intervals. After dividing the pitch frequency range into a plurality of pitch intervals, the interval gain of each pitch interval can be determined.

[0089] Referring to FIG. 9 According to some example embodiments of the present disclosure, the details of step 1511 in the method shown in the flowchart are further illustrated in FIG. 8 As shown in the flowchart, in this example embodiment, step 1511 can further include steps 1511a, 1511b: FIG. 9 As shown in the flowchart, in this example embodiment, step 1511a can further include steps 1511a1, 1511a2:

[0090] In step 1511a, for each pitch interval, the average of the power spectrum of each frequency band of the multiple unoccluded speech signal samples and the average of the power spectrum of each frequency band of the multiple occluded speech signal samples are calculated, respectively;

[0091] In step 1511b, the interval gain of each pitch interval is determined based on the ratio of the average of the power spectrum of each frequency band of the multiple unoccluded speech samples and the average of the power spectrum of each frequency band of the multiple occluded speech samples.

[0092] In step 1511a, the multiple unoccluded speech signal samples and the multiple occluded speech signal samples can be classified into pitch intervals according to their pitch frequencies. Taking the above 9 pitch intervals as non-limiting examples, if the pitch frequencies of 1000 speech signal samples (including 500 unoccluded speech signal samples and 500 occluded speech signal samples) fall in the first pitch interval with a frequency range of 50 Hz to 100 Hz, the 1000 speech signal samples are classified into the first pitch interval, and the interval gain of the first pitch interval is calculated based on the 1000 speech signal samples. The frequency point power spectrum damage analysis can be performed based on the 1000 speech signal samples, for example, the average of the power spectrum of each frequency band of the 500 unoccluded speech signal samples and the average of the power spectrum of each frequency band of the 500 occluded speech signal samples may be calculated, where the subscript 1 represents the first pitch interval, k is the frequency band number.

[0093] Therefore, in step 1511b, the interval gain of the first pitch interval corresponding to each frequency band can be determined based on the ratio of the average of the power spectrum of each frequency band . It should be understood that the interval gains of the remaining pitch intervals can be determined in a similar manner, which will not be described here.

[0094] Referring to FIG. 10 , which schematically shows a speech signal processing method according to some other exemplary embodiments of the present disclosure in the form of a flowchart, which can be applied to the case where the speech signal is occluded by an occluder.

[0095] As FIG. 10 ​As shown, the voice signal processing method 300 can start from step 310. In step 310, a voice signal can be recorded by using any suitable device. In step 320, voice activity detection can be performed on the recorded voice signal to obtain a plurality of active voice frames. In step 330, Fourier transform can be performed on the obtained active voice frames to calculate the power spectrum information of each frequency band of each active voice frame. In step 340, the power spectrum information of each frequency band of each active voice frame can be integrated in the low-mid frequency band range to calculate the low-mid frequency band energy value, and each active voice frame can be ranked based on the obtained low-mid frequency band energy value to select at least one representative voice frame. In step 380, pitch frequency detection can be performed on each active voice frame to obtain the pitch frequency of each active voice frame. In step 350, based on the pitch frequency and the power spectrum information of each representative voice frame, voice signal obstruction detection model can be used to detect the voice quality of the voice signal to output the voice signal obstruction probability. In step 360, based on the voice signal obstruction probability, it can be determined whether the voice signal is obstructed by an obstruction, and if it is determined that the voice signal is not obstructed by an obstruction, the voice signal processing method 300 ends, and if it is determined that the voice signal is obstructed by an obstruction, the voice signal processing method 300 proceeds to step 370. In step 370, the voice signal processing method 300 can perform equalization processing on the voice signal by using the equalization parameters obtained in step 390 to perform voice enhancement. It should be understood that each corresponding step in the voice signal processing method 300 can be implemented by using the corresponding method described above with respect to each exemplary embodiment. For example, in step 390, the voice signal processing method 300 can obtain the equalization parameters by using the method described above with respect to each exemplary embodiment. FIG. 8 and FIG. 9 Therefore, the details of each step in the voice signal processing method 300 will not be described again here.

[0096] Referring to FIG. 11 , which schematically shows the effect of voice enhancement processing on a voice signal obstructed by an obstruction in the form of a spectrogram, where figure (a) is a spectrogram of a voice signal not obstructed by an obstruction, figure (b) is a spectrogram of the same voice signal obstructed by an obstruction, and figure (c) is a spectrogram of the same voice signal obstructed by an obstruction after voice enhancement processing. As shown in FIG. 11 , after voice enhancement processing, each frequency band of the voice signal obstructed by an obstruction is enhanced, and the mid-high frequency is significantly improved, which is close to the spectrogram of the voice signal not obstructed by an obstruction, thereby significantly improving the voice quality in terms of hearing and making the voice clearer and easier to understand.

[0097] Referring to FIG. 12Fig. 5 schematically illustrates a structure of a voice signal processing apparatus according to some example embodiments of the present disclosure. The voice signal processing apparatus 500 can be used to determine whether a voice signal is obstructed by an obstruction (e.g., an application scenario of wearing a mask) in a voice call application. As shown in Fig. 5, the voice signal processing apparatus 500 can include a representative voice frame obtaining module 510, a power spectrum information obtaining module 520, a fundamental frequency obtaining module 530, and a voice signal obstruction determination module 540. FIG. 12 The representative voice frame obtaining module 510 is configured to obtain at least one representative voice frame of a voice signal. The power spectrum information obtaining module 520 is configured to obtain power spectrum information corresponding to each representative voice frame of the at least one representative voice frame. The fundamental frequency obtaining module 530 is configured to obtain a fundamental frequency corresponding to each representative voice frame of the at least one representative voice frame. The voice signal obstruction determination module 540 is configured to determine that the voice signal is obstructed by an obstruction when a voice quality of the voice signal is determined to be impaired based on the fundamental frequency and the power spectrum information corresponding to each representative voice frame.

[0098] The representative voice frame obtaining module 510 is configured to obtain at least one representative voice frame of a voice signal. The power spectrum information obtaining module 520 is configured to obtain power spectrum information corresponding to each representative voice frame of the at least one representative voice frame. The fundamental frequency obtaining module 530 is configured to obtain a fundamental frequency corresponding to each representative voice frame of the at least one representative voice frame. The voice signal obstruction determination module 540 is configured to determine that the voice signal is obstructed by an obstruction when a voice quality of the voice signal is determined to be impaired based on the fundamental frequency and the power spectrum information corresponding to each representative voice frame.

[0099] Referring to Fig. 6, a structure of a voice signal processing apparatus according to some example embodiments of the present disclosure is schematically illustrated. FIG. 13 The voice signal processing apparatus 500' shown in Fig. 6 is different from the voice signal processing apparatus 500 shown in Fig. 5 only in that the voice signal processing apparatus 500' further includes a compensation module 550. The compensation module 550 is configured to compensate the voice signal when it is determined that the voice signal is obstructed by an obstruction. As a non-limiting example, the compensation module 550 can be further configured to determine a compensation gain corresponding to each frequency band based on the fundamental frequency, and perform voice enhancement processing on the voice signal using the compensation gain. FIG. 13 FIG. 12 In addition, in some example embodiments, the representative voice frame obtaining module 510 can be further configured to detect a voice quality of the voice signal using a voice signal obstruction detection model based on the fundamental frequency and the power spectrum information corresponding to each representative voice frame to generate a voice signal obstruction probability, and determine that the voice signal is obstructed by an obstruction when the voice signal obstruction probability is greater than a preset voice signal obstruction threshold. The voice signal obstruction detection model can be obtained by training a corresponding neural network using fundamental frequencies and power spectrum information of a plurality of training voice samples.

[0100] In addition, in some example embodiments, the representative voice frame obtaining module 510 can be further configured to detect a voice quality of the voice signal using a voice signal obstruction detection model based on the fundamental frequency and the power spectrum information corresponding to each representative voice frame to generate a voice signal obstruction probability, and determine that the voice signal is obstructed by an obstruction when the voice signal obstruction probability is greater than a preset voice signal obstruction threshold. The voice signal obstruction detection model can be obtained by training a corresponding neural network using fundamental frequencies and power spectrum information of a plurality of training voice samples.

[0101] ​It should be understood that the neural network used to train to generate the voice signal obstruction detection model described above can have any suitable neural network structure known in the art as long as it is capable of achieving the function of determining whether an input voice signal is obstructed by an obstruction based on the pitch frequency and power spectrum information of the voice signal after training with a plurality of training voice samples. For better understanding of the present application, the structure of a neural network is schematically described below as a non-limiting example.

[0102] Referring to FIG. 14 , a structure of a neural network representing the voice signal obstruction detection model implemented in the voice frame obtaining module 510 is schematically shown. As shown in FIG. 14 , the neural network 700 can include a feature concatenation layer 710, a first fully connected layer (i.e., a first FC layer) 720, a first gated recurrent unit layer (i.e., a first GRU layer) 730, a second gated recurrent unit layer (i.e., a second GRU layer) 740, a second fully connected layer (i.e., a second FC layer) 750, and an activation layer 760. The feature concatenation layer 710 receives the pitch frequency and power spectrum information of each representative voice frame and concatenates the input data into a feature vector. The first fully connected layer 720, the first gated recurrent unit layer 730, the second gated recurrent unit layer 740, and the second fully connected layer 750 perform inference based on the feature vector received from the feature concatenation layer 710 and generate an inference output value. The activation layer 760 normalizes the inference output value to a probability value between 0 and 1, which can be used as the voice signal obstruction probability.

[0103] FIG. 14 The structure of the neural network 700 shown is merely exemplary and not limiting. It should be understood that any suitable neural network structure is possible as long as it is capable of being trained with the pitch frequency and power spectrum information of a plurality of training voice samples to obtain the voice signal obstruction detection model. For example, FIG. 14 The neural network 700 shown includes two layers of gated recurrent unit layers, however, the neural network 700 can include fewer (e.g., one) or more (e.g., three or more) layers of gated recurrent unit layers. In addition, in other exemplary embodiments, a long short-term memory layer (i.e., an LSTM layer) can be used instead of the gated recurrent unit layer. As described above, the activation layer 760 is used to normalize the inference output value to a probability value between 0 and 1, and thus the activation function used can be a Sigmoid function or a ReLU function.

[0104] It should be understood that the concept of the present application lies in utilizing the pitch frequency and power spectrum information of the speech signal to determine whether the speech signal is blocked by a blocking object, and to compensate accordingly when the speech signal is blocked by a blocking object. By utilizing the pitch frequency of the speech signal, on the one hand, it is helpful to distinguish the speech uttered by a person from noise, and on the other hand, since the pitch frequency also reflects the characteristics of the person, it is beneficial to more accurately detect the tone quality of the speech signal, and it is also beneficial to more accurately give the corresponding equalization parameters according to the pitch frequency value range to perform equalization adjustment. The changes in the power spectrum information of the speech signal directly reflect the changes in the tone quality of the speech signal, as shown in FIG. 1B Therefore, if it is found that the power spectrum at each frequency point of the speech signal is reduced, it can be determined that the tone quality of the speech signal is impaired. Therefore, by combining the pitch frequency and power spectrum information of the speech signal, the tone quality of the speech signal can be accurately detected, and the tone quality of the speech signal can be compensated accordingly. As a non-limiting example, the pitch frequency range can be divided into a plurality of pitch intervals, the size of the pitch interval can be set according to actual conditions, and then different weight coefficients can be given to the power spectrum information change values of the speech signal at each frequency point according to the pitch interval in which the pitch frequency of the speech signal is located, so as to calculate according to the pitch frequency and power spectrum information using the corresponding weight coefficients, and finally generate the corresponding speech signal blocking probability.

[0105] Therefore, the plurality of training speech samples used for training should include the pitch frequency and power spectrum information of the speech signal, and also include the determination of whether the speech signal is blocked by a blocking object. After training, the neural network described above can obtain the required weight coefficients and bias coefficients, thereby constructing a speech signal blocking detection model. When the pitch frequency and power spectrum information of the actual speech signal are input, the speech signal blocking detection model can generate the speech signal blocking probability accordingly, so as to determine whether the speech signal is blocked by a blocking object.

[0106] It should be understood that the above-described various modules described in combination with FIG. 12 , FIG. 13 respectively relate to the operation of the related steps in the various methods described above in relation to FIG. 2 to FIG. 10 , and thus will not be described again here. In addition, it should also be understood that the above in combination with FIG. 12 , FIG. 13The various modules described can be implemented in hardware or in hardware combined with software and / or firmware. For example, the modules can be implemented as computer executable code / instructions configured to be executed in one or more processors and stored in a computer-readable storage medium. Alternatively, the modules can be implemented as hardware logic / circuitry. For example, in some embodiments, one or more of the modules can be implemented together in a system on chip (SoC). The SoC can include an integrated circuit chip (which includes one or more of a processor (e.g., a central processing unit (CPU), a microcontroller, a microprocessor, a digital signal processor (DSP), etc.), memory, one or more communication interfaces, and / or other circuitry), and can optionally execute received program code and / or include embedded firmware to perform functions.

[0107] Referring to FIG. 15 which schematically illustrates the structure of a computing device 900 according to some embodiments of the present disclosure in the form of a block diagram. The computing device 900 can be used for various application scenarios described in the present disclosure.

[0108] The computing device 900 can include at least one processor 902, memory 904, communication interface(s) 906, display device 908, other input / output (I / O) devices 910, and one or more mass storage devices 912, which are able to communicate with one another, such as over system bus 914 or other appropriate means.

[0109] The processor 902 can be a single processing unit or a plurality of processing units, all of which can include single or multiple computing units or multiple cores. The processor 902 can be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuitries, and / or any devices that manipulate signals based on operational instructions. Among other capabilities, the processor 902 can be configured to fetch and execute computer-readable instructions stored in the memory 904, the mass storage device 912, or any other computer-readable medium, such as program code for an operating system 916, program code for applications 918, program code for other programs 920, etc.

[0110] Memory 904 and mass storage device 912 are examples of computer readable storage media that store instructions which can be executed by processor 902 to implement the various functionalities described above. By way of example, memory 904 can generally include both volatile memory and nonvolatile memory (e.g., RAM, ROM, etc.). Additionally, mass storage device 912 can generally include hard disk drives, solid state drives, removable media, including external and removable drives, memory cards, flash memory, floppy disks, optical disks (e.g., CD, DVD), storage arrays, network attached storage, storage area networks, etc. Memory 904 and mass storage device 912 can be collectively referred to herein as computer readable storage or computer readable storage media, and can be non-transitory media capable of storing computer readable, processor executable program instructions as computer executable code that can be executed by processor 902 as a particular machine configured to implement the operations and functionalities described in the various example embodiments of the present disclosure.

[0111] A number of program modules can be stored by mass storage device 912. These program modules include operating system 916, one or more application programs 918, other programs 920, and program data 922, and they can be executed by processor 902. Examples of such application programs or program modules can include, for example, computer program logic (e.g., computer executable code or instructions) for implementing the representative speech frame acquisition module 510, the power spectral information acquisition module 520, the pitch frequency acquisition module 530, and the speech signal obstruction condition determination module 540, and can also include a compensation module.

[0112] While the application is susceptible to various modifications and alternative forms, specific embodiments have been shown by way of example in FIG. 15The modules 916, 918, 920, and 922, or portions of them, can be implemented using any form of computer-readable media that is accessible by the computing device 900. As used in this document, "computer-readable media" includes both computer storage media and communication media. Computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non- transitory medium that can be used to store information for access by a computing device. Communication media can be embodied in a computer readable instructions, data structures, program modules or other data in a modulated data signal, such as a carrier wave or other transport mechanism. The computer storage media defined by this disclosure does not include communication media.

[0113] The computing device 900 can also include one or more communication interfaces 906 for exchanging data with other devices, such as over a network, direct connection, or the like. The communication interface 906 can facilitate communication over a variety of network and protocol types, including wired networks (e.g., LAN, cable, etc.) and wireless networks (e.g., WLAN, cellular, satellite, etc.), the Internet, and the like. The communication interface 906 can also provide communication with external storage devices (not shown), such as storage arrays, network attached storage, storage area networks, and the like.

[0114] In some examples, the computing device 900 can also include a display device 908, such as a monitor, for displaying information and images. Other I / O devices 910 can be devices that receive input from a user and provide output to the user, including but not limited to touch input devices, gesture input devices, cameras, keyboards, remote controls, mice, printers, audio input / output devices, and the like.

[0115] The terminology used herein is for the purpose of describing embodiments in the disclosure only and is not intended to be limiting of the disclosure. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and "comprising", when used in this disclosure, specify the presence of stated features but do not preclude the presence or addition of one or more other features. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items. It will be understood that, although the terms "first", "second", "third", etc. can be used herein to describe various features, these features should not be limited by these terms. These terms are used only to distinguish one feature from another.

[0116] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and / or the

[0117] In the description of the specification, the description of the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific feature, structure, material or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the disclosure. The illustrative representations of the above terms in the specification are not necessarily directed to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any suitable manner in one or more embodiments or examples. In addition, the skilled person in the art can combine and combine the different embodiments or examples described in the specification and the features of the different embodiments or examples without contradiction.

[0118] Various techniques are described herein in the general context of software hardware elements or program modules. Generally, these modules include routines, programs, objects, elements, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The terms "module", "functionality" and "component" as used herein generally represent software, firmware, hardware or a combination thereof. The features of the techniques described herein are platform-independent, meaning that the techniques can be implemented on a variety of computing platforms having a variety of processors.

[0119] The logic and / or steps represented in flow diagrams or otherwise described herein, for example, can be considered lists of executable instructions to implement logic functions, and can be embodied in any computer-readable medium for use by an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. In the context of this document, a "computer- readable medium" can be any media or means that can contain, store, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The computer-readable medium can be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device. More specific computer-readable medium examples (a non-exhaustive list) can include an electrical connection having a first end and a second end, a random access memory (RAM), a read-only memory (ROM), a red laser disc, a compact disc (CD), a digital versatile disc (DVD), an optical fiber, a portable computer disk, a hard disk, a floppy disk, a mechanical computer processor, or a data bus implementing a serialization communication protocol. The computer-readable medium can also be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, for example, using optical character recognition, or by other suitable means.

[0120] It should be understood that portions of the present disclosure can be implemented in hardware, software, firmware, or combinations thereof. In the above embodiments, a plurality of steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, it can be implemented using any or a combination of the following technologies, which are well known in the art: discrete logic circuitry having logic gates for implementing logic functions upon data signals, application specific integrated circuits having appropriate combinational logic gates, programmable gate arrays, field programmable gate arrays, etc.

[0121] Those of ordinary skill in the art can understand that all or part of the steps of the above-mentioned embodiment methods can be completed by program instruction-related hardware, and the program can be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiments.

[0122] Although the present disclosure has been described in detail with reference to some exemplary embodiments, it is not intended to be limited to the specific form set forth herein. Rather, the scope of the present disclosure is limited only by the claims that follow.

Claims

1. A method for processing a speech signal, comprising: obtaining at least one representative speech frame of a speech signal; obtaining power spectrum information and a pitch frequency corresponding to each of the at least one representative speech frame; detecting a quality of the speech signal based on the pitch frequency and the power spectrum information corresponding to each of the at least one representative speech frame by using a speech signal obstruction detection model to generate a speech signal obstruction probability, wherein the speech signal obstruction detection model is obtained by training a neural network using pitch frequencies and power spectrum information of a plurality of training speech samples; and determining that the speech signal is obstructed when the speech signal obstruction probability is greater than a predetermined speech signal obstruction threshold; and compensating for the speech signal when it is determined that the speech signal is obstructed.

2. The speech signal processing method of claim 1, wherein, The obtaining at least one representative speech frame of a speech signal comprises: obtaining a plurality of active speech frames of the speech signal; performing Fourier transform on the plurality of active speech frames to obtain power spectrum information corresponding to each of the plurality of active speech frames; obtaining a mid-low frequency band energy value corresponding to each of the plurality of active speech frames based on the power spectrum information corresponding to each of the plurality of active speech frames; and determining the at least one representative speech frame from the plurality of active speech frames based on the mid-low frequency band energy value corresponding to each of the plurality of active speech frames.

3. The speech signal processing method of claim 2, wherein, The obtaining a plurality of active speech frames of the speech signal comprises: performing speech activity detection on the speech signal to obtain a plurality of active speech frames of the speech signal, wherein each of the plurality of active speech frames includes human voice and has a predetermined time period length.

4. The speech signal processing method of claim 2, wherein, The determining the at least one representative speech frame from the plurality of active speech frames based on the mid-low frequency band energy value corresponding to each of the plurality of active speech frames comprises: determining an active speech frame corresponding to the mid-low frequency band energy value as a representative speech frame when the mid-low frequency band energy value is greater than a predetermined mid-low frequency band energy threshold.

5. The speech signal processing method of claim 2, wherein, The determining the at least one representative speech frame from the plurality of active speech frames based on the mid-low frequency band energy value corresponding to each of the plurality of active speech frames comprises: sorting the plurality of active speech frames based on the mid-low frequency band energy value corresponding to each of the plurality of active speech frames; and determining the at least one representative speech frame from the plurality of active speech frames based on a result of the sorting.

6. The speech signal processing method of claim 5, wherein, The determining the at least one representative speech frame from the plurality of active speech frames based on the result of the sorting comprises: selecting representative speech frames from the plurality of active speech frames according to a predetermined number in a decreasing order of the mid-low frequency band energy value starting from an active speech frame corresponding to a maximum mid-low frequency band energy value.

7. The speech signal processing method of claim 5, wherein, The determining the at least one representative speech frame from the plurality of active speech frames based on the result of the sorting comprises: selecting representative speech frames from the plurality of active speech frames according to a predetermined percentage in a decreasing order of the mid-low frequency band energy value starting from an active speech frame corresponding to a maximum mid-low frequency band energy value.

8. The speech signal processing method of claim 2, wherein, The obtaining a pitch frequency corresponding to the at least one representative speech frame comprises: pitch frequency detection is performed on the multiple active speech frames to obtain a pitch frequency corresponding to each active speech frame.

9. The voice signal processing method of claim 1, wherein, The compensation of the speech signal when it is determined that the speech signal is blocked by the blocker comprises: Based on the pitch frequency, a compensation gain corresponding to each frequency band is determined; and The speech signal is subjected to speech enhancement processing by using the compensation gain.

10. The speech signal processing method of claim 9, wherein, The determination of the gain corresponding to each frequency band based on the pitch frequency comprises: a pitch frequency range is divided into multiple pitch intervals, and an interval gain of each pitch interval in the multiple pitch intervals is determined; a pitch interval in which the pitch frequency is located is determined; and the interval gain of the pitch interval in which the pitch frequency is located is determined as the compensation gain.

11. The speech signal processing method of claim 10, wherein, The division of the pitch frequency range into multiple pitch intervals and the determination of the interval gain of each pitch interval in the multiple pitch intervals comprise: for each pitch interval, a power spectrum average of each frequency band of multiple frames of unblocked speech signal samples and a power spectrum average of each frequency band of multiple frames of blocked speech signal samples are calculated respectively; and based on a ratio of the power spectrum average of each frequency band of the multiple frames of unblocked speech samples and the power spectrum average of each frequency band of the multiple frames of blocked speech samples, the interval gain of each pitch interval is determined.

12. The speech signal processing method of claim 10, wherein, The multiple intervals are obtained by equally dividing the pitch frequency range.

13. A speech signal processing apparatus, comprising: a representative speech frame acquisition module configured to acquire at least one representative speech frame of a speech signal; a power spectrum information acquisition module configured to acquire power spectrum information corresponding to each representative speech frame in the at least one representative speech frame; a pitch frequency acquisition module configured to acquire a pitch frequency corresponding to each representative speech frame in the at least one representative speech frame; a speech signal blocking condition determination module configured to: based on the pitch frequency and the power spectrum information corresponding to each representative speech frame, detect a quality of the speech signal by using a speech signal blocking detection model to generate a speech signal blocking probability, wherein the speech signal blocking detection model is obtained by training a neural network in the speech signal blocking condition determination module by using pitch frequencies and power spectrum information of multiple training speech samples; and when the speech signal blocking probability is greater than a preset speech signal blocking threshold, it is determined that the speech signal is blocked by a blocker. and a compensation module configured to compensate the speech signal when it is determined that the speech signal is blocked by the blocker.

14. The speech signal processing apparatus according to claim 13, wherein The compensation module is further configured to: based on the pitch frequency, a compensation gain corresponding to each frequency band is determined; and the speech signal is subjected to speech enhancement processing by using the compensation gain.

15. A computing device comprising a processor and a memory, the memory being configured to store computer-executable instructions configured to, when executed on the processor, cause the processor to perform the speech signal processing method according to any one of claims 1 to 12.

16. A computer-readable storage medium configured to store computer-executable instructions configured to, when executed on a processor, cause the processor to perform the method of speech signal processing according to any one of claims 1 to 12.

Images