Method for detecting music in speech, method for optimizing speech communication and corresponding device

Abstract

The present application provides a method for detecting music in speech, a method for optimizing speech communication, and a corresponding device, the method comprising: obtaining a speech segment, and performing frame processing on the speech segment to obtain at least two frames of speech signals; obtaining a feature vector of each frame of speech signal, and a feature vector of each frame of historical speech signals in the first number of frames before the frame of speech signal; inputting the feature vector of each frame of speech signal and the feature vector of each corresponding frame of historical speech signals into a trained preset GRU model, and determining whether the frame of speech signal contains music based on the output result of the trained preset GRU model. Since the input of the GRU model includes not only the feature vector of the speech signal to be detected, but also the feature vectors of multiple frames of historical speech signals, the GRU model integrates the features of the historical speech signals during the calculation process, improves the detection accuracy of the single-frame speech signal, and thereby improves the continuity of the speech segment detection results.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method for detecting music in speech, a method for optimizing speech communication, and corresponding devices. Background Art

[0002] In voice communication, it is often necessary to identify the types of voice information contained in the voice in order to adopt the corresponding data processing method and obtain better voice communication effects. Specifically, people often hope to retain or enhance useful signals such as speech and music during voice communication, while suppressing or eliminating useless signals such as noise and reverberation, so it is necessary to detect these types of signals.

[0003] At present, the neural network model (NN) is generally used to detect music in speech. The detection process generally takes the vector corresponding to the frequency domain discrete spectrum of a single frame of speech signal in a speech segment as the input of the neural network model, outputs the probability that the frame of speech signal contains music, and then obtains the music detection results of all the frames of speech signals in the speech segment. However, this method has a low detection accuracy for each frame of speech signal, which leads to discontinuous music detection results in speech segments. Summary of the invention

[0004] The purpose of this application is to solve at least one of the above technical defects. The technical solutions provided by the embodiments of this application are as follows:

[0005] In a first aspect, an embodiment of the present application provides a method for detecting music in speech, comprising:

[0006] Acquire a speech segment, and perform frame processing on the speech segment to obtain at least two frames of speech signals;

[0007] Obtaining a feature vector of each frame of speech signal and a feature vector of each frame of historical speech signal in the first number of frames of historical speech signal before the frame of speech signal;

[0008] Inputting the feature vector of each frame of speech signal and the feature vector of each corresponding frame of historical speech signal into a trained preset gated recurrent convolution unit GRU model, and determining whether the frame of speech signal contains music based on the output result of the trained preset GRU model;

[0009] The trained preset GRU model is obtained by training speech signal samples, and the speech signal samples are annotated with labels indicating whether the speech signal samples contain music.

[0010] In an optional embodiment of the present application, obtaining a feature vector of each frame of speech signal includes:

[0011] Extract the logarithmic Mel spectrum features of each frame of speech signal;

[0012] The logarithmic Mel spectrum features of each frame of speech signal are normalized to obtain the feature vector of the frame of speech signal.

[0013] In an optional embodiment of the present application, the method further includes:

[0014] Obtaining a preset number of speech signal samples;

[0015] The preset GRU model is trained based on each speech signal sample to obtain a trained preset GRU model.

[0016] In an optional embodiment of the present application, the output result of the trained preset GRU model is a first probability that each frame of speech signal contains music;

[0017] Determine whether each frame of speech signal contains music based on the output of the trained preset GRU model, including:

[0018] Based on the first probability that each frame of speech signal contains music and a preset threshold value, it is determined whether the frame of speech signal contains music.

[0019] In an optional embodiment of the present application, the method further includes:

[0020] Obtaining a second probability that each frame of historical speech signals in a second number of frames contains music;

[0021] Determining whether each frame of speech signal contains music based on a first probability that the frame of speech signal contains music and a preset threshold value includes:

[0022] Obtain an average value of a first probability corresponding to each frame of speech signal and a second probability corresponding to each frame of historical speech signal, to obtain an average probability that the frame of speech signal contains music;

[0023] If the average probability is not less than the preset threshold value, it is determined that each frame of the speech signal contains music.

[0024] In a second aspect, an embodiment of the present application provides a voice communication optimization method, including:

[0025] Obtain the current frame voice signal of the near-end device in voice communication;

[0026] Obtaining a feature vector of a current frame speech signal and a feature vector of each frame of historical speech signals in a first number of frames of historical speech signals before the current frame speech signal;

[0027] Inputting a feature vector of a current frame speech signal and feature vectors of corresponding historical speech signals of each frame into a trained preset gated recurrent convolution unit GRU model, and determining whether the current frame speech signal contains music based on an output result of the trained preset GRU model, wherein the trained preset GRU model is obtained by training speech signal samples, and the speech signal samples are annotated with a label for indicating whether the speech signal sample contains music;

[0028] If the current frame of voice signal contains music, the current frame of voice signal is encoded using a preset encoding method and then sent to the opposite end device in the voice communication.

[0029] In an optional embodiment of the present application, encoding the current frame speech signal using a preset encoding method includes:

[0030] A preset encoder is used to encode the current frame speech signal, wherein the order of the preset encoder is higher than the order of an encoder used to encode a speech signal not containing music.

[0031] In an optional embodiment of the present application, if the current frame speech signal contains music, the method includes:

[0032] Deactivate the noise suppression algorithm, and / or deactivate the de-reverberation algorithm.

[0033] In a third aspect, an embodiment of the present application provides a device for detecting music in speech, comprising:

[0034] A voice segment acquisition module is used to acquire voice segments and perform frame processing on the voice segments to obtain at least two frames of voice signals;

[0035] A first feature vector acquisition module is used to acquire a feature vector of each frame of speech signal and a feature vector of each frame of historical speech signal in a first number of frames of historical speech signals before the frame of speech signal;

[0036] The first detection module is used to input the feature vector of each frame of speech signal and the feature vector of each corresponding frame of historical speech signal into a trained preset gated recurrent convolution unit GRU model, and determine whether the frame of speech signal contains music based on the output result of the trained preset GRU model;

[0037] The trained preset GRU model is obtained by training speech signal samples, and the speech signal samples are annotated with labels indicating whether the speech signal samples contain music.

[0038] In an optional embodiment of the present application, the first feature vector acquisition module is specifically used to:

[0039] Extract the logarithmic Mel spectrum features of each frame of speech signal;

[0040] The logarithmic Mel spectrum features of each frame of speech signal are normalized to obtain the feature vector of the frame of speech signal.

[0041] In an optional embodiment of the present application, the device may further include a model training module for:

[0042] Obtaining a preset number of speech signal samples;

[0043] The preset GRU model is trained based on each speech signal sample to obtain a trained preset GRU model.

[0044] In an optional embodiment of the present application, the output result of the trained preset GRU model is a first probability that each frame of speech signal contains music;

[0045] Accordingly, the first detection module is specifically used for:

[0046] Based on the first probability that each frame of speech signal contains music and a preset threshold value, it is determined whether the frame of speech signal contains music.

[0047] In an optional embodiment of the present application, the device may further include a historical voice signal acquisition module, which is used to:

[0048] Obtaining a second probability that each frame of historical speech signals in a second number of frames contains music;

[0049] Accordingly, the first detection module is specifically used for:

[0050] Obtain an average value of a first probability corresponding to each frame of speech signal and a second probability corresponding to each frame of historical speech signal, to obtain an average probability that the frame of speech signal contains music;

[0051] If the average probability is not less than the preset threshold value, it is determined that each frame of the speech signal contains music.

[0052] In a fourth aspect, an embodiment of the present application provides a voice communication optimization device, including:

[0053] A voice signal acquisition module, used to acquire the current frame voice signal of the near-end device in voice communication;

[0054] A second feature vector acquisition module is used to acquire a feature vector of a current frame speech signal and a feature vector of each frame of historical speech signals in a first number of frames of historical speech signals before the current frame speech signal;

[0055] A second detection module is used to input the feature vector of the current frame speech signal and the feature vectors of the corresponding historical speech signals of each frame into a trained preset gated recurrent convolution unit GRU model, and determine whether the current frame speech signal contains music based on the output result of the trained preset GRU model, wherein the trained preset GRU model is obtained by training speech signal samples, and the speech signal samples are annotated with a label for indicating whether the speech signal sample contains music;

[0056] The optimization module is used to encode the current frame voice signal in a preset encoding method and then send it to the opposite end device in the voice communication if the current frame voice signal contains music.

[0057] In an optional embodiment of the present application, the optimization module is specifically used to:

[0058] A preset encoder is used to encode the current frame speech signal, wherein the order of the preset encoder is higher than the order of an encoder used to encode a speech signal not containing music.

[0059] In an optional embodiment of the present application, the device further includes an algorithm deactivation module, configured to:

[0060] If the current frame of speech signal contains music, the noise suppression algorithm is disabled and / or the reverberation elimination algorithm is disabled.

[0061] In a fifth aspect, an embodiment of the present application provides an electronic device, including a memory and a processor;

[0062] A computer program is stored in the memory;

[0063] A processor is used to execute a computer program to implement the method provided in the first aspect embodiment or any optional embodiment of the first aspect, the second aspect embodiment or any optional embodiment of the second aspect.

[0064] In the sixth aspect, an embodiment of the present application provides a computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, it implements the method provided in the embodiment of the first aspect or any optional embodiment of the first aspect, the embodiment of the second aspect or any optional embodiment of the second aspect.

[0065] The beneficial effects of the technical solution provided by this application are:

[0066] The solution provided in the present application adopts a one-way many-to-one GRU model to detect whether a speech signal contains music. Since its input contains not only the feature vector of a frame of speech signal to be detected, but also the feature vectors of multiple frames of historical speech signals, the GRU model integrates the characteristics of the historical speech signal during the calculation process, thereby improving the detection accuracy of a single frame of speech signal, and further improving the continuity of the detection results of multiple frames of speech signals in a speech segment. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in describing the embodiments of the present application are briefly introduced below.

[0068] Figure 1 A flowchart of a method for detecting music in speech provided in an embodiment of the present application;

[0069] Figure 2 Schematic diagram of the feature extraction process in the embodiment of the present application;

[0070] Figure 3 This is a schematic diagram of an optional implementation method in the embodiment of the present application;

[0071] Figure 4 This is a schematic diagram of an example of a test result in another embodiment of the present application;

[0072] Figure 5 A flowchart of a voice communication optimization method in an embodiment of the present application;

[0073] Figure 6 A structural block diagram of a device for detecting music in speech provided in an embodiment of the present application;

[0074] Figure 7 A structural block diagram of a voice communication optimization device provided in an embodiment of the present application;

[0075] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0076] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be interpreted as limiting the present invention.

[0077] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0078] With the research and advancement of artificial intelligence technology, artificial intelligence technology has been studied and applied in many fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned driving, automatic driving, drones, robots, smart medical care, smart customer service, etc. I believe that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0079] The solution provided in the embodiments of the present application involves artificial intelligence computer vision technology, machine learning and other technologies, which are specifically described through the following embodiments. First, several terms are explained and described:

[0080] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0081] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0082] The key technologies of speech technology include automatic speech recognition technology (ASR), text-to-speech technology (TTS) and voiceprint recognition technology. Enabling computers to listen, see, speak and feel is the future development direction of human-computer interaction, among which speech has become one of the most promising human-computer interaction methods in the future.

[0083] Machine Learning (ML) is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

[0084] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0085] First, several terms involved in this application are introduced and explained:

[0086] GRU: Gated Recurrent Unit (GRU) is a gating mechanism in Recurrent Neural Network (RNN). Similar to other gating mechanisms, it aims to solve the gradient vanishing / exploding problem in standard RNN while retaining the long-term information of the sequence. GRU contains a reset gate and an update gate.

[0087] Log Mel spectrum feature: Log Mel spectrum feature is obtained by adding a preset Mel filter function to the spectrum of the speech signal and then taking the logarithm. Since the human ear's perception of sound is not linear, the human ear is more sensitive to low-frequency sounds than to high-frequency sounds, so it is often necessary to convert the linear spectrum into a nonlinear Mel spectrum.

[0088] In voice communication, it is often necessary to identify the category of voice information contained in the voice in order to adopt the corresponding data processing method, thereby obtaining a better voice communication effect. Specifically, people often hope to retain or enhance useful signals such as speech and music during voice communication, while suppressing or eliminating useless signals such as noise and reverberation, so it is necessary to detect these categories of signals. At present, a neural network model (NN) is generally used to detect music in speech. The detection process generally takes the vector corresponding to the frequency domain discrete spectrum of a single frame of speech signal in a speech segment as the input of the neural network model, outputs the probability that the frame of speech signal contains music, and then obtains the music detection results of all frames of speech signals in the speech segment. However, the detection accuracy of this method for each frame of speech signal is not high, which leads to discontinuous music detection results in speech segments. Specifically, the detection results of multiple frames of adjacent speech signals corresponding to a piece of music in a speech segment should all be that they contain music. However, due to the low detection accuracy of this method, some of the detection results of the multiple frames of adjacent speech signals may be judged as not containing music, that is, the detection results of the multiple frames of adjacent speech signals are discontinuous. After subsequent processing based on this discontinuous detection result, the music effect in the speech segment will be discontinuous, affecting the user experience. In view of the above problems, an embodiment of the present application provides a method for detecting music in speech.

[0089] Figure 1 A flowchart of a method for detecting music in speech provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the method may include:

[0090] Step S101, obtaining a speech segment, and performing frame processing on the speech segment to obtain at least two frames of speech signals.

[0091] Among them, the speech signal is not stable at the macro level, but stable at the micro level, that is, it has short-term stability (the speech signal can be considered to be approximately unchanged within 10-30ms). Therefore, the speech signal can be divided into several short segments for processing, and each short segment is called a frame (Chunk).

[0092] Specifically, the duration of each frame of speech signal is set (for example, 20ms), and then sampling is performed at the corresponding sampling frequency, thereby completing the framing operation to obtain multiple frames of speech signals, wherein the time point of each sampling is called the sampling moment. It should be noted that in order to avoid errors caused by framing, each frame can be windowed after framing.

[0093] Step S102, obtaining a feature vector of each frame of speech signal and a feature vector of each frame of historical speech signal in the first number of frames of historical speech signal before the frame of speech signal.

[0094] The feature vector of each frame of speech signal refers to the feature vector corresponding to the frequency domain discrete spectrum of the speech signal. The historical speech signal before each frame of speech signal is the speech signal corresponding to each sampling time before the sampling time of the frame of speech signal.

[0095] Specifically, it is understood that the sampling time of a frame of speech signal to be detected and the sampling time of each frame of historical speech signal in the first number of frames of historical speech signal include at least two sampling times, and the at least two sampling times can be sampling times that are adjacent in the sampling order, or can be non-adjacent sampling times. It is understood that if these multiple sampling times are adjacent in sequence, the accuracy of the output result of the GRU model in the subsequent steps is higher.

[0096] It should be noted that the first frame number can affect the processing speed of the subsequent detection process. The larger the first frame number, the slower the processing speed, and the smaller the first frame number, the faster the processing speed. At the same time, the larger the first frame number, the higher the accuracy of the subsequent GRU model output results. Therefore, the specific value of the first frame number can be set according to actual needs.

[0097] Step S103, input the feature vector of each frame of speech signal and the feature vector of each corresponding frame of historical speech signal into the trained preset gated recurrent convolution unit GRU model, and determine whether the frame of speech signal contains music based on the output result of the trained preset GRU model.

[0098] The trained preset GRU model is obtained by training speech signal samples, and the speech signal samples are annotated with labels indicating whether the speech signal samples contain music.

[0099] Among them, the preset GRU model is a unidirectional many-to-one model. In other words, the input of the preset GRU model is the feature vector of a frame of speech signal to be detected, and multiple feature vectors corresponding to multiple frames of historical speech signals corresponding to the frame of speech signal to be detected. The output of the preset GRU model is the probability that the frame of speech signal to be detected contains music, or the probability that it does not contain music (only contains speech). Then, it can be understood that when training the preset GRU model, the training sample set used includes speech signal samples containing music and speech signal samples not containing music.

[0100] Specifically, after the feature vector of a frame of speech signal to be detected and the feature vector of multiple frames of speech signals corresponding to the frame of speech signal to be detected are input into the trained preset GRU model, the calculation process in the GRU model can be understood as a process of multiple sub-calculations performed in sequence, the input of each sub-calculation includes the feature vector of a frame of speech signal, and the multiple sub-calculations are performed in the order of the sampling moments of the speech signals corresponding to each sub-calculation, and the output of the last sub-calculation is the output result of the GRU model. Specifically, the input of the second to the last sub-calculation is the output result of the previous sub-calculation and the feature vector corresponding to this sub-calculation. It can be seen that each sub-calculation takes into account the characteristics of the previous frame of speech signal. Since the frame of speech signal to be detected corresponds to the last sub-calculation, the calculation process of the frame of speech signal to be detected takes into account the characteristics of all historical speech signals. Based on the short-term stability of the speech signal, the processing process of the GRU model takes into account the characteristics of the historical speech signal. Compared with the solution of only inputting the frame of speech signal to be detected in the prior art, the output result of this solution is more accurate. After obtaining the output result corresponding to each frame of speech signal in the speech segment, a detection result can be obtained by judging whether the frame of speech signal to be detected contains music based on the output result.

[0101] For example, a frame of speech signal to be detected becomes the current frame of speech signal, and the feature vector of the current frame of speech signal and the two feature vectors corresponding to the two frames of historical speech signals corresponding to the current frame of speech signal are used as the input of the trained preset GRU model to output the probability that the frame of speech signal to be detected contains speech. The two frames of historical speech signals are the first frame of speech signal and the second frame of speech signal in the order of sampling time. Then the calculation process in the GRU model is to first use the feature vector of the first frame of speech signal as input to perform a first sub-calculation to obtain a first calculation result, then use the first calculation result and the feature vector of the second frame of speech signal as input to perform a second sub-calculation to obtain a second calculation result, then use the second calculation result and the feature vector of the current frame of speech signal as input to perform a third sub-calculation to obtain a third calculation result, then the third calculation result is the output result of the GRU model, that is, the probability that the current frame of speech signal contains music.

[0102] The solution provided in the present application adopts a one-way many-to-one GRU model to detect whether a speech signal contains music. Since its input contains not only the feature vector of a frame of speech signal to be detected, but also the feature vectors of multiple frames of historical speech signals, the GRU model integrates the characteristics of the historical speech signal during the calculation process, thereby improving the detection accuracy of a single frame of speech signal, and further improving the continuity of the detection results of multiple frames of speech signals in a speech segment.

[0103] In an optional embodiment of the present application, obtaining a feature vector of each frame of speech signal includes:

[0104] Extract the logarithmic Mel spectrum features of each frame of speech signal;

[0105] The logarithmic Mel spectrum features of each frame of speech signal are normalized to obtain the feature vector of the frame of speech signal.

[0106] Specifically, Figure 2 As shown in the figure, for a speech segment in the time domain, the speech segment is firstly framed and windowed, and then each frame of speech signal is subjected to a fast Fourier transform (FFT) to obtain the corresponding discrete frequency domain spectrum. The frequency domain history spectrum corresponding to each frame of speech segment is passed through a Mel spectrum filter bank, and the filtering result is logarithmically operated and normalized once to obtain the feature vector of each frame of speech segment.

[0107] In an optional embodiment of the present application, the method may further include:

[0108] Obtaining a preset number of speech signal samples;

[0109] The preset GRU model is trained based on each speech signal sample to obtain a trained preset GRU model.

[0110] Among them, it can be seen from the previous description that the speech signal samples are divided into speech signal samples containing music and speech signal samples not containing music. A speech signal sample containing music specifically includes a frame of speech-music signal containing music, and the first number of frames of historical speech signals corresponding to the frame of speech signal containing music, and the speech sample carries a label indicating that the speech sample contains music. Similarly, a speech signal sample not containing music specifically includes a frame of speech-music signal not containing music, and the first number of frames of historical speech signals corresponding to the frame of speech signal not containing music, and the speech sample carries a label indicating that the speech sample does not contain music.

[0111] Specifically, a continuous speech segment can be obtained and framed, and it can be determined whether each frame of the speech signal in the speech segment contains speech. The determination method can be manual recognition, and then each frame of the speech signal is marked, and then the corresponding speech signal sample is obtained according to the composition of the speech signal sample containing music and the speech signal sample not containing music. For example, a speech signal containing accompaniment sound or singing sound with accompaniment can be determined as a speech signal containing music, and a speech signal containing only a cappella sound or speaking sound can be determined as a speech signal not containing music.

[0112] It is understandable that the preset number can be set according to actual needs and is not limited here.

[0113] Specifically, during the training process, each speech signal sample is input into the preset GRU model, the output result is compared with the corresponding label, and the parameters of the GRU model are adjusted according to the gap between the output result and the label. If the gap between the output result and the label is less than the preset threshold, the training is considered to be completed and a trained preset GRU model is obtained.

[0114] In an optional embodiment of the present application, the output result of the trained preset GRU model is a first probability that each frame of speech signal contains music;

[0115] Determine whether each frame of speech signal contains music based on the output of the trained preset GRU model, including:

[0116] Based on the first probability that each frame of speech signal contains music and a preset threshold value, it is determined whether the frame of speech signal contains music.

[0117] The output of the GRU model also includes the probability that each frame of speech signal does not contain music.

[0118] Specifically, after obtaining the first probability that a frame of speech signal contains music, the first probability can be compared with a preset threshold value, and if the first probability is not less than the preset threshold value, it is determined that the frame of speech signal contains music. The preset threshold value can be set according to actual needs and is not limited again. For example, the preset threshold value can be set to 0.85, that is, as long as the first probability is greater than 0.85, it can be considered that the frame of speech signal contains music.

[0119] In an optional embodiment of the present application, the method may further include:

[0120] Obtaining a second probability that each frame of historical speech signals in a second number of frames contains music;

[0121] Determining whether each frame of speech signal contains music based on a first probability that the frame of speech signal contains music and a preset threshold value includes:

[0122] Obtain an average value of a first probability corresponding to each frame of speech signal and a second probability corresponding to each frame of historical speech signal, to obtain an average probability that the frame of speech signal contains music;

[0123] If the average probability is not less than the preset threshold value, it is determined that each frame of the speech signal contains music.

[0124] Among them, the GRU model adopted in the scheme of the present application is a many-to-one model. As can be seen from the previous description, the GRU model needs to perform sub-calculations on multiple input feature vectors in sequence during the calculation process, and its calculation complexity is relatively high. In order to ensure the calculation speed in practical applications, the number of input feature vectors can be limited. In other words, the first frame number needs to be limited, and the hidden layers (or hidden nodes) in the GRU model can also be reduced. After adopting the above-mentioned restrictions and node reduction operations, in order to further ensure the accuracy of the detection results, after the GRU model outputs the result and before judging whether it contains music, a probability smoothing operation can be performed on the first probability output by the GRU.

[0125] Among them, it can be known from the foregoing description that the sampling time of a frame of speech signal to be detected and the sampling time of each frame of historical speech signal in the second frame number historical speech signal include at least two sampling times, and these at least two sampling times can be sampling times that are adjacent in sequence in the sampling order, or they can be non-adjacent sampling times. It can be understood that if these multiple sampling times are adjacent in sequence, the effect of subsequent probability smoothing is better. Among them, the second frame number is greater than the first frame number, and the specific value of the second frame number can be determined according to actual conditions, and is not limited here.

[0126] It is understandable that the second probability corresponding to the historical speech signal may be obtained by a trained preset GRU model, wherein a speech signal with a very small second probability may correspond to silence or low-energy background sound.

[0127] Specifically, the first probability corresponding to the one-frame speech signal to be detected is averaged with the second probability corresponding to each historical speech signal in the historical speech signal of the second frame number, and the average probability of the one-frame speech signal to be detected is obtained. Based on the stability of the speech signal segment, it can be considered that the historical speech signal and the one-frame speech signal to be detected are a stable speech segment, so the average probability can more accurately reflect the probability that the one-frame speech signal to be detected contains music. Therefore, the average probability is compared with the preset threshold value. If the average probability is not less than the preset threshold value, it is considered that the one-frame speech signal to be detected contains music. It can be understood that probability smoothing can correct the first probability output by the GRU model. For example, the one-frame speech signal to be detected actually contains music, but the first probability output by the GRU model is less than the threshold value because of the error. If the probability is not smoothed, the frame of speech signal to be detected will be judged as not containing music. Then the corresponding average probability obtained after probability smoothing will be greater than the threshold value, so that the frame of speech signal to be detected is judged as containing music, ensuring the continuity of the detection result. For another example, a frame of speech signal to be detected corresponds to the intermittent time in the music (i.e., the time period without sound in the middle of the music), and the first probability output by the GRU model will also be less than the threshold value. If it is not smoothed by probability, the frame of speech signal to be detected will be judged as not containing music. Then, the corresponding average probability obtained after probability smoothing will be greater than the threshold value, so that the frame of speech signal to be detected will be judged as containing music, thus ensuring the continuity of the detection results.

[0128] Furthermore, if Figure 3 As shown, a complete optional scheme of the present application is shown in the figure. For a given speech segment, the Mel spectrum feature is first extracted. For a frame of speech signal to be detected, the feature vector of the frame of speech signal to be detected and the feature vectors of each frame of historical speech signal need to be extracted. The feature extraction step is completed (the feature vector corresponding to each speech signal is identified by comparing the feature vectors in the figure). These feature vectors are input into the trained GRU model, and then the first probability corresponding to the frame of speech to be detected output by the GRU model is probability smoothed to obtain the corresponding average probability. Finally, the average probability is threshold judged, that is, the average probability is compared with the threshold value. If the average probability is greater than the threshold value, it is determined that the frame of speech signal to be detected contains music.

[0129] Figure 4 This is an example of the detection results of music in speech in the present application. The detection results of the music clip are shown in the figure, wherein there are two time periods corresponding to the speech containing music from 0 to 10 s, two time periods corresponding to the speech containing music from 10 to 20 s, and one time period corresponding to the speech containing music from 20 to 25 s.

[0130] Figure 5A flow chart of a voice communication optimization method in an embodiment of the present application is shown in FIG. Figure 5 As shown, the method may include:

[0131] Step S501, obtaining a current frame voice signal of a near-end device in voice communication.

[0132] Specifically, since it is necessary to determine in real time whether each frame of speech signal contains music in speech communication and adopt subsequent processing strategies after the determination, the current frame of speech signal in this step is a frame of speech signal to be detected.

[0133] Step S502: Acquire a feature vector of the current frame speech signal and a feature vector of each frame of historical speech signals in a first number of frames of historical speech signals before the current frame speech signal.

[0134] Step S503, input the feature vector of the current frame speech signal and the feature vectors of the corresponding historical speech signals of each frame into a trained preset gated recurrent convolution unit GRU model, and determine whether the current frame speech signal contains music based on the output result of the trained preset GRU model, wherein the trained preset GRU model is obtained by training speech signal samples, and the speech signal samples are annotated with a label for indicating whether the speech signal sample contains music.

[0135] Specifically, the processing of step S502 and step S503 is the same as the processing of a frame of speech signal to be detected in the foregoing text, and will not be repeated here.

[0136] Step S504: If the current frame voice signal contains music, the current frame voice signal is encoded using a preset encoding method and then sent to a peer device in voice communication.

[0137] Specifically, when it is determined that the current frame voice signal contains music, in order to improve the user experience, the voice signal containing music is encoded using a preset encoding method, that is, the voice signal containing music is encoded with high quality to distinguish it from the voice signal not containing music, thereby improving the listening experience.

[0138] The solution provided in the present application adopts a one-way many-to-one GRU model to detect whether a speech signal contains music. Since its input contains not only the feature vector of a frame of speech signal to be detected, but also the feature vectors of multiple frames of historical speech signals, the GRU model integrates the characteristics of the historical speech signal during the calculation process, thereby improving the detection accuracy of a single frame of speech signal, and further improving the continuity of the detection results of multiple frames of speech signals in a speech segment, facilitating the subsequent real-time processing of speech signals containing music, and improving the user experience.

[0139] In an optional embodiment of the present application, encoding the current frame speech signal using a preset encoding method includes:

[0140] A preset encoder is used to encode the current frame speech signal, wherein the order of the preset encoder is higher than the order of an encoder used to encode a speech signal not containing music.

[0141] Specifically, for voice signals, the quality of the encoding determines the quality of the voice signal obtained after decoding. The most direct way to improve the encoding quality is to increase the order of the encoder used. The higher the order, the higher the encoding quality. Therefore, in order to ensure that the music quality in the voice signal decoded by the peer device is high, the voice signal containing music obtained by the proximal device can be encoded using a high-order encoder.

[0142] In an optional embodiment of the present application, if the current frame speech signal contains music, the method includes:

[0143] Deactivate the noise suppression algorithm, and / or deactivate the de-reverberation algorithm.

[0144] Specifically, when it is determined that the current frame speech signal contains music, the noise suppression algorithm may not be enabled to avoid discontinuity of the music segment, and the dereverberation elimination algorithm may not be enabled to ensure the listening experience.

[0145] It should be noted that the above voice communication optimization method can be used to optimize voice communication in an audio conference or a video conference. Specifically, the optimization process may include:

[0146] (1) A user starts an audio conference or video conference application on a terminal device (such as a mobile phone or a personal computer). After the conference starts, the real-time terminal device collects the current frame voice signal input by the user through a sound receiving device (such as a microphone).

[0147] (2) The terminal device sends the current frame voice signal to the corresponding server, and the server obtains the feature vector of the current frame voice signal and the feature vectors of each frame of historical voice signals in the first number of frames before the current frame voice signal; then the feature vector of the current frame voice signal and the feature vectors of each corresponding frame of historical voice signals are input into a trained preset gated recurrent convolution unit GRU model, and based on the output result of the trained preset GRU model, it is determined whether the current frame voice signal contains music.

[0148] (3) For voice signals containing music, the server processes them accordingly (including high-quality encoding, shutting down noise suppression algorithms and / or reverberation elimination algorithms, etc.) and then sends them to the peer device.

[0149] After optimizing the voice input by each participating user in the voice conference or video conference through the above steps, the heightening effect of the music in the conference can be improved, thereby enhancing the user experience.

[0150] Figure 6 A structural block diagram of a device for detecting music in speech provided in an embodiment of the present application, such as Figure 6 As shown, the device 600 may include: a speech segment acquisition module 601, a first feature vector acquisition module 602 and a first detection module 603, wherein:

[0151] The speech segment acquisition module 601 is used to acquire the speech segment and perform frame processing on the speech segment to obtain at least two frames of speech signals;

[0152] The first feature vector acquisition module 602 is used to acquire the feature vector of each frame of speech signal and the feature vector of each frame of historical speech signal in the first number of frames of historical speech signal before the frame of speech signal;

[0153] The first detection module 603 is used to input the feature vector of each frame of speech signal and the feature vector of each corresponding frame of historical speech signal into the trained preset gated recurrent convolution unit GRU model, and determine whether the frame of speech signal contains music based on the output result of the trained preset GRU model;

[0154] The trained preset GRU model is obtained by training speech signal samples, and the speech signal samples are annotated with labels indicating whether the speech signal samples contain music.

[0155] The solution provided in the present application adopts a one-way many-to-one GRU model to detect whether a speech signal contains music. Since its input contains not only the feature vector of a frame of speech signal to be detected, but also the feature vectors of multiple frames of historical speech signals, the GRU model integrates the characteristics of the historical speech signal during the calculation process, thereby improving the detection accuracy of a single frame of speech signal, and further improving the continuity of the detection results of multiple frames of speech signals in a speech segment.

[0156] In an optional embodiment of the present application, the first feature vector acquisition module is specifically used to:

[0157] Extract the logarithmic Mel spectrum features of each frame of speech signal;

[0158] The logarithmic Mel spectrum features of each frame of speech signal are normalized to obtain the feature vector of the frame of speech signal.

[0159] In an optional embodiment of the present application, the device may further include a model training module for:

[0160] Obtaining a preset number of speech signal samples;

[0161] The preset GRU model is trained based on each speech signal sample to obtain a trained preset GRU model.

[0162] In an optional embodiment of the present application, the output result of the trained preset GRU model is a first probability that each frame of speech signal contains music;

[0163] Accordingly, the first detection module is specifically used for:

[0164] Based on the first probability that each frame of speech signal contains music and a preset threshold value, it is determined whether the frame of speech signal contains music.

[0165] In an optional embodiment of the present application, the device may further include a historical voice signal acquisition module, which is used to:

[0166] Obtaining a second probability that each frame of historical speech signals in a second number of frames contains music;

[0167] Accordingly, the first detection module is specifically used for:

[0168] Obtain an average value of a first probability corresponding to each frame of speech signal and a second probability corresponding to each frame of historical speech signal, to obtain an average probability that the frame of speech signal contains music;

[0169] If the average probability is not less than the preset threshold value, it is determined that each frame of the speech signal contains music.

[0170] Figure 7 A structural block diagram of a voice communication optimization device provided in an embodiment of the present application, such as Figure 7 As shown, the device 700 may include: a speech signal acquisition module 701, a second feature vector acquisition module 702, a second detection module 703 and an optimization module 704, wherein:

[0171] The voice signal collection module 701 is used to collect the current frame voice signal of the near-end device in voice communication;

[0172] The second feature vector acquisition module 702 is used to acquire the feature vector of the current frame speech signal and the feature vector of each frame of historical speech signal in the first number of frames of historical speech signal before the current frame speech signal;

[0173] The second detection module 703 is used to input the feature vector of the current frame speech signal and the feature vectors of the corresponding historical speech signals of each frame into the trained preset gated recurrent convolution unit GRU model, and determine whether the current frame speech signal contains music based on the output result of the trained preset GRU model, wherein the trained preset GRU model is obtained by training speech signal samples, and the speech signal samples are annotated with a label for indicating whether the speech signal sample contains music;

[0174] The optimization module 704 is used to encode the current frame of voice signal using a preset encoding method and then send it to the opposite end device in the voice communication if the current frame of voice signal contains music.

[0175] The solution provided in the present application adopts a one-way many-to-one GRU model to detect whether a speech signal contains music. Since its input contains not only the feature vector of a frame of speech signal to be detected, but also the feature vectors of multiple frames of historical speech signals, the GRU model integrates the characteristics of the historical speech signal during the calculation process, thereby improving the detection accuracy of a single frame of speech signal, and further improving the continuity of the detection results of multiple frames of speech signals in a speech segment, facilitating the subsequent real-time processing of speech signals containing music, and improving the user experience.

[0176] Based on the same principle, an embodiment of the present application further provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method provided in any optional embodiment of the present application is implemented, and specifically the following situations can be implemented:

[0177] Case 1: A speech segment is obtained, and the speech segment is divided into frames to obtain at least two frames of speech signals; a feature vector of each frame of speech signal and a feature vector of each frame of historical speech signal in the first number of frames before the frame of speech signal are obtained; the feature vector of each frame of speech signal and the feature vector of each corresponding frame of historical speech signal are input into a trained preset gated recurrent convolution unit GRU model, and whether the frame of speech signal contains music is determined based on the output result of the trained preset GRU model; wherein the trained preset GRU model is obtained by training speech signal samples, and the speech signal samples are annotated with a label indicating whether the speech signal sample contains music.

[0178] Case 2: Obtain a current frame voice signal of a near-end device in voice communication; obtain a feature vector of the current frame voice signal, and a feature vector of each frame of historical voice signals in the first number of frames before the current frame voice signal; input the feature vector of the current frame voice signal and the feature vectors of each corresponding frame of historical voice signals into a trained preset gated recurrent convolution unit GRU model, and determine whether the current frame voice signal contains music based on the output result of the trained preset GRU model, wherein the trained preset GRU model is obtained by training voice signal samples, and the voice signal samples are annotated with a label for indicating whether the voice signal sample contains music; if the current frame voice signal contains music, the current frame voice signal is encoded using a preset encoding method and sent to the opposite device in the voice communication.

[0179] An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the method shown in any embodiment of the present application is implemented.

[0180] It is understandable that what is stored in the medium may be a computer program corresponding to the method for detecting music in speech or the method for optimizing speech communication.

[0181] Figure 8 A schematic diagram of the structure of an electronic device applicable to an embodiment of the present application is shown in FIG. Figure 8 As shown, Figure 8 The electronic device 800 shown includes: a processor 801 and a memory 803. The processor 801 and the memory 803 are connected, such as through a bus 802. Further, the electronic device 800 may also include a transceiver 804, and the electronic device 800 may interact with other electronic devices through the transceiver 804. It should be noted that in actual applications, the transceiver 804 is not limited to one, and the structure of the electronic device 800 does not constitute a limitation on the embodiments of the present application.

[0182] The processor 801 is used in the embodiment of the present application to implement Figure 6 The function of the device for detecting music in speech shown, or for realizing Figure 7 The functions of the voice communication optimization device are shown.

[0183] The processor 801 may be a CPU, a general-purpose processor, a DSP, an ASIC, an FPGA or other programmable logic device, a transistor logic device, a hardware component or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. The processor 801 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0184] The bus 802 may include a path to transmit information between the above components. The bus 802 may be a PCI bus or an EISA bus, etc. The bus 802 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0185] The memory 803 can be a ROM or other type of static storage device that can store static information and instructions, a RAM or other type of dynamic storage device that can store information and instructions, or an EEPROM, CD-ROM or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to these.

[0186] The memory 803 is used to store the application code for executing the solution of the present application, and the execution is controlled by the processor 801. The processor 801 is used to execute the application code stored in the memory 803 to implement Figure 6 The illustrated embodiment provides an apparatus for detecting music in speech, or Figure 7 The illustrated embodiment provides actions of a voice communication optimization device.

[0187] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0188] The above are only some implementation methods of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for detecting music in speech, characterized in that: include: Acquire a speech segment, and perform frame processing on the speech segment to obtain at least two frames of speech signals; Obtaining a feature vector of each frame of speech signal and a feature vector of each frame of historical speech signal in the first number of frames of historical speech signal before the frame of speech signal; For each frame of speech signal, the feature vector of the frame of speech signal and the feature vectors of each frame of historical speech signal corresponding to the frame of speech signal are input into a trained preset gated recurrent convolution unit GRU model; Based on the sampling moments of the frame of speech signal and the corresponding frames of historical speech signals, multiple sub-calculations are performed on the input frame of speech signal and the input frames of historical speech signals frame by frame through the preset gated recurrent convolution unit GRU model to obtain a first probability that the frame of speech signal contains music; the input of one sub-calculation includes the frame of speech signal and one frame of speech signal in each frame of historical speech signal, and the calculation result of the last sub-calculation, and the calculation result of the last sub-calculation output is the first probability of the frame of speech signal; Based on the sampling time of each frame of the speech signal, according to a second frame number greater than the first frame number, obtaining a second probability that each frame of the historical speech signal of the second frame number contains music; the sampling times corresponding to each frame of the speech signal and the historical speech signal of the second frame number are adjacent in sampling order, and the second probability is obtained by the preset gated recurrent convolution unit GRU model; Obtain an average value of a first probability corresponding to each frame of speech signal and a second probability corresponding to each frame of historical speech signal of a second number of frames, and obtain an average probability that each frame of speech signal contains music; if the average probability is not less than a preset threshold value, it is determined that each frame of speech signal contains music; The preset gated recurrent convolutional unit (GRU) model is trained by the following steps: Obtaining a preset number of speech signal samples; Wherein, the preset number of speech signal samples includes speech signal samples containing music and speech signal samples not containing music; A speech signal sample containing music includes a frame of speech music signal containing music and a first number of frames of historical speech signals corresponding to the frame of speech signal containing music, and the speech signal sample corresponds to a label indicating that the speech signal sample contains music; A speech signal sample not containing music includes a frame of speech music signal not containing music, and a first number of frames of historical speech signals corresponding to the frame of speech signal not containing music, and the speech signal sample corresponds to a label indicating that the speech signal sample does not contain music; The preset gated recurrent convolution unit GRU model is trained based on each speech signal sample to obtain a trained preset gated recurrent convolution unit GRU model.

2. The method according to claim 1, characterized in that The step of obtaining a feature vector of each frame of speech signal comprises: Extract the logarithmic Mel spectrum features of each frame of speech signal; The logarithmic Mel spectrum features of each frame of speech signal are normalized to obtain the feature vector of the frame of speech signal.

3. A voice communication optimization method, characterized in that: include: Obtain the current frame voice signal of the near-end device in voice communication; Acquire a feature vector of the current frame speech signal and a feature vector of each frame of historical speech signals in a first number of frames of historical speech signals before the current frame speech signal; Inputting the feature vector of the current frame speech signal and the feature vectors of the corresponding historical frames of speech signals into a trained preset gated recurrent convolution unit GRU model; Based on the sampling moments of the current frame speech signal and the corresponding historical frames speech signals, the preset gated recurrent convolution unit GRU model sequentially performs multiple sub-calculations on the input current frame speech signal and the historical frames speech signals frame by frame to obtain a first probability that the current frame speech signal contains music; the input of one sub-calculation includes the current frame speech signal and one frame speech signal in each frame historical speech signal, and the calculation result of the last sub-calculation, and the calculation result of the last sub-calculation output is the first probability of the current frame speech signal; Based on the sampling time of the current frame speech signal, according to a second frame number greater than the first frame number, obtaining a second probability that each frame of the historical speech signal of the second frame number contains music; the sampling times corresponding to the current frame speech signal and the historical speech signal of the second frame number are adjacent in sampling order, and the second probability is obtained by the preset gated recurrent convolution unit GRU model; Obtain an average value of a first probability corresponding to a current frame of speech signal and a second probability corresponding to each frame of historical speech signal of a second number of frames, and obtain an average probability that the current frame of speech signal contains music; if the average probability is not less than a preset threshold value, it is determined that the current frame of speech signal contains music; If the current frame voice signal contains music, the current frame voice signal is encoded using a preset encoding method and then sent to the opposite end device in the voice communication; The preset gated recurrent convolutional unit (GRU) model is trained by the following steps: Obtaining a preset number of speech signal samples; Wherein, the preset number of speech signal samples includes speech signal samples containing music and speech signal samples not containing music; A speech signal sample containing music includes a frame of speech music signal containing music and a first number of frames of historical speech signals corresponding to the frame of speech signal containing music, and the speech signal sample corresponds to a label indicating that the speech signal sample contains music; A speech signal sample not containing music includes a frame of speech music signal not containing music, and a first number of frames of historical speech signals corresponding to the frame of speech signal not containing music, and the speech signal sample corresponds to a label indicating that the speech signal sample does not contain music; The preset gated recurrent convolution unit GRU model is trained based on each speech signal sample to obtain a trained preset gated recurrent convolution unit GRU model.

4. The method according to claim 3, characterized in that The encoding of the current frame speech signal using a preset encoding method includes: The current frame speech signal is encoded using a preset encoder, wherein the order of the preset encoder is higher than the order of an encoder used to encode a speech signal that does not contain music.

5. The method according to any one of claims 3 to 4, characterized in that: If the current frame speech signal contains music, the method includes: Deactivate the noise suppression algorithm, and / or deactivate the de-reverberation algorithm.

6. A device for detecting music in speech, characterized in that: include: A voice segment acquisition module, used to acquire a voice segment and perform frame processing on the voice segment to obtain at least two frames of voice signals; A first feature vector acquisition module is used to acquire a feature vector of each frame of speech signal and a feature vector of each frame of historical speech signal in a first number of frames of historical speech signals before the frame of speech signal; The first detection module is used to input the feature vector of each frame of speech signal and the feature vector of each frame of historical speech signal corresponding to the frame of speech signal into a trained preset gated recurrent convolution unit GRU model; based on the sampling time of the frame of speech signal and the corresponding frames of historical speech signals, the preset gated recurrent convolution unit GRU model sequentially performs multiple sub-calculations on the input frame of speech signal and each frame of historical speech signal frame by frame to obtain a first probability that the frame of speech signal contains music; the input of one sub-calculation includes the frame of speech signal and one frame of speech signal in each frame of historical speech signal, and the calculation result of the last sub-calculation, and the calculation result output by the last sub-calculation is the first probability of the frame of speech signal; A first historical voice signal acquisition module is used to obtain, based on the sampling time of each frame of the voice signal, a second probability that each frame of the historical voice signal in the second frame number contains music according to a second frame number greater than the first frame number; the sampling times corresponding to each frame of the voice signal and the second frame number are adjacent in sampling order, and the second probability is obtained by the preset gated recurrent convolution unit GRU model; The first detection module is further used to obtain an average value of a first probability corresponding to each frame of speech signal and a second probability corresponding to each frame of historical speech signal of a second number of frames, to obtain an average probability that each frame of speech signal contains music; if the average probability is not less than a preset threshold value, it is determined that each frame of speech signal contains music; The preset gated recurrent convolutional unit (GRU) model is trained by the following steps: Obtaining a preset number of speech signal samples; Wherein, the preset number of speech signal samples includes speech signal samples containing music and speech signal samples not containing music; A speech signal sample containing music includes a frame of speech music signal containing music and a first number of frames of historical speech signals corresponding to the frame of speech signal containing music, and the speech signal sample corresponds to a label indicating that the speech signal sample contains music; A speech signal sample not containing music includes a frame of speech music signal not containing music, and a first number of frames of historical speech signals corresponding to the frame of speech signal not containing music, and the speech signal sample corresponds to a label indicating that the speech signal sample does not contain music; The preset gated recurrent convolution unit GRU model is trained based on each speech signal sample to obtain a trained preset gated recurrent convolution unit GRU model.

7. The device according to claim 6, characterized in that The first feature vector acquisition module is used for: Extract the logarithmic Mel spectrum features of each frame of speech signal; The logarithmic Mel spectrum features of each frame of speech signal are normalized to obtain the feature vector of the frame of speech signal.

8. A voice communication optimization device, characterized in that: include: A voice signal acquisition module, used to acquire the current frame voice signal of the near-end device in voice communication; A second feature vector acquisition module is used to acquire the feature vector of the current frame speech signal and the feature vector of each frame of historical speech signal in the first number of frames of historical speech signals before the current frame speech signal; The second detection module is used to input the feature vector of the current frame speech signal and the feature vector of each corresponding frame historical speech signal into a trained preset gated recurrent convolution unit GRU model; based on the sampling time of the current frame speech signal and the corresponding frame historical speech signal, the preset gated recurrent convolution unit GRU model sequentially performs multiple sub-calculations on the input current frame speech signal and each frame historical speech signal frame by frame to obtain a first probability that each frame speech signal contains music; the input of a sub-calculation includes the current frame speech signal and one frame speech signal in each frame historical speech signal, and the calculation result of the last sub-calculation, and the calculation result output by the last sub-calculation is the first probability of the current frame speech signal; A second historical voice signal acquisition module is used to obtain, based on the sampling time of the current frame voice signal, a second probability that each frame of the historical voice signal of the second frame number contains music according to a second frame number greater than the first frame number; the sampling times corresponding to the current frame voice signal and the second frame number are adjacent in sampling order, and the second probability is obtained by the preset gated recurrent convolution unit GRU model; The second detection module is further used to obtain an average value of a first probability corresponding to a current frame of speech signal and a second probability corresponding to each frame of historical speech signal of a second number of frames, to obtain an average probability that the current frame of speech signal contains music; if the average probability is not less than a preset threshold value, it is determined that the current frame of speech signal contains music; An optimization module, configured to encode the current frame voice signal in a preset encoding manner and then send the encoded signal to a peer device in voice communication if the current frame voice signal contains music; The preset gated recurrent convolutional unit (GRU) model is trained by the following steps: Obtaining a preset number of speech signal samples; Wherein, the preset number of speech signal samples includes speech signal samples containing music and speech signal samples not containing music; A speech signal sample containing music includes a frame of speech music signal containing music and a first number of frames of historical speech signals corresponding to the frame of speech signal containing music, and the speech signal sample corresponds to a label indicating that the speech signal sample contains music; A speech signal sample not containing music includes a frame of speech music signal not containing music, and a first number of frames of historical speech signals corresponding to the frame of speech signal not containing music, and the speech signal sample corresponds to a label indicating that the speech signal sample does not contain music; The preset gated recurrent convolution unit GRU model is trained based on each speech signal sample to obtain a trained preset gated recurrent convolution unit GRU model.

9. The device according to claim 8, characterized in that The optimization module is used to: The current frame speech signal is encoded using a preset encoder, wherein the order of the preset encoder is higher than the order of an encoder used to encode a speech signal that does not contain music.

10. The device according to any one of claims 8 or 9, characterized in that If the current frame speech signal contains music, the optimization module is further used to: Deactivate the noise suppression algorithm, and / or deactivate the de-reverberation algorithm.

11. An electronic device, characterized in that: including memory and processor; The memory stores a computer program; The processor is configured to execute the computer program to implement the method according to any one of claims 1 to 5.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images