Voice enhancement method, device and robot

Abstract

The application discloses a speech enhancement method, comprising: acquiring audio information collected by a robot; generating a speech presence probability of speech included in the audio information according to the audio information; and enhancing the audio information according to the speech presence probability to generate enhanced speech. According to the embodiment of the application, the speech presence probability of speech is generated according to the audio information, and the audio information can be enhanced according to the speech presence probability when the speech is enhanced, so that the enhancement effect is improved. In the embodiment of the application, the precision of the speech enhancement algorithm can be improved through the speech presence probability, so that the speech enhancement effect is realized.

Description

Technical Field

[0001] This disclosure relates to the field of robotics, and in particular to a speech enhancement method, apparatus, robot, and storage medium. Background Technology

[0002] With the continuous development of robots, robotic pets are becoming increasingly popular. However, robotic pets, such as legged robots, are constantly moving during voice interaction with humans. Unlike traditional stationary smart devices (such as smart speakers), the constant movement of robotic pets generates a lot of noise, such as the noise from the drive motors and the mechanical transmission noise from the joints during movement. This noise can significantly interfere with voice recognition.

[0003] Therefore, it is necessary to suppress noise interference in the audio collected by the robot in order to enhance the speech. Since a large part of the noise in the robot is generated by the robot's movement and is sudden, traditional speech enhancement algorithms often cannot accurately estimate the probability of speech presence, that is, it is difficult to detect the two endpoints of the speech (i.e., the start point and the end point). Therefore, traditional speech enhancement algorithms cannot effectively enhance speech in the case of sudden noise.

[0004] Therefore, it can be seen that how to enhance the speech to make subsequent speech recognition more accurate in the face of the noise generated by the robot itself has become an urgent problem to be solved. Summary of the Invention

[0005] This disclosure provides a speech enhancement method, apparatus, and robot, as well as a storage medium, for addressing the problem of inaccurate extraction of user speech.

[0006] According to one embodiment of this disclosure, a speech enhancement method is provided, comprising: acquiring audio information of a robot; generating a speech presence probability in the audio information that includes speech based on the audio information; and enhancing the audio information based on the speech presence probability to generate enhanced speech.

[0007] In one embodiment of this disclosure, enhancing the audio information based on the probability of speech presence to generate enhanced speech includes: adjusting the gain function of the enhancement algorithm based on the probability of speech presence; and generating enhanced speech from the audio information using the adjusted enhancement algorithm.

[0008] In one embodiment of this disclosure, generating the probability of speech presence in the audio information, which includes speech, based on the audio information includes: performing noise removal on the audio information to generate denoised frequency information; and generating the probability of speech presence in the audio information, which includes speech, based on the audio information and the denoised frequency information.

[0009] In one embodiment of this disclosure, the step of noise removal of the audio information to generate denoised audio information includes: inputting the audio information into a neural network model to generate the denoised audio information, wherein the neural network model is trained based on noise generated by the robot itself.

[0010] In one embodiment of this disclosure, generating the probability of speech presence in the audio information, which includes speech, based on the audio information and the denoised frequency information includes: generating a first feature value based on the audio information and generating a second feature value based on the denoised frequency information; and generating the probability of speech presence based on the first feature value and the second feature value.

[0011] In one embodiment of this disclosure, generating a first feature value based on the audio information and generating a second feature value based on the denoised audio information includes: performing a root mean square operation on the audio information to generate the first feature value; and performing a root mean square operation on the denoised audio information to generate the second feature value.

[0012] In one embodiment of this disclosure, generating the probability of the presence of the speech based on the first feature value and the second feature value includes: determining whether the first feature value and the second feature value are less than a first preset threshold; if both the first feature value and the second feature value are less than the first preset threshold, then determining that the audio information is noise.

[0013] In one embodiment of this disclosure, the method further includes: if the first feature value or the second feature value is greater than or equal to the first preset threshold, then converting the first feature value and the second feature value into a first non-negative feature value and a second non-negative feature value; and generating the speech presence probability based on the first non-negative feature value and the second non-negative feature value.

[0014] In one embodiment of this disclosure, generating the probability of speech presence based on the first non-negative feature value and the second non-negative feature value includes: obtaining the difference between the first non-negative feature value and the second non-negative feature value; if the difference is less than a second preset threshold, determining that the audio information includes speech; if the difference is greater than or equal to the second preset threshold, generating the probability of speech presence based on the difference and the second non-negative feature value.

[0015] In one embodiment of this disclosure, generating the probability of speech presence based on the difference and the second non-negative feature value includes: dividing the difference by the second non-negative feature value to generate the probability that the audio information is noise; and generating the probability of speech presence based on the probability that the audio information is noise.

[0016] According to another aspect of this disclosure, a voice enhancement device is also provided, comprising: an audio acquisition module for acquiring audio information collected by a robot; a probability generation module for generating a probability of speech presence in the audio information, including speech, based on the audio information; and an enhancement module for enhancing the audio information based on the speech presence probability to generate enhanced speech.

[0017] In one embodiment of this disclosure, the enhancement module includes: a gain function adjustment submodule, used to adjust the gain function of the enhancement algorithm according to the probability of the speech presence; and an enhancement submodule, used to generate enhanced speech from the audio information using the adjusted enhancement algorithm.

[0018] In one embodiment of this disclosure, the probability generation module includes: a noise reduction submodule for performing noise removal on the audio information to generate noise-reduced audio information; and a probability generation submodule for generating the probability of speech presence in the audio information, including speech, based on the audio information and the noise-reduced audio information.

[0019] In one embodiment of this disclosure, the denoising submodule inputs the audio information into a neural network model to generate the denoised audio information, wherein the neural network model is trained based on the robot's own noise.

[0020] In one embodiment of this disclosure, the probability generation submodule includes: a feature value generation unit, configured to generate a first feature value based on the audio information and a second feature value based on the denoised audio information; and a probability generation unit, configured to generate the probability of the speech presence based on the first feature value and the second feature value.

[0021] In one embodiment of this disclosure, the feature value generation unit performs root mean square operation on the audio information to generate the first feature value, and performs root mean square operation on the denoised audio information to generate the second feature value.

[0022] In one embodiment of this disclosure, the probability generation unit determines that the audio information is noise when both the first feature value and the second feature value are less than a first preset threshold.

[0023] In one embodiment of this disclosure, when the first feature value or the second feature value is greater than or equal to the first preset threshold, the probability generation unit converts the first feature value and the second feature value into a first non-negative feature value and a second non-negative feature value, and generates the probability of the speech presence based on the first non-negative feature value and the second non-negative feature value.

[0024] In one embodiment of this disclosure, the probability generation unit obtains the difference generated by subtracting the second non-negative feature value from the first non-negative feature value, and determines that the audio information includes speech when the difference is less than a second preset threshold, and generates the probability of speech presence based on the difference and the second non-negative feature value when the difference is greater than or equal to the second preset threshold.

[0025] In one embodiment of this disclosure, the probability generation unit generates the probability that the audio information is noise by dividing the difference by the second non-negative feature value, and generates the probability of the speech presence based on the probability that the audio information is noise.

[0026] According to another aspect of this disclosure, a robot is also provided, including the apparatus described above.

[0027] According to another aspect of this disclosure, an electronic device is also provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method as described above.

[0028] According to another aspect of this disclosure, a non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the methods described above is also provided.

[0029] According to embodiments of this disclosure, a speech presence probability is generated based on audio information. During speech enhancement, the audio information can be enhanced based on this speech presence probability, thereby improving the enhancement effect. In embodiments of this disclosure, the speech presence probability can improve the accuracy of the speech enhancement algorithm, thus achieving a better speech enhancement effect. In embodiments of this disclosure, because the speech enhancement effect can be improved, voice commands can be accurately extracted, thereby improving the robot's response accuracy.

[0030] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0031] The accompanying drawings are provided for a better understanding of this solution and do not constitute a limitation of this application. Wherein:

[0032] Figure 1 This is a flowchart of a speech enhancement method according to an embodiment of the present disclosure;

[0033] Figure 2 This is a flowchart of a speech presence probability generation method according to an embodiment of the present disclosure;

[0034] Figure 3 This is a flowchart of a speech presence probability generation method according to an embodiment of the present disclosure;

[0035] Figure 4 This is a structural diagram of a speech enhancement device according to an embodiment of the present disclosure;

[0036] Figure 5 This is a block diagram of an electronic device for a speech enhancement method according to an embodiment of the present disclosure. Detailed Implementation

[0037] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of this application, including various details to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0038] like Figure 1 The diagram shows a flowchart of a voice enhancement method according to one embodiment of this disclosure. In one embodiment, the voice enhancement method can run in a robot, such as a robotic pet or a legged robot. Of course, in other embodiments of this disclosure, the method can also run on a user's mobile terminal or a server. In one embodiment, the legged robot includes multiple legs, such as a quadruped robot, a bipedal robot, or other legged robots. The method includes the following steps:

[0039] Step 110: Obtain the audio information collected by the robot.

[0040] In one embodiment of this disclosure, a microphone can be mounted on the body of a robot, such as a legged robot, to collect audio and generate audio information. In other embodiments of this disclosure, multiple microphones, such as six microphones, can be mounted on the robot's body: two microphones located at the front and rear of the body, and four microphones located on the sides of the body. In the embodiments of this disclosure, the robot, particularly the legged robot, is constantly in motion. Therefore, the legged robot may be far from the user and its position may not be fixed. To better receive the user's voice commands, multiple microphones facing different directions are used. Based on the signal-to-noise ratio (SNR) of each microphone, the microphone with the lowest SNR can be selected as the voice input. The microphone with the low SNR may be the microphone closer to the user or the microphone facing the user, thus receiving the user's voice more clearly.

[0041] Step 130: Generate the probability of speech presence in the audio information, which includes speech, based on the audio information.

[0042] In one embodiment of this disclosure, the probability of the speech presence is generated based on audio information, thereby providing a basis for subsequent speech enhancement.

[0043] Step 150: Enhance the audio information based on the probability of speech presence to generate enhanced speech.

[0044] In one embodiment of this disclosure, the gain function of a speech enhancement algorithm can be adjusted based on the speech presence probability, thereby improving the accuracy of the speech enhancement algorithm. First, the gain function of the enhancement algorithm is adjusted according to the speech presence probability, and then the adjusted speech enhancement algorithm is used to generate enhanced speech from the audio information.

[0045] In one embodiment of this disclosure, the speech enhancement algorithm can be the OM-LSA (Optimally-Modified Log-Spectral Amplitude) speech enhancement algorithm. Of course, in other embodiments of this disclosure, other speech enhancement algorithms can also be used. Taking the OM-LSA speech enhancement algorithm as an example, the gain function of the OM-LSA speech enhancement algorithm can be updated using the following formula.

[0046]

[0047] Where G(k,l) is the gain function and p(k,l) is the probability of speech presence.

[0048] like Figure 2The diagram shown is a flowchart of a speech presence probability generation method according to an embodiment of this disclosure. The following steps enable speech recognition and generate the probability of speech presence in the collected audio information.

[0049] Step 210: Obtain the audio information collected by the robot.

[0050] In one embodiment of this disclosure, the audio collected by the robot may contain user speech, environmental noise, and, since the robot is constantly moving, noise generated by its own movement. Examples include noise from the robot's drive motors, joint movements, and the impact of the robot's feet on the ground. If the robot is quiet, noise can be detected by detecting features such as spectrogram flatness, for example, detecting the change from a quiet state to the appearance of speech. However, the noise generated by the robot itself is not stable. Because its movement is not fixed, its noise is abrupt, constituting transient interference. This type of interference makes it very difficult to determine whether the collected audio information includes speech in a speech detection scenario.

[0051] Step 230: Perform speech enhancement on the audio information to generate denoised audio information.

[0052] In one embodiment of this disclosure, audio information can be input into a neural network model to generate denoised audio information, wherein the neural network model is trained based on the robot's own noise. The structure of the neural network model is not limited in this disclosure; DNN (Deep Neural Networks), RNN (Recurrent Neural Networks), CNN (Convolutional Neural Networks), and other neural network structures can all be used.

[0053] In the embodiments of this disclosure, although the noise of the robot itself is sudden, the frequency spectrum of the noise is fixed. For example, the noise from the motor drive, the noise from joint rotation, and the collision noise generated by the robot's feet hitting the ground are basically fixed. Therefore, these noises can be sampled and input into the neural network model for learning. In other embodiments of this disclosure, although different ground surfaces will produce different noises, the ground materials are limited. Therefore, sampling can be performed for different ground materials to improve the neural network model.

[0054] In the embodiments of this disclosure, noise removal of audio information to generate denoised audio information refers to filtering out the noise spectrum of the robot itself learned from the neural network model from the audio information. Since the noise is filtered out, it is referred to here as denoised audio information.

[0055] Step 250: Generate the probability of speech presence in the audio information, which includes speech, based on the audio information and the denoised audio information.

[0056] In one embodiment of this disclosure, noise (the robot's own noise) is removed from the audio information using a neural network model, thus obtaining denoised frequency information. Comparing the denoised frequency information with the audio information yields the probability of speech presence in the audio information. For example, if the audio information contains only noise, the denoised frequency information will be very small relative to the audio information (because most of the noise has been filtered out). Therefore, it can be determined that the probability of speech presence in the audio information is very low, or even zero, meaning it is all noise. Conversely, if the denoised frequency information does not change much relative to the audio information, it indicates that very little noise has been removed, leaving a large amount of speech. Therefore, the probability of speech presence in the audio information will be very high.

[0057] According to embodiments of this disclosure, speech enhancement can be performed on audio information collected by a robot to generate denoised audio information. Furthermore, based on the audio information and the denoised audio information, the probability of speech presence within the audio information is generated, thereby improving the accuracy of speech extraction. The method of this disclosure can effectively and accurately extract user speech from audio information collected by a robot, thus providing a foundation for subsequent speech recognition and other processes.

[0058] like Figure 3 The diagram shown is a flowchart of a speech presence probability generation method according to an embodiment of this disclosure. This embodiment is merely one method for implementing speech presence probability recognition. Those skilled in the art can refer to this method to propose other speech presence probability recognition methods, all of which should be covered by the scope of protection of this invention.

[0059] Step 310: Generate a first feature value based on the audio information, and generate a second feature value based on the denoised audio information.

[0060] In one embodiment of this disclosure, a root mean square (RMS) operation is performed on the audio information to generate a first feature value, and a RMS operation is performed on the denoised frequency information to generate a second feature value. In other embodiments of this disclosure, the RMS operation is merely one implementation method, and other operations can also be used to measure the audio information and the denoised frequency information.

[0061] Step 330: Generate the probability of speech presence based on the first feature value and the second feature value.

[0062] In one embodiment of this disclosure, it can be determined whether the first feature value and the second feature value are less than a first preset threshold. If both the first feature value and the second feature value are less than the first preset threshold, the audio information is determined to be noise. As described above, if both the first feature value and the second feature value are less than the first preset threshold, it indicates that both the audio information and the denoised frequency information are very small, therefore the audio information is noise, i.e., the probability of speech presence is 0.

[0063] In another embodiment of this disclosure, if the first feature value or the second feature value is greater than or equal to a first preset threshold, it indicates that speech may exist in the audio information, and therefore further judgment is required. Specifically, the first feature value and the second feature value are first converted into a first non-negative feature value and a second non-negative feature value, and then the probability of speech presence is generated based on the first non-negative feature value and the second non-negative feature value. In one embodiment of this disclosure, since the first feature value and the second feature value are negative, they need to be converted to the non-negative integer domain. In one embodiment of this disclosure, a preset value can be added to the first feature value to generate the first non-negative feature value, and a preset value can be added to the second feature value to generate the second non-negative feature value. In one embodiment of this disclosure, the preset value can be 100. Of course, in other embodiments of this disclosure, a larger or smaller preset value can be selected as needed.

[0064] In one embodiment of this disclosure, a first non-negative feature value can be subtracted from a second non-negative feature value to generate a difference, which is then used for judgment. If the difference is less than a second preset threshold (e.g., 0), the audio information is determined to include speech, i.e., the probability of speech presence is 1. In this embodiment, if the second non-negative feature value is greater than the first non-negative feature value, the audio information is mainly speech. Conversely, if the difference is greater than or equal to the second preset threshold, the probability of speech presence can be generated based on the difference and the second non-negative feature value. Specifically, the difference is divided by the second non-negative feature value to generate the probability that the audio information is noise, and then the probability of speech presence is generated based on the probability that the audio information is noise, i.e., 1 minus the probability that the audio information is noise, thus obtaining the probability of speech presence.

[0065] In one embodiment of this disclosure, after generating the probability that the audio information is noise, boundary detection is also required to limit the probability of the audio information being noise to a range of 0-1. For example, if the probability of detecting that the audio information is noise is less than 0, it is set to 0; conversely, if the probability of detecting that the audio information is noise is greater than 1, it is set to 1. This can filter out extreme cases and improve accuracy.

[0066] According to embodiments of this disclosure, speech enhancement can be performed on audio information collected by a robot to generate denoised audio information. Furthermore, based on the audio information and the denoised audio information, the probability of speech presence within the audio information is generated, thereby improving the accuracy of speech extraction. The method of this disclosure can effectively and accurately extract user speech from audio information collected by a robot, thus providing a foundation for subsequent speech recognition and other processes.

[0067] like Figure 4 The diagram shown is a structural diagram of a speech enhancement device according to an embodiment of this disclosure. The speech enhancement device 400 includes an audio acquisition module 410, a probability generation module 420, and an enhancement module 430. The audio acquisition module 410 is used to acquire audio information collected by a robot. The probability generation module 420 is used to generate a probability of speech presence in the audio information based on the audio information. The enhancement module 430 is used to enhance the audio information based on the speech presence probability to generate enhanced speech.

[0068] In one embodiment of this disclosure, the enhancement module 430 includes a gain function adjustment submodule 431 and an enhancement submodule 432. The gain function adjustment submodule 431 is used to adjust the gain function of the enhancement algorithm according to the probability of speech presence. The enhancement submodule 432 is used to generate enhanced speech from the audio information using the adjusted enhancement algorithm.

[0069] In one embodiment of this disclosure, the gain function of a speech enhancement algorithm can be adjusted based on the speech presence probability, thereby improving the accuracy of the speech enhancement algorithm. First, the gain function of the enhancement algorithm is adjusted according to the speech presence probability, and then the adjusted speech enhancement algorithm is used to generate enhanced speech from the audio information.

[0070] In one embodiment of this disclosure, the speech enhancement algorithm can be the OM-LSA (Optimally-Modified Log-Spectral Amplitude) speech enhancement algorithm. Of course, in other embodiments of this disclosure, other speech enhancement algorithms can also be used. Taking the OM-LSA speech enhancement algorithm as an example, the gain function of the OM-LSA speech enhancement algorithm can be updated using the following formula.

[0071]

[0072] Where G(k,l) is the gain function and p(k,l) is the probability of speech presence.

[0073] In one embodiment of this disclosure, the probability generation module 420 includes a denoising submodule 421 and a probability generation submodule 422. The denoising submodule 421 is used to remove noise from the audio information to generate denoised audio information. The probability generation submodule 422 is used to generate the probability of speech presence in the audio information, including speech, based on the audio information and the denoised audio information.

[0074] In one embodiment of this disclosure, the denoising submodule 421 inputs audio information into a neural network model to generate denoised audio information, wherein the neural network model is trained based on the robot's own noise.

[0075] In one embodiment of this disclosure, the probability generation submodule 422 includes a feature value generation unit and a probability generation unit. The feature value generation unit is used to generate a first feature value based on audio information and a second feature value based on denoised audio information. The probability generation unit is used to generate the probability of speech presence based on the first feature value and the second feature value.

[0076] In one embodiment of this disclosure, the feature value generation unit performs root mean square operation on the audio information to generate a first feature value, and performs root mean square operation on the denoised audio information to generate a second feature value.

[0077] In one embodiment of this disclosure, the probability generation unit determines that the audio information is noise when both the first feature value and the second feature value are less than a first preset threshold.

[0078] In one embodiment of this disclosure, when the first feature value or the second feature value is greater than or equal to a first preset threshold, the probability generation unit converts the first feature value and the second feature value into a first non-negative feature value and a second non-negative feature value, and generates the probability of speech presence based on the first non-negative feature value and the second non-negative feature value.

[0079] In one embodiment of this disclosure, the probability generation unit obtains the difference generated by subtracting the second non-negative feature value from the first non-negative feature value, and when the difference is less than the second preset threshold, determines that the audio information includes speech, and when the difference is greater than or equal to the second preset threshold, generates the speech existence probability based on the difference and the second non-negative feature value.

[0080] In one embodiment of this disclosure, the probability generation unit generates the probability of audio information being noise by dividing the difference by a second non-negative feature value, and generates the probability of speech presence based on the probability that the audio information is noise.

[0081] In one embodiment of this disclosure, a robot is also disclosed, including the apparatus described above.

[0082] In one embodiment of this disclosure, an electronic device is also disclosed, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method described above. This electronic device may be a robot, such as a legged robot, a user's mobile terminal, a server, or other similar devices.

[0083] In one embodiment of this disclosure, a non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the methods described above is also provided.

[0084] According to embodiments of this disclosure, speech enhancement can be performed on audio information collected by a robot to generate denoised audio information. Furthermore, based on the audio information and the denoised audio information, the probability of speech presence within the audio information is generated, thereby improving the accuracy of speech extraction. The method of this disclosure can effectively and accurately extract user speech from audio information collected by a robot, thus providing a foundation for subsequent speech recognition and other processes.

[0085] Furthermore, according to embodiments of this disclosure, a speech presence probability is generated based on audio information. During speech enhancement, the audio information can be enhanced based on this speech presence probability, thereby improving the enhancement effect. In embodiments of this disclosure, the speech presence probability can improve the accuracy of the speech enhancement algorithm, thus achieving a better speech enhancement effect. In embodiments of this disclosure, because the speech enhancement effect can be improved, voice commands can be accurately extracted, thereby improving the robot's response accuracy.

[0086] like Figure 5 The diagram shown is a block diagram of an electronic device for a voice enhancement method according to an embodiment of this application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present application described and / or claimed herein.

[0087] like Figure 5As shown, the electronic device includes one or more processors Y01, a memory Y02, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components are interconnected via different buses and can be mounted on a common motherboard or otherwise as required. The processors can process instructions executed within the electronic device, including instructions stored in or on memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In other embodiments, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple electronic devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 5 Take processor Y01 as an example.

[0088] The memory Y02 is the non-transitory computer-readable storage medium provided in this application. The memory stores instructions executable by at least one processor to cause the at least one processor to perform the speech enhancement method provided in this application. The non-transitory computer-readable storage medium of this application stores computer instructions for causing a computer to perform the speech enhancement method provided in this application.

[0089] Memory Y02, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the speech enhancement method in the embodiments of this application. Processor Y01 executes various functional applications and data processing of the server by running the non-transitory software programs, instructions, and modules stored in memory Y02, thereby implementing the speech enhancement method in the above method embodiments.

[0090] The memory Y02 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created by the use of the electronic device of the voice enhancement method. Furthermore, the memory Y02 may include high-speed random access memory and may also include non-transient memory, such as at least one disk storage device, flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory Y02 may optionally include memory remotely located relative to the processor Y01, and these remote memories can be connected to the electronic device of the voice enhancement method via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0091] The electronic device for the voice enhancement method may further include an input device Y03 and an output device Y04. The processor Y01, memory Y02, input device Y03, and output device Y04 can be connected via a bus or other means. Figure 5 Taking the example of a connection between China and Israel via a bus.

[0092] Input device Y03 can receive input numerical or character information, as well as generate key signal inputs related to user settings and function control of the electronic device for the voice enhancement method, such as touch screens, keypads, mice, trackpads, touchpads, joysticks, one or more mouse buttons, trackballs, joysticks, etc. Output device Y04 may include display devices, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors). The display device may include, but is not limited to, liquid crystal displays (LCDs), light-emitting diode (LED) displays, and plasma displays. In some embodiments, the display device may be a touch screen.

[0093] Various implementations of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, application-specific integrated circuits (ASICs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transferring data and instructions to the storage system, the at least one input device, and the at least one output device.

[0094] These computational programs (also referred to as programs, software, software applications, or code) include machine instructions for a programmable processor and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms “machine-readable medium” and “computer-readable medium” refer to any computer program product, device, and / or apparatus (e.g., disk, optical disk, memory, programmable logic device (PLD)) used to provide machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term “machine-readable signal” refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0095] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0096] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0097] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system. It addresses the shortcomings of traditional physical hosts and VPS (Virtual Private Server, or simply "VPS") services, such as high management difficulty and weak business scalability.

[0098] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this application can be achieved, and this is not limited herein.

[0099] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A speech enhancement method, characterized in that, include: Acquire audio information collected by the robot; Based on the audio information, generate the probability of the presence of speech, which includes speech, in the audio information; The audio information is enhanced based on the probability of the speech to generate enhanced speech; Based on the audio information, generate the probability of the presence of speech, including speech, within the audio information, including: The audio information is subjected to noise removal to generate denoised audio information; and A first feature value is generated based on the audio information, and a second feature value is generated based on the denoised audio information; Determine whether the first feature value and the second feature value are less than a first preset threshold. If the first feature value or the second feature value is greater than or equal to the first preset threshold, then convert the first feature value and the second feature value into a first non-negative feature value and a second non-negative feature value, and generate the probability of the speech presence based on the first non-negative feature value and the second non-negative feature value.

2. The method as described in claim 1, characterized in that, The step of enhancing the audio information based on the probability of the speech presence to generate enhanced speech includes: The gain function of the enhancement algorithm is adjusted according to the probability of the speech presence. The enhanced audio information is then used to generate enhanced speech using the adjusted enhancement algorithm.

3. The method as described in claim 1, characterized in that, The step of noise removal from the audio information to generate denoised audio information includes: The audio information is input into a neural network model to generate the denoised audio information, wherein the neural network model is trained based on the noise generated by the robot itself.

4. The method as described in claim 1, characterized in that, The step of generating a first feature value based on the audio information and generating a second feature value based on the denoised audio information includes: Perform root mean square operation on the audio information to generate the first feature value; and The root mean square operation is performed on the denoised frequency information to generate the second feature value.

5. The method as described in claim 1, characterized in that, Determining whether the first feature value and the second feature value are less than a first preset threshold further includes: If both the first feature value and the second feature value are less than the first preset threshold, then the audio information is determined to be noise.

6. The method as described in claim 5, characterized in that, The step of generating the speech presence probability based on the first non-negative feature value and the second non-negative feature value includes: Obtain the difference between the first non-negative eigenvalue and the second non-negative eigenvalue; If the difference is less than the second preset threshold, then it is determined that the audio information includes speech; If the difference is greater than or equal to the second preset threshold, the probability of the speech presence is generated based on the difference and the second non-negative feature value.

7. The method as described in claim 6, characterized in that, The step of generating the probability of the speech presence based on the difference and the second non-negative feature value includes: Divide the difference by the second non-negative feature value to generate the probability that the audio information is noise; The probability of the speech being present is generated based on the probability that the audio information is noise.

8. A voice enhancement device, characterized in that, include: The audio acquisition module is used to acquire audio information collected by the robot. The probability generation module is used to generate the probability of the presence of speech, including speech, in the audio information based on the audio information; An enhancement module is used to enhance the audio information based on the probability of the speech presence to generate enhanced speech; The probability generation module includes: A noise reduction submodule is used to perform noise removal on the audio information to generate noise-reduced audio information; and The probability generation submodule is used to generate the probability of speech presence in the audio information, which includes speech, based on the audio information and the denoised audio information. The probability generation submodule includes: The feature value generation unit is used to generate a first feature value based on the audio information and a second feature value based on the denoised audio information. The probability generation unit is used to determine whether the first feature value and the second feature value are less than a first preset threshold. If the first feature value or the second feature value is greater than or equal to the first preset threshold, the first feature value and the second feature value are converted into a first non-negative feature value and a second non-negative feature value, based on the first non-negative feature value and the second non-negative feature value.

9. The apparatus as claimed in claim 8, characterized in that, The enhancement module includes: The gain function adjustment submodule is used to adjust the gain function of the enhancement algorithm according to the probability of the speech presence. An enhancement submodule is used to generate enhanced speech from the audio information by adjusting the enhancement algorithm.

10. The apparatus as claimed in claim 8, characterized in that, The noise reduction submodule inputs the audio information into a neural network model to generate the noise-reduced audio information, wherein the neural network model is trained based on the robot's own noise.

11. The apparatus as claimed in claim 8, characterized in that, The feature value generation unit performs root mean square operation on the audio information to generate the first feature value, and performs root mean square operation on the denoised audio information to generate the second feature value.

12. The apparatus as claimed in claim 8, characterized in that, The probability generation unit determines that the audio information is noise when both the first feature value and the second feature value are less than a first preset threshold.

13. The apparatus as claimed in claim 8, characterized in that, The probability generation unit obtains the difference generated by subtracting the second non-negative feature value from the first non-negative feature value, and determines that the audio information includes speech when the difference is less than a second preset threshold, and generates the probability of speech presence based on the difference and the second non-negative feature value when the difference is greater than or equal to the second preset threshold.

14. The apparatus as claimed in claim 13, characterized in that, The probability generation unit divides the difference by the second non-negative feature value to generate the probability that the audio information is noise, and generates the probability of the speech presence based on the probability that the audio information is noise.

15. A robot, characterized in that, Includes the apparatus as described in any one of claims 8-14.

16. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

17. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-7.

Images