Interactive method with voice noise reduction, computer device, storage medium and system

Abstract

The application provides an interactive method with voice noise reduction, a computer device, a storage medium and a system. The interactive method with voice noise reduction comprises the following steps: obtaining a user voice signal picked up by a voice collector; performing voice noise reduction on the user voice signal; uploading the user voice signal after voice noise reduction to a voice recognition cloud; obtaining semantic information obtained after processing by the voice recognition cloud; and generating a task code based on the semantic information, the task code being used to be transmitted to an executing mechanism of an indoor robot to solve the problem that the robot can accurately locate the position of a user in a complex indoor environment.

Description

Technical Field

[0001] This application relates to the field of intelligent robots, and in particular to an interactive method, computer device, storage medium and system with voice noise reduction. Background Technology

[0002] With the continuous development of robotics technology, indoor service robots are now capable of performing most tasks, including retrieving and placing items, controlling home appliances, and providing indoor security. Currently, some robots locate users through sound sources. However, due to the complexity of indoor environments (such as sound field environments and numerous objects inside), the accuracy of sound source localization is not high, preventing the robots from providing adequate service. Summary of the Invention

[0003] This application mainly provides an interactive method, computer device, storage medium, and system with voice noise reduction, to solve the problem of robots accurately locating users' positions in complex indoor environments.

[0004] To address the aforementioned technical problems, this application provides a technical solution: an interactive method with voice noise reduction. This interactive method with voice noise reduction includes: acquiring a user's voice signal picked up by a voice acquisition device; performing voice noise reduction on the user's voice signal; uploading the voice-denoised user voice signal to a voice recognition cloud; acquiring semantic information obtained after processing by the voice recognition cloud; and generating a task code based on the semantic information, the task code being used to transmit to the actuator of the indoor robot.

[0005] To address the aforementioned technical problems, another technical solution adopted in this application is to provide a computer device. This computer device includes a processor and a memory, the processor being coupled to the memory, the memory storing a program, and the processor executing the program to implement the interactive method with voice noise reduction.

[0006] To address the aforementioned technical problems, this application adopts a technical solution: a storage medium. The storage medium stores a program, which, when executed, enables the implementation of the interactive method with voice noise reduction.

[0007] To address the aforementioned technical problems, another technical solution adopted in this application is to provide an interactive system with speech noise reduction. This interactive system with speech noise reduction includes:

[0008] Multiple voice acquisition devices are arranged indoors;

[0009] Voice recognition cloud platform, used to recognize voice signals;

[0010] Indoor robots used to perform tasks;

[0011] Computer equipment that communicates with multiple voice acquisition devices, a voice recognition cloud, and indoor robots.

[0012] The beneficial effects of this application are as follows: Unlike existing technologies, this application discloses an interactive method, computer device, storage medium, and system with speech noise reduction. By acquiring the user's speech signal through a speech collector and performing speech noise reduction on the user's speech signal, a clean speech signal can be obtained. This ensures that the speech signal uploaded to the speech recognition cloud can be accurately recognized, reducing the error rate of recognition, improving the accurate positioning of the user, obtaining semantic information after processing by the speech recognition cloud, and generating a task code based on the speech information. This enables the robot to accurately locate the user's current position and execute tasks, thereby improving the robot's operational capabilities in complex indoor environments. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, wherein:

[0014] Figure 1 This is a schematic diagram illustrating the structure of the living room voice acquisition device for positioning, as shown in an embodiment of this application.

[0015] Figure 2 This is a schematic diagram illustrating the structure of an interactive system with noise reduction function, as shown in the embodiments of this application;

[0016] Figure 3 This is a flowchart illustrating an interactive method with noise reduction functionality, as shown in an embodiment of this application.

[0017] Figure 4 This is a schematic diagram illustrating the structure of a large exhibition hall voice acquisition device for positioning, as shown in an embodiment of this application.

[0018] Figure 5 This is a schematic diagram illustrating the structure of a computer device according to an embodiment of this application;

[0019] Figure 6 This is a schematic diagram illustrating the structure of the storage medium in an embodiment of this application. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0021] The terms "first," "second," and "third" used in the embodiments of this application are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first," "second," or "third" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0022] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0023] In the field of intelligent robot indoor services, the indoor acoustic environment is complex, potentially accompanied by audio and video noise, non-voice command noise from people, and noise generated by the operation of objects, such as kitchen noise, washing machine noise, and robot vacuum cleaner noise. Furthermore, due to the complexity of indoor building structures, robots and users are often separated by obstacles such as sofas, fish tanks, and partitions. These combined factors make it difficult for indoor service robots to achieve natural voice interaction with users in indoor environments.

[0024] For example, such as Figure 1 As shown, Figure 1This is a schematic diagram illustrating the positioning structure of a living room voice collector, as shown in this embodiment of the application. The living room contains a television, windows, and other items. Multiple microphones are arranged in a circular array within the room to collect indoor sound. These microphones are sequentially numbered, such as Microphone 1, Microphone 2, etc. Speakers are distributed in four directions of the living room to broadcast information; these speakers are also sequentially numbered, such as Speaker 1, Speaker 2, etc. The noise sources include the sound of the television, sounds coming from outside the windows, non-voice commands from people inside the room, noise generated by the operation of objects, such as kitchen noise, washing machine noise, and robot vacuum cleaner noise. Users can issue voice commands within the room; the microphones pick up the voice commands, and the speakers broadcast feedback on the task progress.

[0025] This application provides an interactive system with voice noise reduction; please refer to [link / reference]. Figure 1 and Figure 2 ,in, Figure 2 This is a schematic diagram illustrating the structure of an interactive system with voice noise reduction, as shown in an embodiment of this application. In this embodiment, the interactive system with voice noise reduction includes multiple voice acquisition devices 100, a voice recognition cloud platform 200, an indoor robot 300, and a computer device 400. The computer device 400 is communicatively connected to the multiple voice acquisition devices 100, the voice recognition cloud platform 200, and the indoor robot 300. The multiple voice acquisition devices 100 are arranged indoors to collect voice signals from users. The voice recognition cloud platform 200 is used to recognize the semantic information of the user's voice signals. The computer device 400 generates a task code based on the semantic information and sends the task code to the indoor robot 300. After receiving the task code, the indoor robot 300 executes the task to realize its service functions.

[0026] The wake-up signal is monitored by multiple voice acquisition devices 100. When the wake-up signal is successfully acquired, the user's current location in the room can be determined by sound source localization, thereby realizing the user's positioning and improving the positioning accuracy of the indoor robot 300.

[0027] Optionally, multiple voice acquisition units 100 are arranged in a circular array indoors. Each voice acquisition unit 100 can be a microphone or other microphone with sound pickup capabilities. For example, the circular array of multiple voice acquisition units 100 can be arranged on the ceiling, eaves, etc., and the number of microphones can be increased or decreased according to the room size and actual needs. Optionally, the microphone array can be installed in various ways, without being fixed. For example, it can be installed with suction cups for easy removal or pre-installed in an aesthetically pleasing way. The appropriate installation method can be selected according to different scenarios. For example, pre-installation can be used in living rooms, dining rooms, and studies to ensure the aesthetics of the room structure, while suction cup mounting can be used in kitchens, bedrooms, and bathrooms.

[0028] During the operation of the interactive system, multiple voice collectors 100 are in a monitoring and wake-up state. When a user broadcasts a voice signal indoors, such as "Hello, Xiaohai," it can be collected by the voice collectors 100 and recognized by the computer. This can then activate at least some of the voice collectors 100 to collect the user's voice signal. The computer device 400 acquires the user's voice signal and performs voice noise reduction on it before transmitting it to the voice recognition cloud 200. The voice recognition cloud 200 recognizes the semantic information of the user's voice signal. The computer device 400 receives the semantic information transmitted by the voice recognition cloud 200, generates a task code based on the semantic information, and sends the task code to the indoor robot 300. After receiving the task code, the indoor robot 300 executes the preset task action according to the information in the task code.

[0029] This application provides an interactive method with speech noise reduction; please refer to [link / reference]. Figures 1 to 4 ,in, Figure 3 This application provides a flowchart illustrating an embodiment of an interactive method with speech noise reduction. Figure 4 This is a schematic diagram illustrating the structure of a voice acquisition device for locating voice data in a large exhibition hall, as shown in an embodiment of this application. In this embodiment, the interactive method with voice noise reduction includes:

[0030] S11: Acquire the user's voice signal collected by the voice acquisition device.

[0031] The voice acquisition device 100 can be a microphone or other sound-collecting device. The user's voice information can be voice commands issued by the user indoors, such as controlling a robot vacuum cleaner to perform sweeping actions or controlling a robot to deliver documents. Voice commands can be "sweep" or "bring me a glass of water," etc.

[0032] Optionally, prior to step S11, the interaction method further includes:

[0033] Multiple voice acquisition devices are used to collect voice wake-up signals.

[0034] like Figure 1 and Figure 2 As shown, the computer device 400 uses the multiple voice acquisition units 100 to collect voice wake-up signals. The multiple voice acquisition units 100 arranged around the device are all in a low-power monitoring state. In this state, they are used to collect voice wake-up signals, such as "Xiao Hai, hello" or "Xiao Hai, Xiao Hai".

[0035] Identify at least a portion of the voice acquisition unit 100 that corresponds to the target voice wake-up signal among multiple voice wake-up signals that meets the voice quality requirements.

[0036] After the computer device 400 recognizes the voice wake-up signal, it determines the target voice wake-up signal that meets the voice quality requirements from multiple voice wake-up signals. For example, the voice quality requirement is that the volume reaches a set threshold, or the signal-to-noise ratio of the voice wake-up signal reaches a preset condition. Then, it can determine the part of the voice collector 100 that is close to the user.

[0037] The step of acquiring user voice signals collected by voice acquisition device 100 includes: acquiring user voice signals using at least part of the voice acquisition device 100.

[0038] That is, by using this part of the voice acquisition unit 100 to collect user voice signals, higher quality user voice commands can be obtained, which is beneficial to improving the accuracy of voice content recognition.

[0039] Furthermore, the interaction method also includes: responding to a voice wake-up signal and determining the user's location information and the noise source's location information based on the voice wake-up signal. Determining the user's location information and the noise source's location information enables the robot to accurately locate the user's current position indoors, thereby improving the accuracy of user location positioning.

[0040] S12: Perform voice noise reduction on the user's voice signal.

[0041] Specifically, the first voice signal picked up by at least a portion of the voice acquisition device 100 corresponding to the user's location information is acquired, wherein the first voice signal is used as the main signal of the user's voice signal in the voice acquisition device 100.

[0042] When the size of the indoor area meets the preset conditions, the second voice signals picked up by three voice collectors corresponding to the user's location information are acquired, and the three second voice signals are combined into one main signal.

[0043] When the indoor area is not less than a preset condition, the three voice collectors 100 closest to the user's location are activated; when the indoor area is less than a preset condition, the voice collector 100 closest to the user's location and the two voice collectors 100 located on either side of the user's location are activated.

[0044] The second speech signal picked up by the speech acquisition unit 100 corresponding to the location information of the noise source is acquired, and the second speech signal is used as a reference signal for the user's speech signal.

[0045] Furthermore, the speaker audio signal is acquired, and the speaker audio signal is also used as a reference signal for the user's voice signal.

[0046] For example, the preset conditions could refer to an indoor area of ​​50 square meters or 100 square meters, and the microphones would be installed in a circular array indoors.

[0047] Specifically, when the indoor area is not smaller than a preset condition, the three microphones closest to the user's position are activated. For example... Figure 4 As shown, for example, in a large exhibition hall, the microphone 9 closest to the user is turned on to collect the user's voice signal. Considering that the user may turn their body or move slightly after issuing a wake-up command, the microphones 8 and 10 adjacent to the microphone 9 are turned on, while the remaining microphones remain in a low-power monitoring state. The microphones 8 and 10 collect the user's voice signal respectively, and the collected voice signals are combined into a main signal to improve the accuracy of collecting the user's voice signal.

[0048] When the indoor area is smaller than a preset condition, activate the microphone closest to the user's position and the two microphones located on either side of the user's position. For example... Figure 1 As shown, for example, the microphone 9 closest to the user is turned on and the user's voice signal is collected. Considering that the user may turn his body or move slightly after issuing a wake-up command, the microphones 7 and 12 adjacent to the microphone 9 are turned on, while the remaining microphones are kept in a low-power monitoring state. The microphones 7 and 12 collect the user's voice signal respectively, and the collected voice signals are combined into a main signal to improve the accuracy of collecting the user's voice signal.

[0049] Each voice collector 100 has a corresponding identification code. The first voice signal includes the identification code. When acquiring multiple first voice signals picked up by multiple voice collectors 100, the other first voice signals that are different from the identification code are also used as reference signals for the user's voice signal.

[0050] Specifically, the identification code can be a number or another identifiable identifier. For example, multiple microphones can be labeled with identification numbers in sequence, such as microphone 1, microphone 2, microphone 3, etc. The microphones in a circular array labeled with identification numbers are installed indoors, with each microphone evenly distributed to ensure that the microphones can accurately acquire the voice signals issued by the user.

[0051] Based on the reference signal, perform speech noise reduction on the main signal;

[0052] Among them, speech denoising methods include at least one of echo delay estimation, normalized least mean square adaptive algorithm, echo cancellation based on adaptive filtering, and generative adversarial network denoising based on Wiener filtering.

[0053] S13: Upload the user's voice signal after noise reduction to the voice recognition cloud.

[0054] The user's voice signal, after being denoised, is better recognized by the speech recognition cloud, thereby improving the accuracy of recognizing the semantic information of the user's voice signal and enhancing the smoothness of the interaction between the indoor robot and the user.

[0055] S14: Obtain semantic information obtained after speech recognition cloud processing.

[0056] Speech recognition in the cloud can be a cloud service platform that recognizes the user's speech signals and then obtains semantic information. This semantic information can be the task commands issued by the user.

[0057] S15: Generate a task code based on semantic information. The task code is used to pass it to the actuator of the indoor robot.

[0058] Computer device 400 generates a task code based on semantic information and sends the task code to indoor robot 300.

[0059] The task code includes task instructions, user location information, and motion trajectories generated based on indoor semantic maps.

[0060] For example, task instructions could be "sweep the floor" or "bring me a glass of water." The robot will deliver the item to the user's location specified in the task information, while simultaneously providing task progress updates via microphones near the user's location. Updates will not be broadcast for locations farther from the user. Figure 1 In some embodiments, only speaker 3 broadcasts the task feedback process, while the other speakers remain silent. In other embodiments, when the user is far from the microphone used to collect their voice information, the task feedback voice broadcast is delivered via the robot's own speakers so that the user can hear the voice feedback.

[0061] In this embodiment, the indoor semantic map is calibrated by the robot itself, such as by the robot's lidar or other modules capable of modeling the interior. The semantic map labels the location information of indoor objects, including stationary furniture, appliances with sound capabilities (such as televisions, microwave ovens, and smart refrigerators), and the specific dimensions of the room. The semantic map also labels the specific location of each microphone and speaker within the room.

[0062] Based on this, this application also provides a computer device 400, please refer to... Figure 5 , Figure 5 This is a schematic diagram illustrating the structure of a computer device according to an embodiment of this application. In this embodiment, the computer device 400 includes a processor 401 and a memory 402. The processor 401 is coupled to the memory 402, which is used to store programs. The processor 401 is used to execute programs to implement the positioning method of an indoor robot according to any embodiment.

[0063] Computer devices can be codecs. Processor 401 can also be called a CPU (Central Processing Unit). Processor 401 can be an integrated circuit chip with signal processing capabilities. Processor 401 can also be a general-purpose processor 401, a digital signal processor 401 (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. General-purpose processor 401 can be a microprocessor 401, or any conventional processor 401, etc.

[0064] Based on this, this application also provides a storage medium 500, please refer to... Figure 6 , Figure 6 This is a schematic diagram illustrating the structure of a storage medium according to an embodiment of this application. In this embodiment, the storage medium stores a program 501, which, when executed, can implement the positioning method of the indoor robot in any embodiment.

[0065] Storage media are media used in computer memory to store certain discontinuous physical quantities. The aforementioned storage media with storage functions include: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks or optical disks, and various other media that can store program code.

[0066] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of apparatuses or units may be electrical, mechanical, or other forms.

[0067] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0068] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0069] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this application. The above are merely embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. An interactive method with speech noise reduction, characterized in that, The interaction method includes: Multiple voice acquisition devices are used to collect voice wake-up signals separately; Identify at least a portion of the voice acquisition devices that correspond to the target voice wake-up signal among the plurality of voice wake-up signals that meet the voice quality requirements; In response to the voice wake-up signal, the user's location information and the location information of the noise source are determined based on the voice wake-up signal; Acquire user voice signals collected by at least some of the voice acquisition devices; The user's voice signal is subjected to speech noise reduction, wherein the speech noise reduction steps include: The voice collector acquires at least a portion of the first voice signal picked up by the voice collector, which corresponds to the user's location information, and the voice collector uses the first voice signal as the main signal of the user's voice signal. Acquire a second speech signal picked up by a speech acquisition device corresponding to the location information of the noise source, and use the second speech signal as a reference signal for the user's speech signal; Based on the reference signal, speech noise reduction is performed on the main signal; Upload the user's voice signal, after noise reduction, to the speech recognition cloud; Obtain semantic information obtained after cloud-based speech recognition processing; A task code is generated based on the semantic information, and the task code is used to transmit to the actuator of the indoor robot.

2. The interactive method with speech noise reduction according to claim 1, characterized in that, The step of collecting user voice signals using at least some of the voice collectors further includes: Acquire the speaker audio signal, which also serves as a reference signal for the user's voice signal.

3. The interactive method with speech noise reduction according to claim 1, characterized in that, The step of acquiring the first voice signal picked up by at least a portion of the voice acquisition device corresponding to the user's location information includes: When the size of the indoor area meets the preset conditions, the second voice signals picked up by three voice collectors corresponding to the user's location information are acquired, and the three second voice signals are combined into one main signal. When the indoor area is not less than the preset condition, the three voice collectors closest to the user's location are activated; when the indoor area is less than the preset condition, the voice collector closest to the user's location and the two voice collectors located on either side of the user's location are activated.

4. The interactive method with speech noise reduction according to claim 1, characterized in that, Each of the voice collectors is provided with a corresponding identification code, and the first voice signal includes the identification code; When acquiring multiple sets of first voice signals picked up by the voice collectors, the remaining first voice signals that are distinct from the identity code are also used as reference signals for the user's voice signal.

5. The interactive method with speech noise reduction according to any one of claims 1 to 4, characterized in that, The speech denoising methods include at least one of echo delay estimation, normalized least mean square adaptive algorithm, echo cancellation based on adaptive filtering, and generative adversarial network denoising based on Wiener filtering.

6. The interactive method with speech noise reduction according to claim 1, characterized in that, The task code includes task instructions, user location information, and a motion trajectory generated based on an indoor semantic map.

7. A computer device, characterized in that, The computer device includes a processor and a memory, the processor being coupled to the memory for storing a program, and the processor for executing the program to implement the interactive method with voice noise reduction as described in any one of claims 1-6.

8. A storage medium, characterized in that, The storage medium stores a program that, when executed, enables the interactive method with voice noise reduction as described in any one of claims 1-6.

9. An interactive system with voice noise reduction, characterized in that, The interactive system with voice noise reduction includes: Multiple voice acquisition devices are arranged indoors; Voice recognition cloud platform, used to recognize voice signals; Indoor robots used to perform tasks; A computer device is communicatively connected to the plurality of voice acquisition devices, the voice recognition cloud, and the indoor robot, wherein the computer device is the computer device as described in claim 7.

Images