Robot voice control system and voice control method facing complex environment mixed voice

Abstract

The invention discloses a robot sound control system and method for complex environment mixed voice, and belongs to the technical field of robot control, the system comprises a voice input module, a voice separation model, a voice recognition control system and a voice management system; the voice input module is used for receiving mixed voice; the voice separation model is used for extracting high-dimensional voice features by using a convolutional neural network, a self-attention mechanism and a gated attention network, and performing Hamiltonian feature evolution compensation to obtain a plurality of separated voices mixed with human voices; the voice recognition control system is used for recognizing and matching voiceprint features of a control person and driving and controlling the robot through semantic understanding; and the voice management system is used for managing voices of the control persons and configuring priorities of all the control persons in a man-machine interaction mode. According to the method, the problems of strong noise interference, multi-person dialogue interference and safety risk of malicious control by others in a robot man-machine interaction task are solved, and the safety control capability of the robot in a special scene is improved.

Description

Technical Field

[0001] This invention belongs to the field of robot control technology, specifically relating to a robot voice control system and method for complex environments with mixed speech. Background Technology

[0002] With the development of robotics and artificial intelligence technologies, voice interaction has become one of the main control methods for service robots and special-purpose robots. However, in practical application scenarios such as homes, shopping malls, and factories, the voice received by robots is usually a mixed signal including the target human voice, background noise, and other people's voices, which poses technical problems such as strong noise interference and interference from multiple conversations, as well as security risks of malicious control by others.

[0003] Existing technical solutions mostly focus on speech signal recognition and analysis. Invention patent CN114495973B discloses a speaker-specific speech separation method based on a dual-path self-attention mechanism, capable of quickly and accurately extracting the target speaker's voice from mixed corpora containing noise and interference from multiple speakers. Invention patent application CN110675891A discloses a speech separation method based on a multi-layer attention mechanism, which continuously cleanses the mixed speech through a multi-layer attention structure and the speaker's voiceprint information to obtain the separated speech. However, these methods still have the following shortcomings: 1. They require obtaining clean speech information from a specific speaker and fusing it with mixed speech features to obtain the speaker's clean speech signal, making them unsuitable for situations with multiple specific speakers; 2. The models do not fully utilize speech feature information, resulting in insufficient generalization to scenarios outside the training data distribution, especially under limited sample datasets where performance cannot meet requirements; 3. They do not address the security risks associated with complex mixed speech interference or even malicious control by others. Summary of the Invention

[0004] This invention addresses the shortcomings of existing technologies by providing a robot voice control system and method for complex environments with mixed speech. Through a high-performance speech separation model, a speech recognition control system, and a speech management system, it solves the safety risks of strong noise interference, multi-person dialogue interference, and malicious control by others in robot human-robot interaction tasks, thereby improving the robot's safety control capabilities in special scenarios.

[0005] This invention provides the following technical solution: In the first aspect, a robot voice control system for complex environments with mixed speech is provided, including: a voice input module, a voice separation model, a voice recognition control system, and a voice management system; The voice input module is used to receive mixed human voices; The speech separation model uses convolutional neural networks, self-attention mechanisms, and gated attention networks to extract high-dimensional speech features and performs Hamiltonian feature evolution compensation to obtain several separated speech segments of mixed human voice. The voice recognition control system identifies and matches the voiceprint features of the controller, and drives the robot through semantic understanding based on the controller's separated speech. The voice management system manages the voice of the controller and configures the priority of all controllers in a human-computer interaction manner.

[0006] Optionally, the speech separation model includes: a coding layer E1, a convolutional neural network, an average pooling layer P1, a self-attention layer A1, a two-level gated attention network, a Hamiltonian evolution layer HN1, and a sixth convolutional layer C6; The input mixed human voice is encoded by the E1 coding layer to obtain the encoded speech feature EAF; The encoded speech features EAF are sequentially passed through several convolutional layers of a convolutional neural network to obtain the features corresponding to each convolutional layer; The average pooling layer P1 performs average pooling and accumulation on all features output by the convolutional neural network, and then passes them through the self-attention layer A1 to obtain multi-scale fused features. The features of each convolutional layer of the convolutional neural network and the multi-scale fusion features are respectively input into several gated attention layers of the first-level gated attention network for fusion to obtain multiple gated enhancement features, which are then input into several gated attention layers of the second-level gated attention network for cross-level secondary fusion to obtain the high-dimensional speech feature AT9. The encoded speech feature EAF is further evolved through Hamiltonian evolution layer HN1 to obtain channel-level speech evolution feature H1; The channel-level speech evolution feature H1 and the high-dimensional speech feature AT9 are sequentially passed through an adder and a sixth convolutional layer C6 to obtain multiple separated speech samples.

[0007] Optionally, the convolutional neural network has five convolutional layers, including a first convolutional layer C1, a second convolutional layer C2, a third convolutional layer C3, a fourth convolutional layer C4, and a fifth convolutional layer C5, and the output features corresponding to each convolutional layer are features AF1, features AF2, features AF3, features AF4, and features AF5. The first-level gated attention network has five gated attention layers, including the first gated attention layer GA1, the second gated attention layer GA2, the third gated attention layer GA3, the fourth gated attention layer GA4, and the fifth gated attention layer GA5. Feature AF1 is input into the fifth gated attention layer GA5 along with the multi-scale fusion feature to obtain the gated enhancement feature AT1. Feature AF2 is input into the fourth gated attention layer GA4 along with the multi-scale fusion feature to obtain the gated enhancement feature AT2. Feature AF3 is input into the third gated attention layer GA3 along with the multi-scale fusion feature to obtain the gated enhancement feature AT3. Feature AF4 is input into the second gated attention layer GA2 along with the multi-scale fusion feature to obtain the gated enhancement feature AT4. Feature AF5 is input into the first gated attention layer GA1 along with the multi-scale fusion feature to obtain the gated enhancement feature AT5.

[0008] Optionally, the second-level gated attention network has four gated attention layers, including the sixth gated attention layer GA6, the seventh gated attention layer GA7, the eighth gated attention layer GA8, and the ninth gated attention layer GA9. The gated enhancement features AT4 and AT5 are passed through the sixth gated attention layer GA6 to obtain the gated fusion feature AT6. The gated fusion feature AT6 and the gated enhancement feature AT3 are passed through the seventh gated attention layer GA7 to obtain the gated fusion feature AT7. The gated fusion feature AT7 and the gated enhancement feature AT2 are passed through the eighth gated attention layer GA8 to obtain the gated fusion feature AT8. The gated fusion feature AT8 and the gated enhancement feature AT1 are passed through the ninth gated attention layer GA9 to obtain the high-dimensional speech feature AT9.

[0009] Optionally, the output features of each of the gated attention layers The calculation formula is: )

[0010]

[0011] Where A and B are two input features of the gated attention layer. ， For interpolation functions, Given the sequence length of input feature A, , , These are three independent fully connected neural networks. This is the Sigmoid activation function.

[0012] Optionally, the Hamiltonian evolution layer HN1 includes a speech feature Hamiltonian coding module EH1, a Hamiltonian convolutional layer HTC, a potential energy learning module VLM, a kinetic energy calculation module TLM, and a Hamiltonian evolution module HEM; After the speech features are processed by the speech feature Hamiltonian coding module EH1 and the Hamiltonian convolutional layer HTC, the generalized position q and the generalized momentum p are obtained. The potential energy learning module learns the potential energy of speech features evolving in phase space. ; The kinetic energy calculation module then obtains the kinetic energy based on the learnable mass parameter m and the generalized momentum p. ; according to The Hamiltonian is calculated and input into the Hamiltonian Evolution Module (HEM). The Hamiltonian Evolution Module (HEM) uses the Hamiltonian regular equation and the Leapfrog symplectic integral method to perform Hamiltonian evolution on the high-dimensional speech features to obtain the channel-level speech evolution features H1.

[0013] Optionally, the speech feature Hamiltonian coding module EH1 includes a convolutional layer HC1, an activation layer HA1, a convolutional layer HC2, and an activation layer HA2. Input data is processed by the speech feature Hamiltonian coding module EH1 to obtain speech Hamiltonian features, the feature scale of which is... ,in, For batch processing quantity, Bit length, The number of feature channels represents the Hamiltonian features of the speech. Divide the data equally by the number of channels to obtain the generalized position q and the generalized momentum p.

[0014] Secondly, a robot voice control method for complex environments with mixed speech is provided, comprising the following steps: S1: The mixed human voice is separated into several separate speech samples using a semantic separation model; S2: Extract the speaker characteristics of the separated speech; S3: Based on the pre-recorded voiceprint database of the controller's voiceprint information, determine whether the currently separated voiceprint feature matches the voiceprint information in the voiceprint database. If they do not match, wait for the next control command. If they match, determine whether the voiceprint information in the successfully matched voiceprint database is the only controller. If yes, proceed to step S4. If not, select the only controller according to the preset priority. S4: Perform text recognition on the separate speech of the sole controller, understand the control intention based on the text, and convert the control intention into robot control instructions to drive the robot to execute the control instructions.

[0015] Thirdly, a computer device is provided, including a processor and a memory; wherein, when the processor executes a computer program stored in the memory, it implements the steps of the robot voice control method for mixed speech in complex environments as described in any of the second aspects.

[0016] Fourthly, a computer-readable storage medium is provided for storing a computer program; when the computer program is executed by a processor, it implements the steps of the robot voice control method for mixed speech in complex environments as described in any of the second aspects.

[0017] Compared with the prior art, the beneficial effects of the present invention are: (1) This invention can provide a specific human voice recognition and control method for complex environments with high noise and multiple voices, solve the safety risks of strong noise interference, multiple voice interference and malicious control by others that cannot be handled by existing technologies, and improve the safety control capability of robots in special scenarios.

[0018] (2) By introducing Hamiltonian dynamics, this invention ensures the energy conservation characteristics of speech signals during the separation process, and can efficiently and accurately model long-term dependencies in speech signals, thereby improving the separation accuracy in complex scenarios.

[0019] (3) The Hamiltonian feature evolution module in the speech separation model disclosed in this invention uses the Leapfrog symplectic integral method, which can strictly maintain the symplectic geometric structure of the Hamiltonian system. While maintaining long-term accuracy, it provides more stable learning and training, and is suitable for long-term, stable and interpretable modeling tasks of complex systems.

[0020] (4) The present invention can flexibly manage the voiceprint of a specific controller according to the task scenario, and grant the robot voice control permissions according to priority, so as to avoid interference or even malicious control by others. Attached Figure Description

[0021] Figure 1 This is a structural diagram of the speech separation model of the present invention; Figure 2 This is a diagram of the Hamiltonian evolution layer structure of the present invention; Figure 3 This is a control flowchart of the robot voice control method for mixed speech in complex environments according to the present invention. Detailed Implementation

[0022] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and should not be used to limit the scope of protection of the present invention. It should be noted that the term "comprising" and any variations thereof in the specification, claims and the above-mentioned drawings of the present invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products or devices.

[0023] Example 1 A robot voice control system for complex environments with mixed speech includes: a voice input module, a voice separation model, a voice recognition control system, and a voice management system.

[0024] I. Voice Input Module.

[0025] Used to receive mixed human voices, such as those from a microphone.

[0026] II. Speech Separation Model.

[0027] High-dimensional speech features are extracted using convolutional neural networks, self-attention mechanisms, and gated attention networks, and Hamiltonian feature evolution compensation is performed to obtain several separate speech segments of mixed human voice.

[0028] In this embodiment, as Figure 1 As shown, The speech separation model consists of: a coding layer E1, a convolutional neural network, an average pooling layer P1, a self-attention layer A1, a two-level gated attention network, a Hamiltonian evolution layer HN1, and a sixth convolutional layer C6.

[0029] A convolutional neural network has five convolutional layers: the first convolutional layer C1, the second convolutional layer C2, the third convolutional layer C3, the fourth convolutional layer C4, and the fifth convolutional layer C5.

[0030] The first-level gated attention network has five gated attention layers, including the first gated attention layer GA1, the second gated attention layer GA2, the third gated attention layer GA3, the fourth gated attention layer GA4, and the fifth gated attention layer GA5.

[0031] The second-level gated attention network has four gated attention layers, including the sixth gated attention layer GA6, the seventh gated attention layer GA7, the eighth gated attention layer GA8, and the ninth gated attention layer GA9.

[0032] Specifically, the input mixed human voice is encoded by the E1 coding layer to obtain the encoded speech feature EAF.

[0033] The encoded speech feature EAF is passed through the first convolutional layer C1 to obtain feature AF1, through the second convolutional layer C2 to obtain feature AF2, through the third convolutional layer C3 to obtain feature AF3, through the fourth convolutional layer C4 to obtain feature AF4, and through the fifth convolutional layer C5 to obtain feature AF5.

[0034] The average pooling layer P1 performs average pooling on features AF1, AF2, AF3, AF4 and AF5 based on feature AF5 and accumulates them. Then, it passes through the self-attention layer A1 to obtain multi-scale fused features.

[0035] Feature AF1 is input into the fifth gated attention layer GA5 along with the multi-scale fusion feature to obtain the gated enhancement feature AT1. Feature AF2 is input into the fourth gated attention layer GA4 along with the multi-scale fusion feature to obtain the gated enhancement feature AT2. Feature AF3 is input into the third gated attention layer GA3 along with the multi-scale fusion feature to obtain the gated enhancement feature AT3. Feature AF4 is input into the second gated attention layer GA2 along with the multi-scale fusion feature to obtain the gated enhancement feature AT4. Feature AF5 is input into the first gated attention layer GA1 along with the multi-scale fusion feature to obtain the gated enhancement feature AT5.

[0036] The gated enhancement features AT4 and AT5 are passed through the sixth gated attention layer GA6 to obtain the gated fusion feature AT6. The gated fusion feature AT6 and the gated enhancement feature AT3 are passed through the seventh gated attention layer GA7 to obtain the gated fusion feature AT7. The gated fusion feature AT7 and the gated enhancement feature AT2 are passed through the eighth gated attention layer GA8 to obtain the gated fusion feature AT8. The gated fusion feature AT8 and the gated enhancement feature AT1 are passed through the ninth gated attention layer GA9 to obtain the high-dimensional speech feature AT9.

[0037] The encoded speech feature EAF is further evolved through the Hamiltonian evolution layer HN1 to obtain the channel-level speech evolution feature H1.

[0038] The channel-level speech evolution feature H1 and the high-dimensional speech feature AT9 are passed sequentially through an adder and a sixth convolutional layer C6 to obtain multiple separated speech samples. The number of convolutional kernels in the sixth convolutional layer C6 is equal to the number of speech channels to be separated, thus obtaining the final separated output speech.

[0039] In this embodiment, the output features of each gated attention layer The calculation formula is: )

[0040]

[0041] Where A and B are two input features of the gated attention layer. ， For interpolation functions, Given the sequence length of input feature A, , , These are three independent fully connected neural networks. This is the Sigmoid activation function.

[0042] like Figure 2As shown, the Hamiltonian evolution layer HN1 can learn the inherent laws of complex high-dimensional speech features. It changes from the traditional stochastic gradient descent learning mode to intelligent exploration along the energy gradient direction, avoiding ineffective random search and achieving efficient feature learning of mixed speech data.

[0043] The Hamiltonian evolution layer HN1 includes the speech feature Hamiltonian coding module EH1, the Hamiltonian convolutional layer HTC, the potential energy learning module VLM, the kinetic energy calculation module TLM, and the Hamiltonian evolution module HEM.

[0044] After the speech features are processed by the speech feature Hamiltonian coding module EH1 and the Hamiltonian convolutional layer HTC, the generalized position q and generalized momentum p are obtained. Specifically, the speech feature Hamiltonian coding module EH1 includes a convolutional layer HC1, an activation layer HA1, a convolutional layer HC2, and an activation layer HA2. The input data is processed by the speech feature Hamiltonian coding module EH1 to obtain speech Hamiltonian features, the feature scale of which is [missing information]. ,in, For batch processing quantity, Bit length, The number of feature channels represents the Hamiltonian features of the speech. Divide the data equally by the number of channels to obtain the generalized position q and the generalized momentum p.

[0045] The potential energy learning module learns the potential energy of speech features evolving in phase space. The potential learning module (VLM) consists of a fully connected layer FC1, an activation layer FA1, a fully connected layer FC2, and an activation layer FA2, which learns the potential energy of speech features evolving in phase space. .

[0046] The kinetic energy calculation module then obtains the kinetic energy based on the learnable mass parameter m and the generalized momentum p. .

[0047] according to The Hamiltonian is calculated and input into the Hamiltonian evolution module HEM. The Hamiltonian regular equation and the Leapfrog symplectic integral method are used to perform Hamiltonian evolution on the high-dimensional speech features to obtain the channel-level speech evolution features H1, thus realizing the Hamiltonian evolution of the high-dimensional speech features.

[0048] According to Hamilton's canonical equation The Hamiltonian evolution steps based on Leapfrog's symplectic integral are as follows: set up For time step, for Generalized location of high-dimensional features of temporal speech. for Generalized momentum of high-dimensional features of temporal speech; from arrive Half-step momentum update at time / 2 ; from arrive Real-time position updates ; from / 2 to Momentum update at every half-step. ; Output and Merging, as the output of speech Hamiltonian evolution.

[0049] III. Voice Recognition Control System.

[0050] Identify and match the voiceprint features of the controller, and drive the robot through semantic understanding based on the controller's separated speech.

[0051] In this embodiment, the methods for extracting the voiceprint features of the separated speech and matching the voiceprint features of the controller are both based on existing technologies. Furthermore, the method for first performing text recognition on the separated speech of the controller, then recognizing the control intent through semantic understanding, and finally converting the control intent into control commands also refers to existing technologies.

[0052] IV. Voice Management System

[0053] Manage the voice of the controller and configure the priority of all controllers in a human-computer interaction manner.

[0054] In this embodiment, the recording, storage, and deletion of the controller's voice are controlled, and the priority configuration can be configured according to actual needs.

[0055] Example 2 like Figure 3 As shown, a robot voice control method for mixed speech in complex environments includes the following steps: S1: The mixed human voice is separated into several separate speech samples using a semantic separation model; S2: Extract the speaker characteristics of the separated speech; S3: Based on the pre-recorded voiceprint database of the controller's voiceprint information, determine whether the currently separated voiceprint feature matches the voiceprint information in the voiceprint database. If they do not match, wait for the next control command. If they match, determine whether the voiceprint information in the successfully matched voiceprint database is the only controller. If yes, proceed to step S4. If not, select the only controller according to the preset priority. S4: Perform text recognition on the separate speech of the sole controller, understand the control intention based on the text, and convert the control intention into robot control instructions to drive the robot to execute the control instructions.

[0056] Before voice control, voiceprint recording and management, as well as a voice selection process for different control task scenarios including priority, enable flexible configuration of the controller's voice.

[0057] For more detailed information on the above methods, please refer to the relevant content disclosed in the foregoing embodiments, which will not be repeated here.

[0058] Example 3 The present invention provides a computer device, including a processor and a memory; wherein, when the processor executes a computer program stored in the memory, it implements the steps of the above-described robot voice control method for mixed speech in complex environments.

[0059] For a more detailed explanation of the above method, please refer to the relevant content disclosed in the foregoing embodiments, which will not be repeated here.

[0060] Example 4 The present invention provides a computer-readable storage medium for storing a computer program; when the computer program is executed by a processor, it implements the steps of the above-described robot voice control method for mixed speech in complex environments.

[0061] For a more detailed explanation of the above method, please refer to the relevant content disclosed in the foregoing embodiments, which will not be repeated here.

[0062] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. The methods, devices, and storage media disclosed in the embodiments are described simply because they correspond to the systems disclosed in the embodiments; relevant details can be found in the method section.

[0063] Those skilled in the art will clearly understand that the techniques in the embodiments of the present invention can be implemented using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments of the present invention.

[0064] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principle of the present invention should be considered within the scope of protection of the present invention.

Claims

1. A robot voice control system for complex environments with mixed speech, characterized in that, include: Voice input module, voice separation model, voice recognition control system, and voice management system; The voice input module is used to receive mixed human voices; The speech separation model uses convolutional neural networks, self-attention mechanisms, and gated attention networks to extract high-dimensional speech features and performs Hamiltonian feature evolution compensation to obtain several separated speech segments of mixed human voice. The voice recognition control system identifies and matches the voiceprint features of the controller, and drives the robot through semantic understanding based on the controller's separated speech. The voice management system manages the voice of the controller and configures the priority of all controllers in a human-computer interaction manner.

2. The robot voice control system for complex environments with mixed speech as described in claim 1, characterized in that, The speech separation model includes: a coding layer E1, a convolutional neural network, an average pooling layer P1, a self-attention layer A1, a two-level gated attention network, a Hamiltonian evolution layer HN1, and a sixth convolutional layer C6. The input mixed human voice is encoded by the E1 coding layer to obtain the encoded speech feature EAF; The encoded speech features EAF are sequentially passed through several convolutional layers of a convolutional neural network to obtain the features corresponding to each convolutional layer; The average pooling layer P1 performs average pooling and accumulation on all features output by the convolutional neural network, and then passes them through the self-attention layer A1 to obtain multi-scale fused features. The features of each convolutional layer of the convolutional neural network and the multi-scale fusion features are respectively input into several gated attention layers of the first-level gated attention network for fusion to obtain multiple gated enhancement features, which are then input into several gated attention layers of the second-level gated attention network for cross-level secondary fusion to obtain the high-dimensional speech feature AT9. The encoded speech feature EAF is further evolved through Hamiltonian evolution layer HN1 to obtain channel-level speech evolution feature H1; The channel-level speech evolution feature H1 and the high-dimensional speech feature AT9 are sequentially passed through an adder and a sixth convolutional layer C6 to obtain multiple separated speech samples.

3. The robot voice control system for complex environments with mixed speech as described in claim 2, characterized in that, The convolutional neural network has five convolutional layers, including a first convolutional layer C1, a second convolutional layer C2, a third convolutional layer C3, a fourth convolutional layer C4, and a fifth convolutional layer C5. The output features corresponding to each convolutional layer are features AF1, AF2, AF3, AF4, and AF5. The first-level gated attention network has five gated attention layers, including the first gated attention layer GA1, the second gated attention layer GA2, the third gated attention layer GA3, the fourth gated attention layer GA4, and the fifth gated attention layer GA5. Feature AF1 is input into the fifth gated attention layer GA5 along with the multi-scale fusion feature to obtain the gated enhancement feature AT1. Feature AF2 is input into the fourth gated attention layer GA4 along with the multi-scale fusion feature to obtain the gated enhancement feature AT2. Feature AF3 is input into the third gated attention layer GA3 along with the multi-scale fusion feature to obtain the gated enhancement feature AT3. Feature AF4 is input into the second gated attention layer GA2 along with the multi-scale fusion feature to obtain the gated enhancement feature AT4. Feature AF5 is input into the first gated attention layer GA1 along with the multi-scale fusion feature to obtain the gated enhancement feature AT5.

4. The robot voice control system for mixed speech in complex environments according to claim 3, characterized in that, The second-level gated attention network has four gated attention layers, including the sixth gated attention layer GA6, the seventh gated attention layer GA7, the eighth gated attention layer GA8, and the ninth gated attention layer GA9. The gated enhancement features AT4 and AT5 are passed through the sixth gated attention layer GA6 to obtain the gated fusion feature AT6. The gated fusion feature AT6 and the gated enhancement feature AT3 are passed through the seventh gated attention layer GA7 to obtain the gated fusion feature AT7. The gated fusion feature AT7 and the gated enhancement feature AT2 are passed through the eighth gated attention layer GA8 to obtain the gated fusion feature AT8. The gated fusion feature AT8 and the gated enhancement feature AT1 are passed through the ninth gated attention layer GA9 to obtain the high-dimensional speech feature AT9.

5. The robot voice control system for mixed speech in complex environments according to claim 2, characterized in that, Output features of each of the gated attention layers The calculation formula is: ) Where A and B are two input features of the gated attention layer. ， For interpolation functions, Given the sequence length of input feature A, , , These are three independent fully connected neural networks. This is the Sigmoid activation function.

6. The robot voice control system for mixed speech in complex environments according to claim 2, characterized in that, The Hamiltonian evolution layer HN1 includes a speech feature Hamiltonian coding module EH1, a Hamiltonian convolutional layer HTC, a potential energy learning module VLM, a kinetic energy calculation module TLM, and a Hamiltonian evolution module HEM. After the speech features are processed by the speech feature Hamiltonian coding module EH1 and the Hamiltonian convolutional layer HTC, the generalized position q and the generalized momentum p are obtained. The potential energy learning module learns the potential energy of speech features evolving in phase space. ; The kinetic energy calculation module then obtains the kinetic energy based on the learnable mass parameter m and the generalized momentum p. ; according to The Hamiltonian is calculated and input into the Hamiltonian Evolution Module (HEM). The Hamiltonian Evolution Module (HEM) uses the Hamiltonian regular equation and the Leapfrog symplectic integral method to perform Hamiltonian evolution on the high-dimensional speech features to obtain the channel-level speech evolution features H1.

7. The robot voice control system for mixed speech in complex environments according to claim 6, characterized in that, The speech feature Hamiltonian coding module EH1 includes a convolutional layer HC1, an activation layer HA1, a convolutional layer HC2, and an activation layer HA2. Input data is processed by the speech feature Hamiltonian coding module EH1 to obtain speech Hamiltonian features, the feature scale of which is... ,in, For batch processing quantity, Bit length, The number of feature channels represents the Hamiltonian features of the speech. Divide the data equally by the number of channels to obtain the generalized position q and the generalized momentum p.

8. A robot voice control method for mixed speech in complex environments, characterized in that, Includes the following steps: S1: The mixed human voice is separated into several separate speech samples using a semantic separation model; S2: Extract the speaker characteristics of the separated speech; S3: Based on the pre-recorded voiceprint database of the controller's voiceprint information, determine whether the currently separated voiceprint feature matches the voiceprint information in the voiceprint database. If they do not match, wait for the next control command. If they match, determine whether the voiceprint information in the successfully matched voiceprint database is the only controller. If yes, proceed to step S4. If not, select the only controller according to the preset priority. S4: Perform text recognition on the separate speech of the sole controller, understand the control intention based on the text, and convert the control intention into robot control instructions to drive the robot to execute the control instructions.

9. A computer device, characterized in that, It includes a processor and a memory; wherein, when the processor executes the computer program stored in the memory, it implements the steps of the robot voice control method for mixed speech in complex environments as described in claim 8.

10. A computer-readable storage medium, characterized in that, Used to store computer programs; when the computer programs are executed by the processor, they implement the steps of the robot voice control method for mixed speech in complex environments as described in claim 8.

Images