An intelligent humanoid companion robot and its voice interaction system

Abstract

The present invention relates to the field of intelligent robot technology, and in particular to an intelligent humanoid companion robot and a voice interaction system thereof, comprising: a voice input acquisition and preprocessing module: real-time acquisition of composite voice signals and voice triggering instructions in the surrounding environment through a distributed microphone array; a multi-source voice separation module: independent processing of a combination of multiple potential sound source signals, and separation into a plurality of independent sound source signals; a target user sound source identification and extraction module: using a dynamic Bayesian network to predict the change of the target sound source in a continuous time frame, and extracting the target user sound source signal; a semantic analysis and output module: generating a natural and fluent voice response through speech synthesis technology; the present invention solves the problem that the target user sound source is difficult to extract in noisy environments, overlapping multiple sound sources, and voice switching scenarios, and greatly improves the robustness and adaptability of the system in complex voice environments.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent robots, and in particular to an intelligent humanoid companion robot and a voice interaction system thereof. Background Art

[0002] With the rapid development of artificial intelligence and human-computer interaction technology, intelligent companion robots have gradually become an important role in people's daily lives. Real-time conversations with users through voice interaction are one of the core functions of companion robots. However, existing voice interaction systems face many challenges in complex environments, such as noisy environmental noise, overlapping multiple sound sources, voice switching, and insufficient processing capabilities for fuzzy semantic expressions. This significantly limits the widespread popularity of companion robots in multiple scenarios such as homes, public places, and medical care.

[0003] At present, mainstream voice interaction technology usually uses distributed microphone arrays to collect sound source signals, and combines speech separation algorithms to separate mixed signals into multiple independent sound sources. However, traditional separation methods (such as beamforming) are often unable to accurately extract the sound source signals of target users when faced with complex overlapping or dynamically changing scenarios with multiple sound sources. In particular, in terms of target user sound source identification, existing systems lack the ability to deeply model time series and dynamic changes in sound sources, and the voice interaction process is often interrupted by environmental noise, sound source interference or speech switching. Summary of the invention

[0004] The present invention provides an intelligent humanoid companion robot and a voice interaction system thereof.

[0005] A voice interaction system for an intelligent humanoid companion robot, comprising:

[0006] Voice input acquisition and preprocessing module: The distributed microphone array collects the composite voice signals and voice trigger commands in the surrounding environment in real time. The target user uses the voice trigger command to trigger the initial dialogue with the robot.

[0007] Multi-source speech separation module: In continuous conversation, the collected composite speech signal is decomposed in the time-frequency domain by a sparse decomposition algorithm, and the composite speech signal is represented as a combination of multiple potential sound source signals; the separation network is used to generate a time-frequency mask, and multiple potential sound source signal combinations are independently processed to separate them into multiple independent sound source signals;

[0008] Target user sound source identification and extraction module: Based on the separated multiple sound source signals, a dynamic Bayesian network is constructed to model the time characteristics and frequency characteristics of multiple sound source signals, and the dynamic Bayesian network is used to predict the changes of the target sound source in continuous time frames to extract the target user sound source signal;

[0009] Semantic parsing and output module: Use natural language processing technology to parse the speech content of the target user's sound source signal to extract semantic information, and match it with the knowledge base; based on the matching results, generate natural and fluent speech responses through speech synthesis technology; specifically include:

[0010] Target user speech content analysis: Based on the natural language processing technology of the pre-trained language model, the target user's sound source signal is converted into text content, and the text content is semantically annotated and dependency analyzed to extract key information units, including entity, action, time, and location semantic components;

[0011] Knowledge base matching of semantic information: The extracted semantic information is matched with the robot's built-in knowledge base to identify the user's intention. The knowledge base includes predefined dialogue templates and context-related information. During the matching process, the fuzzy instructions are matched or completed in combination with semantic similarity calculation (cosine similarity);

[0012] Natural and fluent voice response generation: Based on the knowledge base matching results, speech synthesis technology (WaveNet's speech synthesis model) is used to generate voice response content.

[0013] Optionally, the multi-source speech separation module specifically includes:

[0014] Time-frequency domain decomposition submodule: Perform short-time Fourier transform on the collected composite speech signal, convert the signal from the time domain to the time-frequency domain, generate the signal's time-frequency spectrogram, apply the non-negative matrix decomposition algorithm based on sparse decomposition, and use the sparse characteristics of the signal to decompose the composite speech signal into multiple basic signal components, and map the decomposed basic signal components into multiple potential sound source signal combinations, each of which corresponds to a specific time-frequency feature;

[0015] Separation network generates time-frequency mask submodule: Use the separation network to analyze the time-frequency spectrogram, extract the feature weights of each potential sound source signal, generate a time-frequency mask based on the feature weights, each time-frequency mask corresponds to the time-frequency feature distribution of a potential sound source signal, apply the generated time-frequency mask to reconstruct the composite speech signal, and reallocate each time-frequency unit of the composite speech signal to the original composite speech signal;

[0016] Separation submodule of independent sound source signals: restore the time-frequency signal processed by time-frequency masking to a time domain signal, generate multiple independent sound source signals, and output the separated independent sound source signals to provide high-quality input for the target user sound source identification and extraction module.

[0017] Optionally, the non-negative matrix decomposition algorithm is expressed as: ,in, is the time-frequency spectrogram matrix, is the basis matrix, of size is the number of potential sound source signals, is the coefficient matrix, size is , which represents the time activation intensity of each basis, is solved by defining the optimization objective and , decomposing the composite signal into multiple basic signal components;

[0018] The optimization objective is defined as: ,in, is the input time-frequency spectrum matrix, size is is the frequency dimension, is the number of time frames, Represents the F norm of the matrix, which is used to measure the error of matrix decomposition. , represents the non-negativity constraint, ensuring the physical meaning of the signal.

[0019] Optionally, the input of the separation network is a time-frequency spectrogram , output time-frequency mask , indicating the sound source At time-frequency point (time-frequency point) The weights on are used to distribute the energy of the composite signal. The generation of the time-frequency mask is expressed as: ;

[0020] The signal reconstruction is expressed as: ,in, Indicates the sound source The time-frequency mask of Indicates the sound source The amplitude at the time-frequency point, Represents the sum of the amplitudes of all sound sources at the time-frequency points, Indicates the separated sound source The time-frequency representation of is the time-frequency representation of the original composite signal, using the time-frequency mask The original composite signal The time-frequency points are weighted to obtain the separated signal, and the separated time-frequency domain representation signal The signal is converted back to the time domain by inverse short-time Fourier transform.

[0021] Optionally, the time-frequency signal processed by the time-frequency mask is restored to a time-domain signal by inverse short-time Fourier transform, which is expressed as:

[0022] ,in, Indicates the sound source The restored time domain signal is Is the sound source The time-frequency domain representation of is a window function, is the inverse Fourier transform kernel function.

[0023] Optionally, the target user sound source identification and extraction module specifically includes:

[0024] Construct a dynamic Bayesian network model: The separated multiple sound source signals are represented as observation variables of the dynamic Bayesian network. Including time features and frequency characteristics , define the state variables For sound source In time frame active state, and through the transition probability Modeling the changing probability trend of sound sources in continuous time frames;

[0025] Temporal feature modeling: For each sound source , extract its time features , including short-term energy characteristics and time continuity, using time series analysis methods to define the time characteristic observation probability , indicating the status Impact on temporal characteristics;

[0026] Frequency feature modeling: Extract each sound source Frequency characteristics , including the main frequency components and spectral energy distribution, defining the frequency feature observation probability , indicating the status Impact on frequency characteristics;

[0027] Joint modeling and prediction: Based on the Bayesian formula, the time characteristics and frequency characteristics are combined to calculate the sound source through joint modeling In time frame The posterior probability of the state on all time frames is calculated by recursive algorithm , predict the target user sound source Changing trends in consecutive time frames;

[0028] Target sound source identification and extraction: According to the prior information (directional characteristics) of the target user's sound source, the posterior probabilities of all sound sources are sorted, and the sound source corresponding to the maximum probability is selected as the target user's sound source:

[0029] ,in, Represents the sound source number of the target user, which is obtained by maximizing the posterior probability Obtained, among all separated sound sources In the above example, the sound source with the highest posterior probability is selected as the sound source of the target user.

[0030] Optionally, the time feature modeling is expressed as: ;

[0031] The temporal feature observation probability is defined as:

[0032] ;in, Is the sound source In time frame The short-term energy characteristics of Is the sound source At the time-frequency point The frequency amplitude, It is a time feature The observation probability follows a Gaussian distribution , and Representation and Status The relevant mean and variance characterize the impact of the state on the temporal characteristics.

[0033] Optionally, the frequency characteristic modeling is expressed as: ；

[0034] The frequency feature observation probability is defined as:

[0035] ;in, Is the sound source In frequency The normalized frequency component (frequency characteristic) of is the observation probability of the frequency feature, which follows a multinomial distribution, Representation and Status The relevant frequency component distribution parameters are used to describe the influence of the state on the frequency characteristics.

[0036] Optionally, the sound source is calculated by joint modeling In time frame The posterior probability is expressed as:

[0037] ,in, Yes Status The posterior probability of is the observation probability of the time feature, is the observation probability of the frequency feature, is the transition probability, indicating that the state changes from Transfer to probability.

[0038] An intelligent humanoid companion robot uses the above-mentioned voice interaction system to realize voice interaction with a user.

[0039] Beneficial effects of the present invention:

[0040] The present invention realizes efficient separation of composite speech signals by combining sparse decomposition algorithm and deep learning separation network, and jointly models time features and frequency features through dynamic Bayesian network, accurately predicts the changes of target user sound source in continuous time frames, and extracts the speech signal of target user. This method effectively solves the problem of difficulty in extracting target user sound source in noisy environment, overlapping of multiple sound sources and speech switching scenarios, and greatly improves the robustness and adaptability of the system in complex speech environments.

[0041] The present invention jointly models the time characteristics and frequency characteristics of the target user's sound source through a dynamic Bayesian network, and uses its powerful time series analysis and probabilistic reasoning capabilities to accurately predict the changes in the sound source state, thereby significantly improving the accuracy and consistency of target sound source recognition in a complex multi-sound source environment: on the one hand, by modeling the time-frequency characteristics and state transition laws of the sound source, the weight of the target user's sound source is dynamically adjusted to ensure that the target user can still be locked in the voice switching or sudden noise scenario; on the other hand, based on the real-time calculation of the recursive algorithm, the probability of all sound sources can be quickly updated, thereby achieving efficient dynamic sound source priority adjustment, further solving the technical problem in the prior art that the target sound source is easily interrupted and the extraction is incomplete due to environmental interference or overlapping of multiple sound sources, and providing a more stable and intelligent basic support for the voice interaction system in complex scenarios with high noise and multiple voice competition. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only for the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 A schematic diagram of system function modules according to an embodiment of the present invention;

[0044] Figure 2 Schematic diagram of a target user sound source recognition and extraction module according to an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. At the same time, it is explained here that in order to make the embodiments more detailed, the following embodiments are the best and preferred embodiments, and those skilled in the art may also adopt other alternatives to implement some known technologies; and the accompanying drawings are only for more specific description of the embodiments, and are not intended to specifically limit the present invention.

[0046] It should be noted that the references to "one embodiment", "embodiment", "exemplary embodiments", "some embodiments" and the like in the specification indicate that the embodiments described may include specific features, structures or characteristics, but not every embodiment may include the specific features, structures or characteristics. In addition, when a specific feature, structure or characteristic is described in conjunction with an embodiment, it should be within the knowledge of a person skilled in the art to implement such feature, structure or characteristic in conjunction with other embodiments (whether or not explicitly described).

[0047] In general, a term can be understood, at least in part, from its use in context. For example, depending, at least in part, on the context, the term "one or more" as used herein can be used to describe any feature, structure, or characteristic in the singular sense, or can be used to describe a combination of features, structures, or characteristics in the plural sense. Additionally, the term "based on" can be understood as not necessarily intended to convey an exclusive set of factors, but can instead, depending, at least in part, on the context, allow for the presence of other factors that are not necessarily explicitly described.

[0048] like Figure 1-Figure 2 As shown, a voice interaction system of an intelligent humanoid companion robot comprises:

[0049] Voice input acquisition and preprocessing module: The distributed microphone array collects the composite voice signals and voice trigger commands in the surrounding environment in real time. The target user uses the voice trigger command to trigger the initial dialogue with the robot. The voice trigger command is a specific command set by the target user to activate the dialogue with the robot. For example, the activation voice command in Xiaomi's Xiaoai speaker is "Xiaoai classmate". After the initial dialogue activation trigger, the target user and the robot have a normal dialogue. However, in the subsequent continuous dialogue, if a noisy environment is processed, there may be multiple environmental voice sources. At this time, the robot will be unable to identify which is the target user's voice and which is the voice of other people in the environmental background.

[0050] Multi-source speech separation module: In continuous conversation, the collected composite speech signal is decomposed in the time-frequency domain by a sparse decomposition algorithm, and the composite speech signal is represented as a combination of multiple potential sound source signals; the separation network is used to generate a time-frequency mask, and multiple potential sound source signal combinations are independently processed to separate them into multiple independent sound source signals;

[0051] Target user sound source identification and extraction module: Based on the separated multiple sound source signals, a dynamic Bayesian network is constructed to model the time characteristics and frequency characteristics of multiple sound source signals, and the dynamic Bayesian network is used to predict the changes of the target sound source in continuous time frames to extract the target user sound source signal;

[0052] Semantic parsing and output module: Use natural language processing technology to parse the speech content of the target user's sound source signal to extract semantic information, and match it with the knowledge base; based on the matching results, generate natural and fluent speech responses through speech synthesis technology; specifically include:

[0053] Target user speech content analysis: Based on the natural language processing technology of the pre-trained language model, the target user's sound source signal is converted into text content, and the text content is semantically annotated and dependency analyzed to extract key information units, including entity, action, time, and location semantic components;

[0054] Knowledge base matching of semantic information: The extracted semantic information is matched with the robot’s built-in knowledge base to identify the user’s intention. The knowledge base includes predefined dialogue templates and context-related information. During the matching process, semantic similarity calculation (cosine similarity) is combined to match or complete fuzzy instructions.

[0055] Natural and fluent voice response generation: Based on the knowledge base matching results, speech synthesis technology (WaveNet's speech synthesis model) is used to generate voice response content, and speech synthesis parameters (such as speaking speed, intonation, and emotional expression) are adjusted to ensure that the voice response is natural and close to the user's emotional state.

[0056] The multi-source speech separation module specifically includes:

[0057] Time-frequency domain decomposition submodule: Perform short-time Fourier transform on the collected composite speech signal, convert the signal from the time domain to the time-frequency domain, generate the signal's time-frequency spectrogram, apply the non-negative matrix decomposition algorithm based on sparse decomposition, and use the sparse characteristics of the signal to decompose the composite speech signal into multiple basic signal components, and map the decomposed basic signal components into multiple potential sound source signal combinations, each of which corresponds to a specific time-frequency feature;

[0058] The short-time Fourier transform is expressed as:

[0059] ,in, represents the composite signal representation in the time-frequency domain, is the time-frequency point, is the time frame, is the frequency, is the composite speech signal in the time domain, is the time point index, It is a window function, which is used to intercept a section of the time domain signal for transformation. represents the Fourier transform kernel function, which represents the frequency The sine wave component of

[0060] Separation network generates time-frequency mask submodule: Use the separation network to analyze the time-frequency spectrogram, extract the feature weights of each potential sound source signal, generate a time-frequency mask based on the feature weights, each time-frequency mask corresponds to the time-frequency feature distribution of a potential sound source signal, apply the generated time-frequency mask to reconstruct the composite speech signal, and reallocate each time-frequency unit of the composite speech signal to the original composite speech signal;

[0061] Separation submodule of independent sound source signals: restore the time-frequency signal processed by time-frequency masking to a time domain signal, generate multiple independent sound source signals, and output the separated independent sound source signals to provide high-quality input for the target user sound source identification and extraction module.

[0062] The non-negative matrix factorization algorithm is expressed as: ,in, is the time-frequency spectrogram matrix, is the basis matrix, of size is the number of potential sound source signals, is the coefficient matrix, size is , which represents the time activation intensity of each basis, is solved by defining the optimization objective and , decomposing the composite signal into multiple basic signal components;

[0063] The optimization objective is defined as: ,in, is the input time-frequency spectrum matrix, size is is the frequency dimension, is the number of time frames, Represents the F norm of the matrix, which is used to measure the error of matrix decomposition. , represents the non-negativity constraint, ensuring the physical meaning of the signal.

[0064] The input of the separation network is a time-frequency spectrogram , output time-frequency mask , indicating the sound source At time-frequency point (time-frequency point) The weights on are used to distribute the energy of the composite signal, and the generation of the time-frequency mask is expressed as: ;

[0065] The signal reconstruction is expressed as: ,in, Indicates the sound source The time-frequency mask of Indicates the sound source The amplitude at the time-frequency point, Represents the sum of the amplitudes of all sound sources at the time-frequency points, Indicates the separated sound source The time-frequency representation of is the time-frequency representation of the original composite signal, using the time-frequency mask The original composite signal The time-frequency points are weighted to obtain the separated signal, and the separated time-frequency domain representation signal The signal is converted back to the time domain by inverse short-time Fourier transform.

[0066] The time-frequency signal processed by the time-frequency mask is restored to the time domain signal using the inverse short-time Fourier transform, which is expressed as:

[0067] ,in, Indicates the sound source The restored time domain signal is Is the sound source The time-frequency domain representation of is a window function, is the inverse Fourier transform kernel function.

[0068] It also includes the training of the separation network and the time domain reconstruction error training based on the loss function:

[0069] ,in, is the training loss function of the network, is the predicted sound source The time domain signal, It is a real sound source The time domain signal, is the Euclidean distance, which is used to measure the error of the reconstructed signal.

[0070] The target user sound source recognition and extraction module specifically includes:

[0071] Construct a dynamic Bayesian network model: The separated multiple sound source signals are represented as observation variables of the dynamic Bayesian network. Including time features and frequency characteristics , define the state variables For sound source In time frame active state, and through the transition probability Modeling the changing probability trend of sound sources in continuous time frames;

[0072] Transition probability ; The state is stable, the state The probability of remaining unchanged is , the probability of changing to other states is ;

[0073] Temporal feature modeling: For each sound source , extract its time features , including short-term energy characteristics and time continuity, using time series analysis methods to define the time characteristic observation probability , indicating the status Impact on temporal characteristics;

[0074] Frequency feature modeling: Extract each sound source Frequency characteristics , including the main frequency components and spectral energy distribution, defining the frequency feature observation probability , indicating the status Impact on frequency characteristics;

[0075] Joint modeling and prediction: Based on the Bayesian formula, the time characteristics and frequency characteristics are combined to calculate the sound source through joint modeling In time frame The posterior probability of the state on all time frames is calculated by recursive algorithm , predict the target user sound source Changing trends in consecutive time frames;

[0076] Target sound source identification and extraction: According to the prior information (directional characteristics) of the target user's sound source, the posterior probabilities of all sound sources are sorted, and the sound source corresponding to the maximum probability is selected as the target user's sound source:

[0077] ,in, Represents the sound source number of the target user, which is obtained by maximizing the posterior probability Obtained, among all separated sound sources In the above example, the sound source with the highest posterior probability is selected as the sound source of the target user.

[0078] The temporal feature modeling is expressed as: ;

[0079] The temporal feature observation probability is defined as:

[0080] ;in, Is the sound source In time frame The short-term energy characteristics of Is the sound source At the time-frequency point The frequency amplitude, It is a time feature The observation probability follows a Gaussian distribution , and Representation and Status The relevant mean and variance characterize the impact of the state on the temporal characteristics.

[0081] The frequency characteristic modeling is expressed as: ；

[0082] The frequency feature observation probability is defined as:

[0083] ;in, Is the sound source In frequency The normalized frequency component (frequency characteristic) of is the observation probability of the frequency feature, which follows a multinomial distribution, Representation and Status The relevant frequency component distribution parameters are used to describe the influence of the state on the frequency characteristics.

[0084] Calculating sound sources through joint modeling In time frame The posterior probability is expressed as:

[0085] ,in, Yes Status The posterior probability of is the observation probability of the time feature, is the observation probability of the frequency feature, is the transition probability, indicating that the state changes from Transfer to probability.

[0086] The recursive algorithm uses the forward recursive-backward recursive algorithm:

[0087] The forward recursive algorithm is expressed as:

[0088] ;in, is the forward probability, indicating the sound source In time frame Status is The joint probability of is the transition probability, is the joint observation probability of time features and frequency features, which is equivalent to: , Represents all possible previous states Perform the summation and implement recursion.

[0089] The backward recursive algorithm is expressed as:

[0090] ,in, is the backward probability, which means from the time frame Start, the status is The possibility of is similar to the forward recursive algorithm, and the backward probability is calculated by recursively from back to front.

[0091] An intelligent humanoid companion robot uses the above-mentioned voice interaction system to realize voice interaction with a user.

[0092] The present invention covers any substitution, modification, equivalent method and scheme made on the essence and scope of the present invention. In order to make the public have a thorough understanding of the present invention, specific details are described in detail in the following preferred embodiments of the present invention, but those skilled in the art can fully understand the present invention without the description of these details. In addition, in order to avoid unnecessary confusion about the essence of the present invention, well-known methods, processes, procedures, components and circuits are not described in detail.

[0093] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A voice interaction system for an intelligent humanoid companion robot, characterized in that: include: Voice input acquisition and preprocessing module: The distributed microphone array collects the composite voice signals and voice trigger commands in the surrounding environment in real time. The target user uses the voice trigger command to trigger the initial dialogue with the robot. Multi-source speech separation module: decomposes the collected composite speech signal in the time-frequency domain through a sparse decomposition algorithm, and represents the composite speech signal as a combination of multiple potential sound source signals; The separation network is used to generate a time-frequency mask, and multiple potential sound source signals are combined and processed independently to separate them into multiple independent sound source signals, including: Time-frequency domain decomposition submodule: Perform short-time Fourier transform on the collected composite speech signal, convert the signal from the time domain to the time-frequency domain, generate the signal's time-frequency spectrogram, apply the non-negative matrix decomposition algorithm based on sparse decomposition, and use the sparse characteristics of the signal to decompose the composite speech signal into multiple basic signal components, and map the decomposed basic signal components into multiple potential sound source signal combinations, each of which corresponds to a specific time-frequency feature; Separation network generates time-frequency mask submodule: Use the separation network to analyze the time-frequency spectrogram, extract the feature weights of each potential sound source signal, generate a time-frequency mask based on the feature weights, each time-frequency mask corresponds to the time-frequency feature distribution of a potential sound source signal, apply the generated time-frequency mask to reconstruct the composite speech signal, and reallocate each time-frequency unit of the composite speech signal to the original composite speech signal; Separation submodule of independent sound source signals: restore the time-frequency signal processed by the time-frequency mask to a time domain signal to generate multiple independent sound source signals; Target user sound source identification and extraction module: Based on the separated multiple sound source signals, a dynamic Bayesian network is constructed to model the time characteristics and frequency characteristics of multiple sound source signals, and the dynamic Bayesian network is used to predict the changes of the target sound source in continuous time frames to extract the target user sound source signal; Semantic parsing and output module: Use natural language processing technology to parse the speech content of the target user's sound source signal to extract semantic information, and match it with the knowledge base; based on the matching results, generate natural and fluent speech responses through speech synthesis technology; specifically include: Target user speech content analysis: Based on the natural language processing technology of the pre-trained language model, the target user's sound source signal is converted into text content, and the text content is semantically annotated and dependency analyzed to extract key information units, including entity, action, time, and location semantic components; Knowledge base matching of semantic information: The extracted semantic information is matched with the robot's built-in knowledge base to identify the user's intention. The knowledge base includes predefined dialogue templates and context-related information. During the matching process, the fuzzy instructions are matched or completed in combination with semantic similarity calculation; Natural and fluent voice response generation: Based on the knowledge base matching results, speech synthesis technology is used to generate voice response content.

2. The voice interaction system of an intelligent humanoid companion robot according to claim 1, characterized in that: The non-negative matrix factorization algorithm is expressed as: ,in, is the time-frequency spectrogram matrix, is the basis matrix, of size is the number of potential sound source signals, is the coefficient matrix, size is , which represents the time activation intensity of each basis, is solved by defining the optimization objective and , decomposing the composite signal into multiple basic signal components; The optimization objective is defined as: ,in, is the input time-frequency spectrum matrix, size is is the frequency dimension, is the number of time frames, Represents the F norm of the matrix, which is used to measure the error of matrix decomposition. , represents the non-negativity constraint.

3. The voice interaction system of an intelligent humanoid companion robot according to claim 2, characterized in that: The input of the separation network is a time-frequency spectrogram , output time-frequency mask , indicating the sound source At the time-frequency point The generation of the time-frequency mask is expressed as: ; The signal reconstruction is expressed as: ,in, Indicates the sound source The time-frequency mask of Indicates the sound source The amplitude at the time-frequency point, Represents the sum of the amplitudes of all sound sources at the time-frequency points, Indicates the separated sound source The time-frequency representation of is the time-frequency representation of the original composite signal, using the time-frequency mask The original composite signal The time-frequency points are weighted.

4. The voice interaction system of an intelligent humanoid companion robot according to claim 3, characterized in that: The time-frequency signal processed by the time-frequency mask is restored to a time-domain signal by inverse short-time Fourier transform, which is expressed as: ,in, Indicates the sound source The restored time domain signal is Is the sound source The time-frequency domain representation of is a window function, is the inverse Fourier transform kernel function.

5. The voice interaction system of an intelligent humanoid companion robot according to claim 1, characterized in that: The target user sound source recognition and extraction module specifically includes: Construct a dynamic Bayesian network model: The separated multiple sound source signals are represented as observation variables of the dynamic Bayesian network. Including time features and frequency characteristics , define the state variables For sound source In time frame active state, and through the transition probability Modeling the changing probability trend of sound sources in continuous time frames; Temporal feature modeling: For each sound source , extract its time features , including short-term energy characteristics and time continuity, using time series analysis methods to define the time characteristic observation probability , indicating the status Impact on temporal characteristics; Frequency feature modeling: Extract each sound source Frequency characteristics , including the main frequency components and spectral energy distribution, defining the frequency feature observation probability , indicating the status Impact on frequency characteristics; Joint modeling and prediction: Combine time features and frequency features to calculate the sound source through joint modeling In time frame The posterior probability of the state on all time frames is calculated by recursive algorithm , predict the target user sound source Changing trends in consecutive time frames; Target sound source identification and extraction: According to the prior information of the target user's sound source, the posterior probabilities of all sound sources are sorted, and the sound source corresponding to the maximum probability is selected as the target user's sound source: ,in, Represents the sound source number of the target user, which is obtained by maximizing the posterior probability Got it.

6. The voice interaction system of an intelligent humanoid companion robot according to claim 5, characterized in that: The temporal feature modeling is expressed as: ; The temporal feature observation probability is defined as: ;in, Is the sound source In time frame The short-term energy characteristics of Is the sound source At the time-frequency point The frequency amplitude, It is a time feature The observation probability of and Representation and Status The relevant mean and variance characterize the impact of the state on the temporal characteristics.

7. The voice interaction system of an intelligent humanoid companion robot according to claim 6, characterized in that: The frequency characteristic modeling is expressed as: ； The frequency feature observation probability is defined as: ;in, Is the sound source In frequency The normalized frequency component of is the observation probability of the frequency feature, which follows a multinomial distribution, Representation and Status Related frequency component distribution parameters.

8. The voice interaction system of an intelligent humanoid companion robot according to claim 7, characterized in that: The sound source is calculated by joint modeling In time frame The posterior probability is expressed as: ,in, Yes Status The posterior probability of is the observation probability of the time feature, is the observation probability of the frequency feature, is the transition probability, indicating that the state changes from Transfer to probability.

9. An intelligent humanoid companion robot, using the voice interaction system as described in any one of claims 1 to 8, to achieve voice interaction with a user.

Images