Intelligent magic mirror interaction method and system based on voice recognition

Abstract

The invention relates to the technical field of bioinformatics, and discloses an intelligent magic mirror interaction method and system based on voice recognition, and the method comprises the steps: collecting a voice signal of an intelligent magic mirror through a target multi-microphone module, carrying out the noise suppression of the voice signal, and obtaining a noise-suppressed voice signal, extracting a voice signal voiceprint of the noise suppression voice signal, determining a target voice signal and a target voice user of the intelligent magic mirror, calculating a user position of the target voice user, recognizing a user intention of the target voice signal by using a multi-language intention recognition model, calculating an intention priority of the user intention, and obtaining an intention priority of the user intention; determining a sequence intention task of the target voice user; constructing a mirror surface interaction instruction of the intelligent magic mirror, and defining a voice interaction instruction of the intelligent magic mirror; and determining an interaction behavior in the process of executing the sequence intention task, and constructing an interaction page of the target voice user and an interaction mirror surface in the intelligent magic mirror. The interaction effect of the intelligent magic mirror can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of bioinformatics, and in particular to an intelligent magic mirror interaction method and system based on speech recognition. Background Art

[0002] Smart interactive magic mirror refers to a modern smart home device that integrates multiple technologies such as display, touch, voice recognition, and artificial intelligence. The main purpose of the smart interactive magic mirror is to provide users with a convenient and personalized interactive experience. At the same time, by integrating the smart home control system, it becomes an important part of smart family life.

[0003] At present, smart interactive magic mirrors mainly rely on basic voice recognition technology. Users obtain information or perform operations by directly issuing commands to the mirror. This method has some limitations: first, it is relatively simple to recognize the user's voice commands and cannot effectively distinguish the voices of different users, resulting in the inability to provide a personalized service experience, which makes the magic mirror's interaction not smart enough. Summary of the invention

[0004] The present invention provides an intelligent magic mirror interaction method and system based on voice recognition, the main purpose of which is to improve the interaction effect of the intelligent magic mirror.

[0005] To achieve the above object, the present invention provides a smart magic mirror interaction method based on speech recognition, comprising:

[0006] Identify the magic mirror structure parameters of the smart magic mirror, construct a multi-microphone module of the smart magic mirror based on the magic mirror structure parameters, calculate the far-field speech recognition coefficient of the multi-microphone module, and optimize the multi-microphone module according to the far-field speech recognition coefficient to obtain a target multi-microphone module;

[0007] The target multi-microphone module is used to collect the voice signal of the smart magic mirror, and noise suppression is performed on the voice signal to obtain a noise-suppressed voice signal, and the voice signal voiceprint of the noise-suppressed voice signal is extracted, and the matching degree between the voice signal voiceprint and the voiceprint of a preset magic mirror voiceprint library is calculated;

[0008] Based on the voiceprint matching degree, the target voice signal and the target voice user of the smart magic mirror are determined, the user position of the target voice user is calculated, and the user intention of the target voice signal is identified by using the trained multilingual intention recognition model, and the intention order of the user intention is identified. When the intention order does not meet the preset conditions, the intention priority of the user intention is calculated, and the sequence intention task of the target voice user is determined according to the intention priority;

[0009] Based on the sequence intention task, construct the mirror interaction instruction of the smart magic mirror, and when the mirror interaction instruction requires voice broadcast, define the voice interaction instruction of the smart magic mirror based on the user position;

[0010] Based on the mirror interaction instructions and the voice interaction instructions, the interactive behavior during the execution of the sequence intention task is determined, and according to the interactive behavior, an interactive page between the target voice user and the interactive mirror in the smart magic mirror is constructed.

[0011] Optionally, constructing a multi-microphone module of the smart magic mirror based on the magic mirror structure parameters includes:

[0012] Determine the voice collection function index of the smart magic mirror;

[0013] Based on the voice collection function indicator, define the target microphone of the smart magic mirror;

[0014] Constructing a multi-microphone layout of the smart magic mirror according to the magic mirror structure parameters and the voice collection function indicators;

[0015] Calculating the sound receiving efficiency of the target microphone corresponding to the multi-microphone layout;

[0016] When the sound receiving efficiency meets a preset sound receiving efficiency threshold, a multi-microphone module of the smart magic mirror is constructed.

[0017] Optionally, the calculating the sound receiving efficiency of the target microphone corresponding to the multi-microphone layout includes:

[0018] analyzing microphone parameters of a target microphone corresponding to the multi-microphone layout;

[0019] Based on the microphone parameters, calculating the signal reception strength of the target microphone;

[0020] According to the signal reception strength and the microphone parameters, the sound reception efficiency of the target microphone is calculated using the following formula:

[0021]

[0022] Among them, ρ represents the sound receiving efficiency of the target microphone, RSSI a Indicates the signal reception strength of the ath microphone, RSSI max,a Indicates the maximum signal receiving strength of the ath microphone, S a Indicates the directional factor of the ath microphone in the microphone parameters, DF a represents the attenuation transmission factor of the a-th microphone in the microphone parameters, N represents the number of target microphones, and a represents the a-th microphone.

[0023] Optionally, calculating the far-field speech recognition coefficient of the multi-microphone module includes:

[0024] defining a far-field condition of a sound source of the multi-microphone module;

[0025] Based on the far-field condition of the sound source, construct a sound source model and an environment model of the multi-microphone module;

[0026] Establishing an acoustic model of the multi-microphone module through the sound source model and the environment model;

[0027] Based on the acoustic model, analyzing the sound source distribution of the multi-microphone module corresponding to the sound source scene;

[0028] Calculating the sound pressure level of the target microphone corresponding to the multi-microphone module according to the sound source distribution;

[0029] Based on the sound pressure level, analyzing the speech recognition rate of the target microphone corresponding to the multi-microphone module under the far-field condition of the sound source;

[0030] The far-field speech recognition coefficient of the multi-microphone module is calculated according to the speech recognition rate.

[0031] Optionally, performing noise suppression on the speech signal to obtain a noise suppressed speech signal includes:

[0032] Performing time domain processing on the speech signal to obtain a time domain processed signal;

[0033] Performing frequency domain processing on the time domain processed signal to obtain a frequency domain processed signal;

[0034] Constructing a sparse representation dictionary of the frequency domain processed signal;

[0035] Marking the sparse representation of the frequency domain processed signal by the sparse representation dictionary;

[0036] Determining a signal noise of the frequency-domain processed signal according to the sparse representation;

[0037] The signal noise is suppressed to obtain a noise-suppressed speech signal of the frequency domain processed signal.

[0038] Optionally, extracting a voice signal voiceprint of the noise suppressed voice signal includes:

[0039] marking an active portion of the noise suppressed speech signal;

[0040] According to the active part, the noise suppressed speech signal is enhanced to obtain an enhanced speech signal;

[0041] Segmenting the enhanced speech signal to obtain a short-time frame speech signal;

[0042] Extracting the voiceprint features of the short-time frame speech signal;

[0043] Identifying a feature contribution of the voiceprint feature, and determining a target voiceprint feature among the voiceprint features based on the feature contribution;

[0044] Aggregating the target voiceprint features into a fixed-length feature vector;

[0045] Based on the fixed-length feature vector, a speech signal voiceprint of the short-time frame speech signal is determined.

[0046] Optionally, extracting the voiceprint feature of the short-time frame speech signal includes:

[0047] Preprocessing the short-time frame speech signal to obtain a spectrogram;

[0048] Output the power spectrum of the spectrogram using a preset Mel filter;

[0049] Based on the power spectrum of the Mel filter output spectrogram, the Mel frequency cepstrum coefficients of the short-time frame speech signal are calculated using the following formula:

[0050]

[0051] Among them, MFCC(r) represents the rth Mel frequency cepstrum coefficient of the short-time frame speech signal, F(v) represents the power spectrum output by the vth Mel filter of the short-time frame speech signal, v represents the index of the Mel filter, m represents the number of Mel filters, cos represents the cosine function, π represents pi, r represents the index of the Mel frequency cepstrum coefficient, and log represents the natural logarithm function;

[0052] Based on the Mel-frequency cepstral coefficients, the voiceprint features of the short-time frame speech signal are extracted.

[0053] Optionally, the identifying the user intention of the target voice signal by using a trained multilingual intention recognition model includes:

[0054] Constructing a multilingual model of the target speech signal;

[0055] Annotating the target speech signal based on the multilingual model to obtain an annotated speech signal;

[0056] Extracting language features of the annotated speech signal;

[0057] Based on the language features, using the multilingual intention recognition model to recognize the initial user intention of the annotated voice signal;

[0058] calculating the validity of the initial user intent;

[0059] When the validity meets a preset validity threshold, the initial user intention is used as the user intention.

[0060] Optionally, constructing an interactive page between the target voice user and the interactive mirror surface in the smart magic mirror according to the interactive behavior includes:

[0061] Identifying the target voice user's response interaction information to the interaction behavior;

[0062] Analyzing the interaction intention of the smart magic mirror based on the response interaction information;

[0063] Determining the interaction logic of the smart magic mirror according to the interaction intention;

[0064] According to the interaction logic, an interaction page between the target voice user and the interactive mirror in the smart magic mirror is constructed.

[0065] In order to solve the above problems, the present invention also provides an intelligent magic mirror interaction system based on speech recognition, the system comprising:

[0066] A multi-microphone module construction module is used to identify the magic mirror structure parameters of the smart magic mirror, construct the multi-microphone module of the smart magic mirror based on the magic mirror structure parameters, calculate the far-field speech recognition coefficient of the multi-microphone module, and optimize the multi-microphone module according to the far-field speech recognition coefficient to obtain a target multi-microphone module;

[0067] A voiceprint matching module, used to collect the voice signal of the smart magic mirror through the target multi-microphone module, perform noise suppression on the voice signal to obtain a noise-suppressed voice signal, extract the voice signal voiceprint of the noise-suppressed voice signal, and calculate the voiceprint matching degree between the voice signal voiceprint and a preset magic mirror voiceprint library;

[0068] An intention task recognition module is used to determine the target voice signal and the target voice user of the smart magic mirror based on the voiceprint matching degree, calculate the user position of the target voice user, and use the trained multilingual intention recognition model to recognize the user intention of the target voice signal, recognize the intention order of the user intention, and when the intention order does not meet the preset conditions, calculate the intention priority of the user intention, and determine the sequence intention task of the target voice user according to the intention priority;

[0069] An interactive instruction construction module, used to construct a mirror interaction instruction of the smart magic mirror based on the sequence intention task, and when the mirror interaction instruction requires voice broadcast, define the voice interaction instruction of the smart magic mirror based on the user position;

[0070] An interactive page generation module is used to determine the interactive behavior in the process of executing the sequence intention task based on the mirror interaction instructions and the voice interaction instructions, and to construct an interactive page between the target voice user and the interactive mirror in the smart magic mirror according to the interactive behavior.

[0071] The present invention constructs a multi-microphone module of the smart magic mirror based on the magic mirror structural parameters to construct an efficient, stable and user-friendly smart magic mirror multi-microphone module, thereby efficiently collecting voice signals; the present invention calculates the far-field voice recognition coefficient of the multi-microphone module, which can be used as a basis for optimizing the microphone layout, thereby improving the voice reception capability of the microphone, thereby improving the user experience; further, the present invention optimizes the multi-microphone module according to the far-field voice recognition coefficient to obtain a target multi-microphone module, which can optimize the multi-microphone module to improve its voice recognition performance under far-field conditions; further, the present invention performs noise suppression on the voice signal to obtain a noise-suppressed voice signal, which can effectively separate noise and voice to obtain a high-quality noise-suppressed voice signal, thereby improving The later speech recognition effect, further, the present invention can provide a basis for the later voiceprint matching by extracting the voice signal voiceprint of the noise suppression voice signal, and then, the present invention uses the trained multilingual intention recognition model to identify the user intention of the target voice signal to meet the needs of users of different languages ​​and provide a better user experience. The present invention identifies the intention order of the user intention. When the intention order does not meet the preset conditions, the intention priority of the user intention is calculated. According to the intention priority, the sequence intention task of the target voice user is determined to be able to adapt to the user scenarios and needs of multi-tasks, thereby improving the user experience. Finally, the present invention can realize the intelligent interaction of the smart magic mirror by constructing the interactive page between the target voice user and the interactive mirror in the smart magic mirror according to the interactive behavior. Therefore, the present invention can improve the interactive effect of the smart magic mirror. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 A flow chart of a smart magic mirror interaction method based on speech recognition provided by an embodiment of the present invention;

[0073] Figure 2 A functional module diagram of an intelligent magic mirror interaction system based on speech recognition provided by an embodiment of the present invention;

[0074] Figure 3A schematic diagram of the structure of an electronic device of an intelligent magic mirror interaction system based on speech recognition provided by an embodiment of the present invention;

[0075] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings in conjunction with the embodiments. DETAILED DESCRIPTION

[0076] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0077] The embodiment of the present application provides an intelligent magic mirror interaction method based on voice recognition. The execution subject of the intelligent magic mirror interaction method based on voice recognition includes but is not limited to at least one of the electronic devices such as a server, a terminal, etc. that can be configured to execute the method provided by the embodiment of the present application. In other words, the intelligent magic mirror interaction method based on voice recognition can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms.

[0078] Reference Figure 1 FIG. 1 is a flow chart of a method for interacting with a smart magic mirror based on speech recognition according to an embodiment of the present invention. In this embodiment, the method for interacting with a smart magic mirror based on speech recognition includes:

[0079] S1. Identify the magic mirror structure parameters of the smart magic mirror, construct a multi-microphone module of the smart magic mirror based on the magic mirror structure parameters, calculate the far-field speech recognition coefficient of the multi-microphone module, and optimize the multi-microphone module according to the far-field speech recognition coefficient to obtain a target multi-microphone module.

[0080] It should be explained that the magic mirror structural parameters refer to a series of technical parameters used to describe the physical form and functional layout of the smart magic mirror. These parameters usually include the size, shape, material, display technology specifications and other parameters of the magic mirror.

[0081] The present invention constructs a multi-microphone module of the intelligent magic mirror based on the magic mirror structural parameters to construct an efficient, stable and user-friendly intelligent magic mirror multi-microphone module, thereby efficiently collecting voice signals.

[0082] In detail, the multi-microphone module of the smart magic mirror is constructed based on the magic mirror structure parameters, including:

[0083] Determine the voice collection function index of the smart magic mirror;

[0084] Based on the voice collection function indicator, define the target microphone of the smart magic mirror;

[0085] Constructing a multi-microphone layout of the smart magic mirror according to the magic mirror structure parameters and the voice collection function indicators;

[0086] Calculating the sound receiving efficiency of the target microphone corresponding to the multi-microphone layout;

[0087] When the sound receiving efficiency meets a preset sound receiving efficiency threshold, a multi-microphone module of the smart magic mirror is constructed.

[0088] Among them, the voice collection function index refers to a set of technical parameters used to measure the performance of the smart magic mirror voice collection system, such as signal-to-noise ratio, frequency response, sensitivity and other parameters; the target microphone refers to the microphone model selected according to the voice collection function index; the multi-microphone layout refers to the specific arrangement of microphones on the smart magic mirror; the sound reception efficiency refers to the ability of the microphone layout to capture the sound source; the sound reception efficiency threshold refers to the threshold used to judge whether the microphone layout is effective; the multi-microphone module refers to a system composed of multiple microphones, which includes not only microphone hardware, but also related electronic circuits, signal processing algorithms and software.

[0089] Furthermore, the calculating the sound receiving efficiency of the target microphone corresponding to the multi-microphone layout includes:

[0090] analyzing microphone parameters of a target microphone corresponding to the multi-microphone layout;

[0091] Based on the microphone parameters, calculating the signal reception strength of the target microphone;

[0092] According to the signal reception strength and the microphone parameters, the sound reception efficiency of the target microphone is calculated using the following formula:

[0093]

[0094] Among them, ρ represents the sound receiving efficiency of the target microphone, RSSI a Indicates the signal reception strength of the ath microphone, RSSI max,a Indicates the maximum signal receiving strength of the ath microphone, S a Indicates the directional factor of the ath microphone in the microphone parameters, DF arepresents the attenuation transmission factor of the a-th microphone in the microphone parameters, N represents the number of target microphones, and a represents the a-th microphone.

[0095] Among them, the microphone parameters include parameters such as the microphone's directivity, sensitivity, frequency response, noise coefficient, dynamic range, directivity factor, attenuation transmission factor, etc. The signal reception intensity refers to the signal intensity actually received by the microphone, which is affected by factors such as the sound source intensity, microphone directivity, and attenuation during sound wave propagation. The directivity factor refers to the description of the sound pressure sensitivity of the microphone in different directions. The attenuation transmission factor refers to the degree of influence of factors such as environmental noise, echo and reverberation on the microphone's received signal. Among them, the attenuation transmission factor is determined by simulation.

[0096] The far-field speech recognition coefficient of the multi-microphone module calculated by the present invention can be used as a basis for optimizing the microphone layout, thereby improving the speech receiving capability of the microphone and thus improving the user experience.

[0097] In detail, the calculating the far-field speech recognition coefficient of the multi-microphone module includes:

[0098] defining a far-field condition of a sound source of the multi-microphone module;

[0099] Based on the far-field condition of the sound source, construct a sound source model and an environment model of the multi-microphone module;

[0100] Establishing an acoustic model of the multi-microphone module through the sound source model and the environment model;

[0101] Based on the acoustic model, analyzing the sound source distribution of the multi-microphone module corresponding to the sound source scene;

[0102] Calculating the sound pressure level of the target microphone corresponding to the multi-microphone module according to the sound source distribution;

[0103] Based on the sound pressure level, analyzing the speech recognition rate of the target microphone corresponding to the multi-microphone module under the far-field condition of the sound source;

[0104] The far-field speech recognition coefficient of the multi-microphone module is calculated according to the speech recognition rate.

[0105] Among them, the far-field condition of the sound source means that in acoustics, when the distance between the sound source and the receiver is much larger than the size of the sound source, the sound wave can be regarded as a plane wave. This condition is called the far-field condition. The sound source model refers to the mathematical description of the characteristics of the sound source, including parameters such as the position, intensity, and directivity of the sound source. The environmental model refers to the mathematical description of the environmental characteristics of the sound wave propagation, including parameters such as the geometry of the room, the wall material, and the layout of furniture in the room. The acoustic model refers to a model that combines the sound source model and the environmental model, which is used to predict the propagation of sound waves in a specific environment. The sound source distribution refers to the distribution of the sound source in space in the acoustic model, including the position and intensity distribution of the sound source. The sound pressure level refers to an important parameter for measuring the intensity of sound waves. The speech recognition rate refers to the proportion of voice commands that the speech recognition system can correctly recognize under specific conditions. The far-field speech recognition coefficient refers to a performance indicator used to measure the speech recognition system of a multi-microphone module under far-field conditions.

[0106] Optionally, the sound source model and the environment model of the multi-microphone module based on the far-field condition of the sound source can be constructed and simulated by acoustic simulation software (such as CST, COMSOL Multiphysics, ANSYS Acoustics, etc.).

[0107] The present invention optimizes the multi-microphone module according to the far-field speech recognition coefficient to obtain a target multi-microphone module, and the multi-microphone module can be optimized to improve its speech recognition performance under far-field conditions. The target multi-microphone module refers to a microphone model obtained after optimizing the multi-microphone module. In detail, the optimization of the multi-microphone module includes adjusting the parameters of the microphone, such as microphone spacing, microphone sensitivity and other parameters.

[0108] S2. Collect the voice signal of the smart magic mirror through the target multi-microphone module, perform noise suppression on the voice signal to obtain a noise-suppressed voice signal, extract the voice signal voiceprint of the noise-suppressed voice signal, and calculate the voiceprint matching degree between the voice signal voiceprint and a preset magic mirror voiceprint library.

[0109] It should be explained that the voice signal refers to the meaningful information extracted from the user's voice by the target multi-microphone module.

[0110] The present invention performs noise suppression on the speech signal to obtain a noise suppressed speech signal which can effectively separate noise and speech, and obtain a high-quality noise suppressed speech signal, thereby improving the subsequent speech recognition effect.

[0111] In detail, the performing noise suppression on the speech signal to obtain a noise suppressed speech signal includes:

[0112] Performing time domain processing on the speech signal to obtain a time domain processed signal;

[0113] Performing frequency domain processing on the time domain processed signal to obtain a frequency domain processed signal;

[0114] Constructing a sparse representation dictionary of the frequency domain processed signal;

[0115] Marking the sparse representation of the frequency domain processed signal by the sparse representation dictionary;

[0116] Determining a signal noise of the frequency-domain processed signal according to the sparse representation;

[0117] The signal noise is suppressed to obtain a noise-suppressed speech signal of the frequency domain processed signal.

[0118] Among them, the time domain processed signal refers to the signal obtained after preprocessing the original speech signal, usually including filtering, amplification and other operations to improve the signal quality, the frequency domain processed signal refers to the signal obtained after fast Fourier transform (FFT) of the time domain processed signal, the sparse representation dictionary refers to a matrix containing speech signal features, the sparse representation refers to the process of encoding the feature vector of the frequency domain processed signal into a sparse form using a sparse representation dictionary, the signal noise refers to the noise part related to the speech signal in the frequency domain processed signal, and the noise suppressed speech signal refers to a pure speech signal obtained by subtracting the estimated noise part from the frequency domain processed signal.

[0119] Optionally, the sparse representation of the frequency-domain processed signal marked by the sparse representation dictionary can be used to represent sparse features of the speech signal through the atoms (basis) therein.

[0120] The present invention can provide a basis for subsequent voiceprint matching by extracting the voice signal voiceprint of the noise suppressed voice signal.

[0121] In detail, the extracting the voice signal voiceprint of the noise suppressed voice signal includes:

[0122] marking an active portion of the noise suppressed speech signal;

[0123] According to the active part, the noise suppressed speech signal is enhanced to obtain an enhanced speech signal;

[0124] Segmenting the enhanced speech signal to obtain a short-time frame speech signal;

[0125] Extracting the voiceprint features of the short-time frame speech signal;

[0126] Identifying a feature contribution of the voiceprint feature, and determining a target voiceprint feature among the voiceprint features based on the feature contribution;

[0127] Aggregating the target voiceprint features into a fixed-length feature vector;

[0128] Based on the fixed-length feature vector, a speech signal voiceprint of the short-time frame speech signal is determined.

[0129] Among them, the active part refers to the part of the speech signal that actually contains speech data, usually refers to the speech signal after removing silence, silence or non-speech content. The enhanced speech signal refers to the speech signal that may need to be enhanced during the voiceprint recognition process. The enhancement process may include gain adjustment, filtering, waveform shaping and other steps. The short-time frame speech signal refers to the continuous speech signal that is usually divided into short-time frames in order to better analyze the speech signal. Each frame contains one or more short-time period of speech data. The voiceprint feature refers to the feature extracted from the short-time frame speech signal for characterizing the speech signal, such as MFCC, LPCC and other features. The feature contribution refers to the contribution of each feature to the model performance. The target voiceprint feature extraction can effectively distinguish the voiceprint features of the speaker. The fixed-length feature vector refers to the extracted voiceprint features that are usually combined into a feature vector of a fixed length and containing all important voiceprint information in order to facilitate model training and recognition. The speech signal voiceprint refers to the voiceprint model constructed through the voiceprint feature extraction and voiceprint modeling process.

[0130] Furthermore, the extracting the voiceprint feature of the short-time frame speech signal includes:

[0131] Preprocessing the short-time frame speech signal to obtain a spectrogram;

[0132] Output the power spectrum of the spectrogram using a preset Mel filter;

[0133] Based on the power spectrum of the Mel filter output spectrogram, the Mel frequency cepstrum coefficients of the short-time frame speech signal are calculated using the following formula:

[0134]

[0135] Among them, MFCC(r) represents the Mel frequency cepstral coefficient of the rth short-time frame speech signal, F(v)

[0136] represents the power spectrum of the vth Mel filter output of the short-time frame speech signal, v represents the index of the Mel filter, m represents the number of Mel filters, cos represents the cosine function, π represents pi, r represents the index of the Mel frequency cepstrum coefficient, and log represents the natural logarithm function;

[0137] Based on the Mel-frequency cepstral coefficients, the voiceprint features of the short-time frame speech signal are extracted.

[0138] Among them, the spectrogram refers to a two-dimensional image obtained after performing short-time Fourier transform (STFT) on a short-time frame speech signal, the power spectrum refers to the power distribution of the signal at different frequencies, the Mel-frequency cepstrum coefficients refer to coefficients that reflect the spectral characteristics of the signal, and the Mel filter refers to a filter that simulates the perception characteristics of the human ear to sound.

[0139] The present invention can determine the voice of the target user by calculating the voiceprint matching degree between the voice signal voiceprint and the preset magic mirror voiceprint library, thereby improving the efficiency and reliability of subsequent voice processing. The magic mirror voiceprint library refers to a library constructed to pre-store user voiceprints, and the voiceprint matching degree refers to the similarity between the voice signal voiceprint and the existing voiceprint in the preset magic mirror voiceprint library. In detail, the voiceprint matching degree can be calculated using a similarity function.

[0140] S3. Based on the voiceprint matching degree, determine the target voice signal and target voice user of the smart magic mirror, calculate the user position of the target voice user, and use the trained multilingual intention recognition model to identify the user intention of the target voice signal and the intention order of the user intention. When the intention order does not meet the preset conditions, calculate the intention priority of the user intention, and determine the sequence intention task of the target voice user according to the intention priority.

[0141] It should be explained that the target voice signal refers to the voice signal of the preset registered voiceprint user selected, and the target voice user is the user associated with the target voice signal.

[0142] The user position refers to the specific position information of the user who sends the voice signal. Specifically, the user position can be located by a sound source localization algorithm (such as maximum likelihood estimation, beamforming, multi-signal classification, etc.).

[0143] The present invention utilizes a trained multilingual intention recognition model to recognize the user intention of the target voice signal to meet the needs of users of different languages ​​and provide a better user experience.

[0144] In detail, the step of using the trained multilingual intention recognition model to recognize the user intention of the target voice signal includes:

[0145] Constructing a multilingual model of the target speech signal;

[0146] Annotating the target speech signal based on the multilingual model to obtain an annotated speech signal;

[0147] Extracting language features of the annotated speech signal;

[0148] Based on the language features, using the multilingual intention recognition model to recognize the initial user intention of the annotated voice signal;

[0149] calculating the validity of the initial user intent;

[0150] When the validity meets a preset validity threshold, the initial user intention is used as the user intention.

[0151] Among them, the multilingual model refers to a model that can handle multiple languages, usually including an acoustic model and a language model. The acoustic model is used to identify the sound features in the speech signal, while the language model is used to understand the grammar and semantics of the language. The labeled speech signal refers to the signal after the speech signal content is marked. The language feature refers to the language-related features extracted from the speech signal, such as intonation, speech speed, vocabulary, grammatical structure, etc. The initial user intent refers to the user intent initially predicted by the model. The validity refers to the degree of match between the user intent predicted by the model and the actual user intent. The effective threshold refers to the threshold used to determine whether the user intent predicted by the model is effective. The user intent refers to the final user intent.

[0152] Optionally, the effectiveness of calculating the initial user intention may be calculated in a variety of ways, such as intention probability, confidence, relevance, etc.

[0153] The present invention identifies the intention order of the user's intention, and when the intention order does not meet the preset conditions, calculates the intention priority of the user's intention, and determines, based on the intention priority, that the sequence intention task of the target voice user can adapt to the user scenarios and needs of multi-tasks, thereby improving the user experience. Among them, the intention order refers to the hierarchical structure of the system's recognition of user intentions, the preset conditions refer to a set of rules for determining whether the intention order meets the requirements, the intention priority refers to which intention is the most important among multiple intentions, and the sequence intention task refers to multiple intentions issued continuously by the user, and these intentions are formed into a sequence in order from high to low priority.

[0154] S4. Based on the sequence intention task, construct the mirror interaction instructions of the smart magic mirror. When the mirror interaction instructions require voice broadcast, define the voice interaction instructions of the smart magic mirror based on the user position.

[0155] It should be explained that the mirror interaction instructions refer to interactive prompts or commands displayed to the user on the mirror surface of the smart magic mirror. These instructions may include text, icons, graphical interface instructions, etc. The voice broadcast refers to the process of converting the mirror interaction instructions into voice and playing them to the user through the speaker. The voice interaction instructions refer to the instructions used when the smart magic mirror system interacts with the user through voice. In detail, the voice interaction instructions of the smart magic mirror based on the user's position can determine the volume of the voice broadcast according to the user's position.

[0156] S5. Based on the mirror interaction instructions and the voice interaction instructions, determine the interactive behavior during the execution of the sequence intention task, and according to the interactive behavior, construct an interactive page between the target voice user and the interactive mirror in the smart magic mirror.

[0157] The present invention determines that the interactive behavior in the process of executing the sequence intention task can execute the sequence intention task based on the mirror interaction instruction and the voice interaction instruction, and provides a smooth and intuitive interactive experience. In detail, the interactive behavior refers to the behavioral information of the smart magic mirror that generates the interaction, such as the smart magic mirror opening the beauty page.

[0158] Finally, the present invention can realize the intelligent interaction of the smart magic mirror by constructing an interaction page between the target voice user and the interactive mirror in the smart magic mirror according to the interactive behavior.

[0159] In detail, the step of constructing an interactive page between the target voice user and the interactive mirror surface in the smart magic mirror according to the interactive behavior includes:

[0160] Identifying the target voice user's response interaction information to the interaction behavior;

[0161] Analyzing the interaction intention of the smart magic mirror based on the response interaction information;

[0162] Determining the interaction logic of the smart magic mirror according to the interaction intention;

[0163] According to the interaction logic, an interaction page between the target voice user and the interactive mirror in the smart magic mirror is constructed.

[0164] Among them, the response interaction information refers to the user's reactions and operations to the interactive behavior of the smart magic mirror. This information may include the user's voice commands, touch operations, gestures, expressions, etc. The interaction intention refers to the purpose expressed by the user through the interactive behavior. The interaction logic refers to the response actions and operation procedures taken by the smart magic mirror system according to the user's intention. The interaction page refers to the user interface on the mirror surface of the smart magic mirror, which displays interactive elements and information.

[0165] The present invention constructs a multi-microphone module of the smart magic mirror based on the magic mirror structural parameters to construct an efficient, stable and user-friendly smart magic mirror multi-microphone module, thereby efficiently collecting voice signals; the present invention calculates the far-field voice recognition coefficient of the multi-microphone module, which can be used as a basis for optimizing the microphone layout, thereby improving the voice reception capability of the microphone, thereby improving the user experience; further, the present invention optimizes the multi-microphone module according to the far-field voice recognition coefficient to obtain a target multi-microphone module, which can optimize the multi-microphone module to improve its voice recognition performance under far-field conditions; further, the present invention performs noise suppression on the voice signal to obtain a noise-suppressed voice signal, which can effectively separate noise and voice to obtain a high-quality noise-suppressed voice signal, thereby improving The later speech recognition effect, further, the present invention can provide a basis for the later voiceprint matching by extracting the voice signal voiceprint of the noise suppression voice signal, and then, the present invention uses the trained multilingual intention recognition model to identify the user intention of the target voice signal to meet the needs of users of different languages ​​and provide a better user experience. The present invention identifies the intention order of the user intention. When the intention order does not meet the preset conditions, the intention priority of the user intention is calculated. According to the intention priority, the sequence intention task of the target voice user is determined to be able to adapt to the user scenarios and needs of multi-tasks, thereby improving the user experience. Finally, the present invention can realize the intelligent interaction of the smart magic mirror by constructing the interactive page between the target voice user and the interactive mirror in the smart magic mirror according to the interactive behavior. Therefore, the present invention can improve the interactive effect of the smart magic mirror.

[0166] like Figure 2 , which is a functional module diagram of an intelligent magic mirror interaction system based on speech recognition provided by an embodiment of the present invention.

[0167] The smart magic mirror interaction system 200 based on speech recognition of the present invention can be installed in an electronic device. According to the functions implemented, the smart magic mirror interaction system 200 based on speech recognition can include a multi-microphone module construction module 201, a voiceprint matching module 202, an intention task recognition module 203, an interaction instruction construction module 204 and an interaction page generation module 205. The module of the present invention can also be called a unit, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, which are stored in the memory of the electronic device.

[0168] In this embodiment, the functions of each module / unit are as follows:

[0169] The multi-microphone module construction module 201 is used to identify the magic mirror structure parameters of the smart magic mirror, construct the multi-microphone module of the smart magic mirror based on the magic mirror structure parameters, calculate the far-field speech recognition coefficient of the multi-microphone module, and optimize the multi-microphone module according to the far-field speech recognition coefficient to obtain a target multi-microphone module;

[0170] The voiceprint matching module 202 is used to collect the voice signal of the smart magic mirror through the target multi-microphone module, perform noise suppression on the voice signal to obtain a noise-suppressed voice signal, extract the voice signal voiceprint of the noise-suppressed voice signal, and calculate the voiceprint matching degree between the voice signal voiceprint and the preset magic mirror voiceprint library;

[0171] The intention task identification module 203 is used to determine the target voice signal and the target voice user of the smart magic mirror based on the voiceprint matching degree, calculate the user position of the target voice user, and use the trained multilingual intention recognition model to identify the user intention of the target voice signal, identify the intention order of the user intention, and when the intention order does not meet the preset conditions, calculate the intention priority of the user intention, and determine the sequence intention task of the target voice user according to the intention priority;

[0172] The interactive instruction construction module 204 is used to construct the mirror interaction instruction of the smart magic mirror based on the sequence intention task, and when the mirror interaction instruction needs voice broadcast, define the voice interaction instruction of the smart magic mirror based on the user position;

[0173] The interactive page generation module 205 is used to determine the interactive behavior in the process of executing the sequence intention task based on the mirror interaction instructions and the voice interaction instructions, and to construct an interactive page between the target voice user and the interactive mirror in the smart magic mirror according to the interactive behavior.

[0174] In detail, each module described in the smart magic mirror interaction system 200 based on voice recognition described in the embodiment of the present invention adopts the same technical means as the smart magic mirror interaction method based on voice recognition described in the accompanying drawings when used, and can produce the same technical effects, which will not be repeated here.

[0175] An embodiment of the present invention provides an electronic device for implementing an intelligent magic mirror interaction method based on voice recognition.

[0176] See also Figure 3 As shown, the electronic device may include a processor 30, a memory 31, a communication bus 32 and a communication interface 33, and may also include a computer program stored in the memory 31 and executable on the processor 30, such as a smart magic mirror interaction method program based on voice recognition.

[0177] In some embodiments, the processor may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips. The processor is the control core (Control Unit) of the electronic device, and uses various interfaces and lines to connect the various components of the entire electronic device, and executes or executes the programs or modules stored in the memory (for example, executing the intelligent magic mirror interaction program based on voice recognition, etc.), and calls the data stored in the memory to execute various functions of the electronic device and process data.

[0178] The memory includes at least one type of readable storage medium, and the readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (for example: SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. In some embodiments, the memory can be an internal storage unit of an electronic device, such as a mobile hard disk of the electronic device. In other embodiments, the memory can also be an external storage device of an electronic device, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device. Further, the memory can also include both an internal storage unit of the electronic device and an external storage device. The memory can not only be used to store application software and various types of data installed in the electronic device, such as the code of the smart magic mirror interactive program based on voice recognition, but also can be used to temporarily store data that has been output or is to be output.

[0179] The communication bus may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to realize connection and communication between the memory and at least one processor, etc.

[0180] The communication interface is used for communication between the above-mentioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device and other electronic devices. The user interface may be a display (Display), an input unit (such as a keyboard (Keyboard)), and optionally, the user interface may also be a standard wired interface, a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode, organic light-emitting diode) touch device, etc. Among them, the display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device and to display a visual user interface.

[0181] For example, although not shown, the electronic device may also include a power source (such as a battery) for supplying power to each component. Preferably, the power source may be logically connected to the at least one processor through a power management system, so that the power management system can realize functions such as charging management, discharging management, and power consumption management. The power source may also include any components such as one or more DC or AC power sources, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators. The electronic device may also include a variety of sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.

[0182] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0183] The smart magic mirror interaction program based on voice recognition stored in the memory of the electronic device is a combination of multiple instructions. When running in the processor, it can achieve:

[0184] Identify the magic mirror structure parameters of the smart magic mirror, construct a multi-microphone module of the smart magic mirror based on the magic mirror structure parameters, calculate the far-field speech recognition coefficient of the multi-microphone module, and optimize the multi-microphone module according to the far-field speech recognition coefficient to obtain a target multi-microphone module;

[0185] The target multi-microphone module is used to collect the voice signal of the smart magic mirror, and noise suppression is performed on the voice signal to obtain a noise-suppressed voice signal, and the voice signal voiceprint of the noise-suppressed voice signal is extracted, and the matching degree between the voice signal voiceprint and the voiceprint of a preset magic mirror voiceprint library is calculated;

[0186] Based on the voiceprint matching degree, the target voice signal and the target voice user of the smart magic mirror are determined, the user position of the target voice user is calculated, and the user intention of the target voice signal is identified by using the trained multilingual intention recognition model, and the intention order of the user intention is identified. When the intention order does not meet the preset conditions, the intention priority of the user intention is calculated, and the sequence intention task of the target voice user is determined according to the intention priority;

[0187] Based on the sequence intention task, construct the mirror interaction instruction of the smart magic mirror, and when the mirror interaction instruction requires voice broadcast, define the voice interaction instruction of the smart magic mirror based on the user position;

[0188] Based on the mirror interaction instructions and the voice interaction instructions, the interactive behavior during the execution of the sequence intention task is determined, and according to the interactive behavior, an interactive page between the target voice user and the interactive mirror in the smart magic mirror is constructed.

[0189] Specifically, the specific implementation method of the processor for the above instructions can refer to the description of the relevant steps in the corresponding embodiment of the accompanying drawings, which will not be repeated here.

[0190] Furthermore, if the module / unit integrated in the electronic device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium may include: any entity or system capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).

[0191] The present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor of an electronic device, the computer program can implement:

[0192] Identify the magic mirror structure parameters of the smart magic mirror, construct a multi-microphone module of the smart magic mirror based on the magic mirror structure parameters, calculate the far-field speech recognition coefficient of the multi-microphone module, and optimize the multi-microphone module according to the far-field speech recognition coefficient to obtain a target multi-microphone module;

[0193] The target multi-microphone module is used to collect the voice signal of the smart magic mirror, and noise suppression is performed on the voice signal to obtain a noise-suppressed voice signal, and the voice signal voiceprint of the noise-suppressed voice signal is extracted, and the matching degree between the voice signal voiceprint and the voiceprint of a preset magic mirror voiceprint library is calculated;

[0194] Based on the voiceprint matching degree, the target voice signal and the target voice user of the smart magic mirror are determined, the user position of the target voice user is calculated, and the user intention of the target voice signal is identified by using the trained multilingual intention recognition model, and the intention order of the user intention is identified. When the intention order does not meet the preset conditions, the intention priority of the user intention is calculated, and the sequence intention task of the target voice user is determined according to the intention priority;

[0195] Based on the sequence intention task, construct the mirror interaction instruction of the smart magic mirror, and when the mirror interaction instruction requires voice broadcast, define the voice interaction instruction of the smart magic mirror based on the user position;

[0196] Based on the mirror interaction instructions and the voice interaction instructions, the interactive behavior during the execution of the sequence intention task is determined, and according to the interactive behavior, an interactive page between the target voice user and the interactive mirror in the smart magic mirror is constructed.

[0197] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0198] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0199] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0200] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0201] Therefore, no matter from which point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is limited by the appended claims rather than the above description, so it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any attached figure mark in the claims should not be regarded as limiting the claims involved.

[0202] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0203] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or systems stated in a system claim can also be implemented by one unit or system through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any particular order.

[0204] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. An intelligent magic mirror interaction method based on speech recognition, characterized in that: The method comprises: Identify the magic mirror structure parameters of the smart magic mirror, construct a multi-microphone module of the smart magic mirror based on the magic mirror structure parameters, calculate the far-field speech recognition coefficient of the multi-microphone module, and optimize the multi-microphone module according to the far-field speech recognition coefficient to obtain a target multi-microphone module; The target multi-microphone module is used to collect the voice signal of the smart magic mirror, and noise suppression is performed on the voice signal to obtain a noise-suppressed voice signal, and the voice signal voiceprint of the noise-suppressed voice signal is extracted, and the matching degree between the voice signal voiceprint and the voiceprint of a preset magic mirror voiceprint library is calculated; Based on the voiceprint matching degree, the target voice signal and the target voice user of the smart magic mirror are determined, the user position of the target voice user is calculated, and the user intention of the target voice signal is identified by using the trained multilingual intention recognition model, and the intention order of the user intention is identified. When the intention order does not meet the preset conditions, the intention priority of the user intention is calculated, and the sequence intention task of the target voice user is determined according to the intention priority; Based on the sequence intention task, construct the mirror interaction instruction of the smart magic mirror, and when the mirror interaction instruction requires voice broadcast, define the voice interaction instruction of the smart magic mirror based on the user position; Based on the mirror interaction instructions and the voice interaction instructions, the interactive behavior during the execution of the sequence intention task is determined, and according to the interactive behavior, an interactive page between the target voice user and the interactive mirror in the smart magic mirror is constructed.

2. The intelligent magic mirror interaction method based on speech recognition as claimed in claim 1, characterized in that: The method of constructing a multi-microphone module of the smart magic mirror based on the magic mirror structure parameters includes: Determine the voice collection function index of the smart magic mirror; Based on the voice collection function indicator, define the target microphone of the smart magic mirror; Constructing a multi-microphone layout of the smart magic mirror according to the magic mirror structure parameters and the voice collection function indicators; Calculating the sound receiving efficiency of the target microphone corresponding to the multi-microphone layout; When the sound receiving efficiency meets a preset sound receiving efficiency threshold, a multi-microphone module of the smart magic mirror is constructed.

3. The intelligent magic mirror interaction method based on speech recognition as claimed in claim 2, characterized in that: The calculating the sound receiving efficiency of the target microphone corresponding to the multi-microphone layout includes: analyzing microphone parameters of a target microphone corresponding to the multi-microphone layout; Based on the microphone parameters, calculating the signal reception strength of the target microphone; According to the signal reception strength and the microphone parameters, the sound reception efficiency of the target microphone is calculated using the following formula: Among them, ρ represents the sound receiving efficiency of the target microphone, RSSI a Indicates the signal reception strength of the ath microphone, RSSI max,a Indicates the maximum signal receiving strength of the ath microphone, S a Indicates the directional factor of the ath microphone in the microphone parameters, DF a represents the attenuation transmission factor of the a-th microphone in the microphone parameters, N represents the number of target microphones, and a represents the a-th microphone.

4. The intelligent magic mirror interaction method based on speech recognition as claimed in claim 3, characterized in that: The calculating the far-field speech recognition coefficient of the multi-microphone module includes: defining a far-field condition of a sound source of the multi-microphone module; Based on the far-field condition of the sound source, construct a sound source model and an environment model of the multi-microphone module; Establishing an acoustic model of the multi-microphone module through the sound source model and the environment model; Based on the acoustic model, analyzing the sound source distribution of the multi-microphone module corresponding to the sound source scene; Calculating the sound pressure level of the target microphone corresponding to the multi-microphone module according to the sound source distribution; Based on the sound pressure level, analyzing the speech recognition rate of the target microphone corresponding to the multi-microphone module under the far-field condition of the sound source; The far-field speech recognition coefficient of the multi-microphone module is calculated according to the speech recognition rate.

5. The intelligent magic mirror interaction method based on speech recognition as claimed in claim 4, characterized in that: The step of performing noise suppression on the speech signal to obtain a noise suppressed speech signal comprises: Performing time domain processing on the speech signal to obtain a time domain processed signal; Performing frequency domain processing on the time domain processed signal to obtain a frequency domain processed signal; Constructing a sparse representation dictionary of the frequency domain processed signal; Marking the sparse representation of the frequency domain processed signal by the sparse representation dictionary; Determining a signal noise of the frequency-domain processed signal according to the sparse representation; The signal noise is suppressed to obtain a noise-suppressed speech signal of the frequency domain processed signal.

6. The intelligent magic mirror interaction method based on speech recognition as claimed in claim 5, characterized in that: The extracting the voice signal voiceprint of the noise suppressed voice signal includes: marking an active portion of the noise suppressed speech signal; According to the active part, the noise suppressed speech signal is enhanced to obtain an enhanced speech signal; Segmenting the enhanced speech signal to obtain a short-time frame speech signal; Extracting the voiceprint features of the short-time frame speech signal; Identifying a feature contribution of the voiceprint feature, and determining a target voiceprint feature among the voiceprint features based on the feature contribution; Aggregating the target voiceprint features into a fixed-length feature vector; Based on the fixed-length feature vector, a speech signal voiceprint of the short-time frame speech signal is determined.

7. The intelligent magic mirror interaction method based on speech recognition as claimed in claim 6, characterized in that: The extracting the voiceprint feature of the short-time frame speech signal includes: Preprocessing the short-time frame speech signal to obtain a spectrogram; Output the power spectrum of the spectrogram using a preset Mel filter; Based on the power spectrum of the Mel filter output spectrogram, the Mel frequency cepstrum coefficients of the short-time frame speech signal are calculated using the following formula: Among them, MFCCr represents the rth Mel frequency cepstrum coefficient of the short-time frame speech signal, Fv represents the power spectrum output by the vth Mel filter of the short-time frame speech signal, v represents the index of the Mel filter, m represents the number of Mel filters, cos represents the cosine function, π represents pi, r represents the index of the Mel frequency cepstrum coefficient, and log represents the natural logarithm function; Based on the Mel-frequency cepstral coefficients, the voiceprint features of the short-time frame speech signal are extracted.

8. The intelligent magic mirror interaction method based on speech recognition as claimed in claim 7, characterized in that: The step of using the trained multilingual intention recognition model to recognize the user intention of the target voice signal includes: Constructing a multilingual model of the target speech signal; Annotating the target speech signal based on the multilingual model to obtain an annotated speech signal; Extracting language features of the annotated speech signal; Based on the language features, using the multilingual intention recognition model to recognize the initial user intention of the annotated voice signal; calculating the validity of the initial user intent; When the validity meets a preset validity threshold, the initial user intention is used as the user intention.

9. The intelligent magic mirror interaction method based on speech recognition as claimed in claim 8, characterized in that: The step of constructing an interactive page between the target voice user and the interactive mirror surface in the smart magic mirror according to the interactive behavior includes: Identifying the target voice user's response interaction information to the interaction behavior; Analyzing the interaction intention of the smart magic mirror based on the response interaction information; Determining the interaction logic of the smart magic mirror according to the interaction intention; According to the interaction logic, an interaction page between the target voice user and the interactive mirror in the smart magic mirror is constructed.

10. An intelligent magic mirror interactive system based on speech recognition, characterized in that: The system is used to execute the intelligent magic mirror interaction method based on speech recognition as described in any one of claims 1 to 9, and the system comprises: A multi-microphone module construction module is used to identify the magic mirror structure parameters of the smart magic mirror, construct the multi-microphone module of the smart magic mirror based on the magic mirror structure parameters, calculate the far-field speech recognition coefficient of the multi-microphone module, and optimize the multi-microphone module according to the far-field speech recognition coefficient to obtain a target multi-microphone module; A voiceprint matching module, used to collect the voice signal of the smart magic mirror through the target multi-microphone module, perform noise suppression on the voice signal to obtain a noise-suppressed voice signal, extract the voice signal voiceprint of the noise-suppressed voice signal, and calculate the voiceprint matching degree between the voice signal voiceprint and a preset magic mirror voiceprint library; An intention task recognition module is used to determine the target voice signal and the target voice user of the smart magic mirror based on the voiceprint matching degree, calculate the user position of the target voice user, and use the trained multilingual intention recognition model to recognize the user intention of the target voice signal, recognize the intention order of the user intention, and when the intention order does not meet the preset conditions, calculate the intention priority of the user intention, and determine the sequence intention task of the target voice user according to the intention priority; An interactive instruction construction module, used to construct a mirror interaction instruction of the smart magic mirror based on the sequence intention task, and when the mirror interaction instruction requires voice broadcast, define the voice interaction instruction of the smart magic mirror based on the user position; An interactive page generation module is used to determine the interactive behavior in the process of executing the sequence intention task based on the mirror interaction instructions and the voice interaction instructions, and to construct an interactive page between the target voice user and the interactive mirror in the smart magic mirror according to the interactive behavior.

Images