Intelligent interaction system and method based on far-field voice module

Abstract

The invention discloses an intelligent interaction system and method based on a far-field voice module, and relates to the technical field of voice recognizing.The system forms different processing feedback mechanisms according to different network states by capturing voice input signals of a user; recognizing and optimizing the preprocessed voice input signal in combination with an acoustic model and a language model to form a final processed text; starting a self-protection mechanism, and forming first response instruction data of a next period based on big data training; speculating second response instruction data forming a next period by using a period mode set by the system; and setting a storage space, and storing coincident response instruction data in the first response instruction data and the second response instruction data, thereby forming voice response prediction in an offline state, not only being capable of ensuring that purposeful voice interaction is realized in an offline environment and voice control is realized in a private environment, but also being capable of reducing storage for configuring offline voice, and improving the voice response prediction efficiency. And the portability of voice interaction is improved.

Description

Technical Field

[0001] The present invention relates to the field of speech recognition technology, and in particular to an intelligent interaction system and method based on a far-field speech module. Background Art

[0002] Far-field voice refers to the technology of voice collection and processing at a long distance (usually more than 1 meter). With the continuous development of artificial intelligence, Internet of Things, edge computing and other technologies, the high scalability of far-field voice technology enables it to be combined with a variety of advanced technologies to support a wider range of application scenarios and more complex functions. In the fields of smart factories, medical care, and security, far-field voice interaction can be used for It brings users a smarter and more convenient interactive experience.

[0003] However, currently, although the access of various Internet devices has greatly enriched the functions and playability of voice interaction, the privacy and network problems that come with it are also endless. For example, in an unstable or disconnected network environment, various voice interaction devices lose their basic functions or have huge delays; or from the perspective of security, smart factories, and indoor home scenarios, excessive access to Internet devices leads to a large amount of data being transmitted on the Internet, and privacy cannot be guaranteed. How to freely configure offline voice control in a networked environment, achieve purposeful voice interaction in a disconnected environment, and achieve voice control in a private environment is one of the current problems. Summary of the invention

[0004] The purpose of the present invention is to provide an intelligent interaction system and method based on a far-field voice module to solve the problems raised in the prior art.

[0005] To achieve the above object, the present invention provides the following technical solution: an intelligent interaction method based on a far-field voice module, the method comprising the following steps: Capture the user's voice input signal, perform preliminary processing on the voice input signal to form a preprocessed voice input signal, detect the network status, and form different processing feedback mechanisms according to different network status; Receive the preprocessed speech input signal, combine the acoustic model and the language model, recognize and optimize the preprocessed speech input signal, and form the final processed text; Call the final processing text, start the self-protection mechanism, use the periodic method set by the system, call the historical response instruction data in the big database, and form the first response instruction data of the next cycle based on big data training; Obtain current time data, and use the periodic method set by the system to infer the second response instruction data of the next period; Setting up a storage space to store the coincident response instruction data in the first response instruction data and the second response instruction data, waking up a local voice recognition engine, establishing a local control logic, and forming an offline voice control; Intelligent interactions are formed for processing texts in different states.

[0006] According to the above technical solution, the preliminary processing of the voice input signal includes: De-noise the received voice input signal, remove background noise, and normalize the audio signal to the same volume level to form a pre-processed voice input signal; The detecting of the network status includes: testing the network status, including online network and offline network.

[0007] According to the above technical solution, the recognition and optimization of the preprocessed speech input signal includes: Modeling based on acoustic models forms the relationship between speech features and phonemes; The probability of word sequences is predicted based on language model analysis to form the final processed text.

[0008] According to the above technical solution, the first response instruction data for the next cycle formed based on big data training includes: Acquire the response instruction data of adjacent cycles, and in the historical data, take the response instruction data of every two adjacent cycles as a group of analysis data; Select any response instruction data of the current cycle, record any response instruction data as data A, search for data A in the analysis data of historical data, select all the next cycle response instruction data of data A in the historical data, and mark them once; Select all the response instruction data of the current cycle as a response data group, search the response data group in the analysis data of the historical data, select all the response instruction data of the next cycle that exist in the response data group in the historical data, and mark them once; All labeled data are re-labeled, and the labels include: For any set of historical data analysis data, calculate the occupied data ratio of data A or response data group: or , where e represents the occupied data ratio, Represents the total number of response command data in the current cycle; The total number of response command data representing the response data group; The occupied data proportion e is marked on the corresponding selected next cycle response instruction data, and all next cycle response instruction data are adjusted. The same next cycle response instruction data are set to the same group, and the sum of the e marked in each group is calculated as the output value of the same group of next cycle response instruction data. The system sets a threshold, extracts the group whose output value exceeds the threshold, and selects the next cycle response instruction data corresponding to the extracted group as the first response instruction data of the next cycle.

[0009] According to the above technical solution, the second response instruction data of the next cycle is inferred to include: Based on the clock module, the time information data is obtained. Based on the response instruction data of each time information data under the historical data, the time period of the current time information data is selected, and each day is used as a cycle range to obtain the response instruction data under the same time period. The same response instruction data is classified. For any category, the total number of response instruction data under the current category is recorded as the category output. If there is , then judge The corresponding response instruction data is the second response instruction data inferred to form the next cycle, wherein, Represents the category output under each i-th category; represents the total number of all response instruction data in the same time period; h represents the number of categories formed in the same time period.

[0010] According to the above technical solution, waking up the local voice recognition engine, establishing local control logic, and forming offline voice control include: An open source speech recognition engine is embedded in the system, and the control logic is called according to the recognition results. The corresponding hardware control code is called according to the control logic to realize offline speech control.

[0011] An intelligent interactive system based on a far-field voice module, the intelligent interactive system comprising: The voice collection and preprocessing module is responsible for capturing the user's voice input signal, performing preliminary processing on the voice input signal, forming a preprocessed voice input signal, detecting the network status, and forming different processing feedback mechanisms according to different network statuses; The post-processing and feedback module is used to receive the pre-processed speech input signal, and recognize and optimize the pre-processed speech input signal in combination with the acoustic model and the language model to form the final processed text; The feature expansion module is used to call the final processing text, start the self-protection mechanism, use the periodic method set by the system, call the historical response instruction data in the big database, and form the first response instruction data of the next cycle based on big data training; A time estimation module is used to obtain current time data and estimate the second response instruction data of the next cycle using a periodic method set by the system; The offline voice control module is used to set up a storage space, store the coincident response instruction data in the first response instruction data and the second response instruction data, wake up the local voice recognition engine, establish the local control logic, and form offline voice control; The intelligent interaction module is used to form intelligent interaction based on processed text.

[0012] According to the above technical solution, the voice collection and preprocessing module includes: Automatic network status recognition, including online and offline networks. When the network is normal, work is performed with the online network. When the network is abnormal, close the online network entrance and work with the offline network. Manual network status identification allows administrators to set corresponding passwords and manually adjust to offline networks.

[0013] According to the above technical solution, the time estimation module is also connected to a clock module, and the clock module provides time information data; The clock module further comprises a clock database, in which time information data corresponding to any response instruction in the historical data is stored.

[0014] Compared with the prior art, the beneficial effects of the present invention are: in the era of rapid development of voice interaction, the present application proposes to protect privacy as the main focus, while being able to solve the problem of offline voice control caused by network disconnection or network interference, greatly enriching the functions and playability of voice interaction, and can be applied to smart factories, private homes, and smart security and other fields. Through continuous speculation in a networked environment, a prediction of voice response commands is formed in each cycle, thereby forming a voice response prediction in an offline state, which can not only ensure the realization of purposeful voice interaction in a network disconnected environment and voice control in a private environment, but also reduce the storage of configured offline voice and improve the portability of voice interaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 The figure is a schematic diagram of module connections of an intelligent interactive system based on a far-field voice module of the present invention. DETAILED DESCRIPTION

[0016] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0017] Embodiment: The present invention provides an intelligent interaction method based on a far-field voice module, the method comprising the following steps: Capture the user's voice input signal, perform preliminary processing on the voice input signal to form a preprocessed voice input signal, detect the network status, and form different processing feedback mechanisms according to different network status; Receive the preprocessed speech input signal, combine the acoustic model and the language model, recognize and optimize the preprocessed speech input signal, and form the final processed text; Call the final processing text, start the self-protection mechanism, use the periodic method set by the system, call the historical response instruction data in the big database, and form the first response instruction data of the next cycle based on big data training; Obtain current time data, and use the periodic method set by the system to infer the second response instruction data of the next period; Setting up a storage space to store the coincident response instruction data in the first response instruction data and the second response instruction data, waking up a local voice recognition engine, establishing a local control logic, and forming an offline voice control; Intelligent interactions are formed for processing texts in different states.

[0018] The preliminary processing of the voice input signal comprises: Use the corresponding speech detection equipment to receive the speech signal, sample and quantize the received speech input signal, and implement the first step of processing by framing (for example, dividing the signal into short-time frames, usually 20-40ms per frame) and windowing (reducing frame edge effects); then use the minimum statistic or recursive averaging method to estimate the noise, perform noise reduction, and remove background noise. In the noise reduction process, a variety of algorithms can be used, such as spectral subtraction or Wiener filtering. If the signal data is in a non-stationary state, adaptive filtering or wavelet transform can be used.

[0019] At the same time, the audio signals are normalized to the same volume level to form a preprocessed speech input signal; The detecting of the network status includes: testing the network status, including online network and offline network.

[0020] The identifying and optimizing the preprocessed speech input signal comprises: Modeling based on acoustic models forms the relationship between speech features and phonemes; The probability of word sequences is predicted based on language model analysis to form the final processed text.

[0021] The first response instruction data for the next cycle formed based on the big data training includes: Acquire the response instruction data of adjacent cycles, and in the historical data, take the response instruction data of every two adjacent cycles as a group of analysis data; Select any response instruction data of the current cycle, record any response instruction data as data A, search for data A in the analysis data of historical data, select all the next cycle response instruction data of data A in the historical data, and mark them once; Select all the response instruction data of the current cycle as a response data group, search the response data group in the analysis data of the historical data, select all the response instruction data of the next cycle that exist in the response data group in the historical data, and mark them once; All labeled data are re-labeled, and the labels include: For any set of historical data analysis data, calculate the occupied data ratio of data A or response data group: or , where e represents the occupied data ratio, Represents the total number of response command data in the current cycle; The total number of response command data representing the response data group; For example, take any response data instruction "turn on the light", then in the analysis data, all analysis data with the "turn on the light" instruction in the previous cycle, all its response instructions in the next cycle will be marked; the response data group is marked in a team manner. For example, if there are instructions such as "turn on the light" and "turn on the air conditioner" in the previous cycle, the instructions will be packaged and analyzed in teams. If there is an identical group in the historical data, all corresponding response instructions in the next cycle will be marked.

[0022] The occupied data proportion e is marked on the corresponding selected next cycle response instruction data, and all next cycle response instruction data are adjusted. The same next cycle response instruction data are set to the same group, and the sum of the e marked in each group is calculated as the output value of the same group of next cycle response instruction data. The system sets a threshold, extracts the group whose output value exceeds the threshold, and selects the next cycle response instruction data corresponding to the extracted group as the first response instruction data of the next cycle.

[0023] The second response instruction data of the next cycle is formed by inferring: Based on the clock module, the time information data is obtained. Based on the response instruction data of each time information data under the historical data, the time period of the current time information data is selected, and each day is used as a cycle range to obtain the response instruction data under the same time period. The same response instruction data is classified. For any category, the total number of response instruction data under the current category is recorded as the category output. If there is , then judge The corresponding response instruction data is the second response instruction data inferred to form the next cycle, wherein, Represents the category output under each i-th category; represents the total number of all response instruction data in the same time period; h represents the number of categories formed in the same time period.

[0024] The waking up the local voice recognition engine, establishing the local control logic, and forming the offline voice control comprises: An open source speech recognition engine is embedded in the system, and the control logic is called according to the recognition results. The corresponding hardware control code is called according to the control logic to realize offline speech control.

[0025] In the second embodiment, Figure 1 As shown, it also includes an intelligent interaction system based on a far-field voice module, and the intelligent interaction system includes: The voice collection and preprocessing module is responsible for capturing the user's voice input signal, performing preliminary processing on the voice input signal, forming a preprocessed voice input signal, detecting the network status, and forming different processing feedback mechanisms according to different network statuses; The post-processing and feedback module is used to receive the pre-processed speech input signal, and recognize and optimize the pre-processed speech input signal in combination with the acoustic model and the language model to form the final processed text; The feature expansion module is used to call the final processing text, start the self-protection mechanism, use the periodic method set by the system, call the historical response instruction data in the big database, and form the first response instruction data of the next cycle based on big data training; A time estimation module is used to obtain current time data and estimate the second response instruction data of the next cycle using a periodic method set by the system; The offline voice control module is used to set up a storage space, store the coincident response instruction data in the first response instruction data and the second response instruction data, wake up the local voice recognition engine, establish the local control logic, and form offline voice control; The intelligent interaction module is used to form intelligent interaction based on processed text.

[0026] The voice collection and preprocessing module includes: Automatic network status recognition, including online and offline networks. When the network is normal, work is performed with the online network. When the network is abnormal, close the online network entrance and work with the offline network. Manual network status identification allows administrators to set corresponding passwords and manually adjust to offline networks.

[0027] The time estimation module is also connected to a clock module, and the clock module provides time information data; The clock module further comprises a clock database, in which time information data corresponding to any response instruction in the historical data is stored.

[0028] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. An intelligent interaction method based on a far-field voice module, characterized in that: The method comprises the following steps: Capture the user's voice input signal, perform preliminary processing on the voice input signal to form a preprocessed voice input signal, detect the network status, and form different processing feedback mechanisms according to different network status; Receive the preprocessed speech input signal, combine the acoustic model and the language model, recognize and optimize the preprocessed speech input signal, and form the final processed text; Call the final processing text, start the self-protection mechanism, use the periodic method set by the system, call the historical response instruction data in the big database, and form the first response instruction data of the next cycle based on big data training; Obtain current time data, and use the periodic method set by the system to infer the second response instruction data of the next period; Setting up a storage space to store the coincident response instruction data in the first response instruction data and the second response instruction data, waking up a local voice recognition engine, establishing a local control logic, and forming an offline voice control; Intelligent interactions are formed for processing texts in different states.

2. The intelligent interaction method based on the far-field voice module according to claim 1, characterized in that: The preliminary processing of the voice input signal comprises: De-noise the received voice input signal, remove background noise, and normalize the audio signal to the same volume level to form a pre-processed voice input signal; The detecting of the network status includes: testing the network status, including online network and offline network.

3. The intelligent interaction method based on the far-field voice module according to claim 1, characterized in that: The identifying and optimizing the preprocessed speech input signal comprises: Modeling based on acoustic models forms the relationship between speech features and phonemes; The probability of word sequences is predicted based on language model analysis to form the final processed text.

4. The intelligent interaction method based on the far-field voice module according to claim 1, characterized in that: The first response instruction data for the next cycle formed based on the big data training includes: Acquire the response instruction data of adjacent cycles, and in the historical data, take the response instruction data of every two adjacent cycles as a group of analysis data; Select any response instruction data of the current cycle, record any response instruction data as data A, search for data A in the analysis data of historical data, select all the next cycle response instruction data of data A in the historical data, and mark them once; Select all the response instruction data of the current cycle as a response data group, search the response data group in the analysis data of the historical data, select all the response instruction data of the next cycle that exist in the response data group in the historical data, and mark them once; All labeled data are re-labeled, and the labels include: For any set of historical data analysis data, calculate the occupied data ratio of data A or response data group: or , where e represents the occupied data ratio, Represents the total number of response command data in the current cycle; The total number of response command data representing the response data group; The occupied data proportion e is marked on the corresponding selected next cycle response instruction data, and all next cycle response instruction data are adjusted. The same next cycle response instruction data are set to the same group, and the sum of the e marked in each group is calculated as the output value of the same group of next cycle response instruction data. The system sets a threshold, extracts the group whose output value exceeds the threshold, and selects the next cycle response instruction data corresponding to the extracted group as the first response instruction data of the next cycle.

5. The intelligent interaction method based on the far-field voice module according to claim 1, characterized in that: The second response instruction data of the next cycle is formed by inferring: Based on the clock module, the time information data is obtained. Based on the response instruction data of each time information data under the historical data, the time period of the current time information data is selected, and each day is used as a cycle range to obtain the response instruction data under the same time period. The same response instruction data is classified. For any category, the total number of response instruction data under the current category is recorded as the category output. If there is , then judge The corresponding response instruction data is the second response instruction data inferred to form the next cycle, wherein, Represents the category output under each i-th category; represents the total number of all response instruction data in the same time period; h represents the number of categories formed in the same time period.

6. The intelligent interaction method based on the far-field voice module according to claim 1, characterized in that: The waking up the local voice recognition engine, establishing the local control logic, and forming the offline voice control comprises: An open source speech recognition engine is embedded in the system, and the control logic is called according to the recognition results. The corresponding hardware control code is called according to the control logic to realize offline voice control.

7. Intelligent interactive system based on far-field voice module, characterized by: The intelligent interaction system comprises: The voice collection and preprocessing module is responsible for capturing the user's voice input signal, performing preliminary processing on the voice input signal, forming a preprocessed voice input signal, detecting the network status, and forming different processing feedback mechanisms according to different network statuses; The post-processing and feedback module is used to receive the pre-processed speech input signal, and recognize and optimize the pre-processed speech input signal in combination with the acoustic model and the language model to form the final processed text; The feature expansion module is used to call the final processing text, start the self-protection mechanism, use the periodic method set by the system, call the historical response instruction data in the big database, and form the first response instruction data of the next cycle based on big data training; A time estimation module is used to obtain current time data and estimate the second response instruction data of the next cycle using a periodic method set by the system; The offline voice control module is used to set up a storage space, store the coincident response instruction data in the first response instruction data and the second response instruction data, wake up the local voice recognition engine, establish the local control logic, and form offline voice control; The intelligent interaction module is used to form intelligent interaction based on processed text.

8. The intelligent interactive system based on the far-field voice module according to claim 7, characterized in that: The voice collection and preprocessing module includes: Automatic network status recognition, including online and offline networks. When the network is normal, work is performed with the online network. When the network is abnormal, close the online network entrance and work with the offline network. Manual network status identification allows administrators to set corresponding passwords and manually adjust to offline networks.

9. The intelligent interactive system based on the far-field voice module according to claim 7, characterized in that: The time estimation module is also connected to a clock module, and the clock module provides time information data; The clock module further comprises a clock database, in which time information data corresponding to any response instruction in the historical data is stored.

Images