A method, system, apparatus and storage medium for voice interaction

Abstract

The application discloses a voice interaction method, system, device and storage medium. The method is based on a voice interaction toolkit of a smart terminal to perform voice interaction. The method acquires input voice information through the smart terminal. Then, a voice recognition SDK is loaded to recognize the voice information, thereby generating text information corresponding to the voice information. Subsequently, the voice generation SDK generates voice interaction information according to the text information. Then, the voice recognition SDK is cleared, and the voice playback SDK is loaded to play the voice interaction information. The method integrates the voice recognition SDK, the voice generation SDK and the voice playback SDK into the voice interaction toolkit, realizes voice interaction through the voice recognition SDK, the voice generation SDK and the voice playback SDK, limits the calling sequence of the SDKs, effectively avoids the problem that the work of the SDKs will interfere with other SDKs, and thereby improves the recognition rate and accuracy of voice recognition.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a voice interaction method, system, device and storage medium. BACKGROUND

[0002] Human-computer interaction refers to an information exchange process between a person and a computer using a certain dialogue language and in a certain interactive manner to complete a determined task. With the rapid development of Internet technology and Internet of Things technology, intelligent voice interaction has become one of the mainstream human-computer interaction methods.

[0003] In the process of voice interaction with the intelligent terminal, a voice recognition SDK, a voice generation SDK and a voice playback SDK are usually used. However, the intelligent terminal does not pay attention to the calling order between the SDKs when performing voice interaction, so that the SDKs interfere with each other when working, resulting in low recognition rate and accuracy of voice recognition. SUMMARY

[0004] The present application aims to at least partly solve one of the technical problems in the related art.

[0005] To this end, one object of the present application is to provide a voice interaction method that can avoid interference between SDKs when working, thereby improving the recognition rate and accuracy of voice recognition.

[0006] In order to achieve the above technical purpose, the technical solution adopted by the embodiments of the present application comprises:

[0007] A voice interaction method based on a voice interaction tool kit of an intelligent terminal, the voice interaction tool kit comprising a voice recognition SDK, a voice generation SDK and a voice playback SDK, the method comprising the following steps:

[0008] obtaining input voice information through the intelligent terminal;

[0009] loading the voice recognition SDK to recognize the voice information and generate text information corresponding to the voice information;

[0010] generating voice interaction information according to the text information through the voice generation SDK;

[0011] clearing the voice recognition SDK and loading the voice playback SDK to play the voice interaction information.

[0012] In addition, the voice interaction method according to the above embodiments of the present application can also have the following additional technical features:

[0013] Further, in an embodiment of the present application, the step of loading the speech recognition SDK to recognize the voice information and generate text information corresponding to the voice information specifically includes:

[0014] Obtaining a voiceprint feature of the voice information;

[0015] Comparing the voiceprint feature with a voiceprint database corresponding to the user to determine whether the voice information is voice information generated by the user;

[0016] When the voice information is voice information generated by the user, recognizing the voice information through the speech recognition SDK to generate text information corresponding to the voice information.

[0017] Further, in an embodiment of the present application, the step of comparing the voiceprint feature with the voiceprint database corresponding to the user to determine whether the voice information is voice information generated by the user specifically includes:

[0018] Extracting a first voiceprint feature of the voice information;

[0019] Extracting a second voiceprint feature of the user from the voiceprint database;

[0020] Determining a similarity between the first voiceprint feature and the second voiceprint feature;

[0021] Determining whether the similarity is greater than a first preset threshold;

[0022] When the similarity is greater than the first preset threshold, the voice information is voice information generated by the user;

[0023] When the similarity is less than or equal to the first preset threshold, the voice information is not voice information generated by the user.

[0024] Further, in an embodiment of the present application, the step of loading the speech recognition SDK to recognize the voice information and generate text information corresponding to the voice information specifically includes:

[0025] Obtaining a volume intensity of the voice information;

[0026] When the volume intensity is greater than a second preset threshold, recognizing the voice information through the speech recognition SDK to generate text information corresponding to the voice information.

[0027] Further, in an embodiment of the present application, the step of generating voice interaction information according to the text information through the speech generation SDK specifically includes:

[0028] extract text feature information of the text information;

[0029] input the text feature information into a voice interaction model to generate voice interaction information.

[0030] Further, in an embodiment of the present application, the following steps are further included:

[0031] input the voice information into a noise detection model to obtain a noise detection result output by the noise detection model; the noise detection result is used to represent whether the voice information contains noise data;

[0032] determine a confidence degree of the voice interaction information according to the noise detection result.

[0033] Further, in an embodiment of the present application, the following steps are further included:

[0034] recognize the voice information through the voice recognition SDK to generate an operation instruction corresponding to the voice information;

[0035] control the intelligent terminal according to the operation instruction.

[0036] On the other hand, an embodiment of the present application provides a voice interaction system, which performs voice interaction based on a voice interaction toolkit of an intelligent terminal, the voice interaction toolkit including a voice recognition SDK, a voice generation SDK and a voice playback SDK, and the system includes the following modules:

[0037] A first module is configured to acquire input voice information through an intelligent terminal;

[0038] A second module is configured to load the voice recognition SDK to recognize the voice information and generate text information corresponding to the voice information;

[0039] A third module is configured to generate voice interaction information according to the text information through the voice generation SDK;

[0040] A fourth module is configured to clear the voice recognition SDK and load the voice playback SDK to play the voice interaction information.

[0041] On the other hand, an embodiment of the present application provides a voice interaction device, which includes:

[0042] at least one processor;

[0043] at least one memory configured to store at least one program;

[0044] When the at least one program is executed by the at least one processor, the at least one processor implements the above-mentioned voice interaction method.

[0045] A computer-readable storage medium, wherein a processor-executable instruction is stored, the processor-executable instruction is used to implement the above-mentioned voice interaction method when executed by a processor.

[0046] The advantages and beneficial effects of the present application will be partially given in the following description, partially will become apparent from the following description, or will be understood by the practice of the present application:

[0047] The voice interaction method disclosed in the embodiments of the present application is applied to intelligent terminals including but not limited to smart phones, palm computers, tablet computers, smart TVs (Smart TV) and the like. The method performs voice interaction based on a voice interaction toolkit of the intelligent terminal. The method acquires input voice information through the intelligent terminal. Then, the voice recognition SDK is loaded to recognize the voice information, thereby generating text information corresponding to the voice information. Subsequently, the voice generation SDK is used to generate voice interaction information according to the text information. Then, the voice recognition SDK is cleared, and the voice playback SDK is loaded to play the voice interaction information. The voice recognition SDK, the voice generation SDK and the voice playback SDK are integrated into the voice interaction toolkit. The voice interaction is realized through the voice recognition SDK, the voice generation SDK and the voice playback SDK. The calling order of the SDKs is limited. The problem that the SDKs interfere with other SDKs during work is effectively avoided. Therefore, the recognition rate and the accuracy of voice recognition are improved. BRIEF DESCRIPTION OF DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following introduces the drawings of the related technical solutions in the embodiments of the present application or the prior art. It should be understood that the drawings in the following introduction are only for the convenience of clearly describing part of the embodiments of the technical solutions of the present application. For those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0049] Figure 1 The implementation environment schematic diagram of the voice interaction method provided in the embodiments of the present application;

[0050] Figure 2 The flowchart schematic diagram of the voice interaction method provided in the embodiments of the present application;

[0051] Figure 3 The specific flowchart schematic diagram of step 120 in the voice interaction method provided in the embodiments of the present application; Figure 2 ​

[0052] Figure 4 Provided in the embodiments of this application Figure 3 A specific flowchart of step 122;

[0053] Figure 5 Provided in the embodiments of this application Figure 2 Another specific flowchart for step 120;

[0054] Figure 6 Provided in the embodiments of this application Figure 2 A specific flowchart of step 130 in the middle section;

[0055] Figure 7 This is a flowchart illustrating the noise detection process in a voice interaction method provided in this application embodiment;

[0056] Figure 8 This is a schematic diagram of the terminal control process in a voice interaction method provided in this application embodiment;

[0057] Figure 9 This is a schematic diagram of the structure of a voice interaction system provided in the embodiments of this application;

[0058] Figure 10 This is a schematic diagram of the structure of a voice interaction device provided in an embodiment of this application. Detailed Implementation

[0059] The present application will be further described below with reference to the accompanying drawings and specific embodiments. The described embodiments should not be considered as limitations on the present application, and all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of the present application.

[0060] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0061] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0062] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0063] 1) Artificial Intelligence (AI) is the theory, method, technology and application system of using digital computer or machine controlled by digital computer to simulate, extend and expand human intelligence, perceive environment, acquire knowledge and use knowledge to obtain the best results. Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, both hardware and software level technology. Among them, the basic technology of artificial intelligence generally includes technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc.; the software technology of artificial intelligence mainly includes computer vision technology, speech processing technology, natural language processing technology, machine learning / deep learning, automatic driving and intelligent transportation, etc.

[0064] 2) Nature Language processing (NLP) is an important direction in the field of computer science and artificial intelligence. It studies various theories and methods that can realize effective communication between people and computers using natural language. Natural language processing is a science that integrates linguistics, computer science and mathematics. This field involves natural languages used in daily life, so it is closely related to the study of linguistics. Natural language processing technology usually includes text processing, semantic understanding, machine translation, robot question and answer, knowledge graph and other technologies.

[0065] 3) Software Development Kit (SDK) is a collection of development tools used by software engineers to create applications for a particular software package, software framework, hardware platform, operating system, etc. Generally speaking, SDK is the SDK used to develop applications on the Windows platform. It promotes the creation of applications through compilers, debuggers, software frameworks, etc. It can simply provide some files of application program interface API for a certain program design language, but it can also include complex hardware that can communicate with some embedded systems. General tools include utilities for debugging and other purposes. SDK often includes sample code, supporting technical notes or other supporting documents to clarify doubts for basic reference materials.

[0066] 4) Machine Learning (ML) is a multi-disciplinary subject, involving probability theory, statistics, approximation theory, convex analysis, algorithmic complexity theory, etc., which is dedicated to studying how computers simulate or implement human learning behavior to acquire new knowledge or skills, reorganize existing knowledge structure to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental approach to making computers intelligent. Its applications are widespread in various fields of artificial intelligence. Machine learning (deep learning) usually includes artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and rule-based learning.

[0067] 5) Automatic Speech Recognition (ASR): This technology enables computers to "transcribe" continuous speech spoken by different people, commonly known as "speech transcription machines". It is a technology that converts human speech information into text information.

[0068] In the process of voice interaction of intelligent terminals, voice recognition SDK, voice generation SDK and voice playback SDK are needed. Each SDK has a corresponding initialization configuration, and the calling and response between them affect each other. For example, voice playback must be stopped during voice recognition, otherwise the sound played will be recognized by the voice recognition SDK. In related technologies, only the functions of voice recognition and voice playback are concerned, and the initialization and mutual influence between SDKs are not concerned.

[0069] In order to solve the problem that the calling order between SDKs is not concerned when intelligent terminals perform voice interaction, resulting in low recognition rate and accuracy of voice recognition, the method for voice interaction provided by the embodiments of the present application is applied to intelligent terminals including but not limited to smartphones, palmtop computers, tablet computers, smart TVs (Smart TV), etc. The method performs voice interaction based on a voice interaction toolkit of the intelligent terminal. The method acquires input voice information through the intelligent terminal; then, the voice recognition SDK is loaded to recognize the voice information, thereby generating text information corresponding to the voice information; subsequently, the voice generation SDK is used to generate voice interaction information according to the text information; then, the voice recognition SDK is cleared, and the voice playback SDK is loaded to play the voice interaction information. The method integrates the voice recognition SDK, the voice generation SDK and the voice playback SDK into the voice interaction toolkit, realizes voice interaction through the voice recognition SDK, the voice generation SDK and the voice playback SDK, limits the calling order of the SDKs, effectively avoids the problem that the SDKs interfere with each other when working, and improves the recognition rate and accuracy of voice recognition.

[0070] When using the SDK, the SDK can be initialized. First, configure the AppId, AppKey, AppSecret and other related parameters. After successful configuration, configure the ASR, TTS initialization code. When configuring ASR, configure the audio sampling rate, timeout time, scene ID and proxy address and other information, and when configuring TTS, configure the timeout time, scene ID and proxy address and other information. In this way, the order of the call can be ensured as much as possible, and the relevant initialization configuration information can also be prevented from being missed.

[0071] Figure 1 is an implementation environment schematic diagram of a voice interaction method provided by an embodiment of the present application. Referring to Figure 1 , the software and hardware main body of the implementation environment mainly includes an operation terminal 101 and a server 102, and the operation terminal 101 is in communication connection with the server 102. Wherein, the voice interaction method can be configured to be executed on the operation terminal 101 alone, or can be configured to be executed on the server 102 alone, or can be executed based on the interaction between the operation terminal 101 and the server 102, and the specific selection can be appropriately selected according to the actual application, and the embodiment does not make specific limitation on this. In addition, the operation terminal 101 and the server 102 can be nodes in a block chain, and the embodiment does not make specific limitation on this.

[0072] Specifically, the operation terminal 101 in the present application can include but is not limited to any one or more of a smart watch, a smart phone, a computer, a personal digital assistant (PDA), a smart voice interaction device, a smart home appliance or a vehicle-mounted terminal. The server 102 can be a stand-alone physical server, or a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and basic cloud computing services such as big data and artificial intelligence platforms. The operation terminal 101 and the server 102 can establish a communication connection through a wireless network or a wired network, and the wireless network or the wired network uses standard communication technology and / or protocol, and the network can be set as the Internet, or any other network, for example, including but not limited to a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a private network or any combination of a virtual private network.

[0073] Figure 2is a flowchart of a voice interaction method provided by an embodiment of the present application. The execution subject of the method can be at least one of an operation terminal or a server, Figure 2 The voice interaction method is configured in the operation terminal for execution, for example. Referring to Figure 2 The voice interaction method includes, but is not limited to, steps 110 to 140.

[0074] The voice interaction method is based on a voice interaction toolkit of a smart terminal, which includes a voice recognition SDK, a voice generation SDK, and a voice playback SDK.

[0075] Step 110: Obtain input voice information through a smart terminal.

[0076] In the embodiment of the present application, the voice information mainly includes voice data of a user through a smart terminal. Specifically, in the embodiment, the voice information can be directly collected through a sound collecting device of the smart terminal.

[0077] Step 120: Load the voice recognition SDK to recognize the voice information and generate text information corresponding to the voice information.

[0078] In this step, the voice recognition SDK in the voice interaction toolkit needs to be loaded. Since the voice information is unstructured data, the voice recognition SDK can use automatic speech recognition technology (ASR) to recognize the voice information. Then, the text content of the voice information can be converted into structured data, such as a vector, through natural language processing technology, so as to take the converted structured data as text feature information.

[0079] Step 130: Generate voice interaction information according to the text information through the voice generation SDK.

[0080] In this step, after obtaining the text information, it can be input into a voice interaction model trained in the voice generation SDK to obtain voice interaction information. The voice interaction model can perform sentence-level segmentation processing on the obtained text information to obtain multiple sentences. Then, each sentence is processed for word segmentation to obtain word groups that constitute the sentence. Next, the obtained word groups are input into a previously constructed knowledge graph to obtain voice interaction information corresponding to the word groups.

[0081] Here, the speech interaction model can be trained by a labeled training data set, and the training data set can be input into the initialized speech interaction model for training. Specifically, after the data in the training data set is input into the initialized speech interaction model, a recognition result, i.e., a keyword prediction result, output by the model can be obtained, and the accuracy of the recognition model prediction can be evaluated according to the keyword prediction result and the aforementioned label, so as to update the parameters of the model. For the speech interaction model, the accuracy of the model prediction result can be measured by a loss function. The loss function is defined on a single training data and is used to measure the prediction error of a training data. Specifically, the loss value of the training data is determined by the label of the single training data and the prediction result of the model for the training data. In actual training, a training data set has many training data, and therefore a cost function is generally used to measure the overall error of the training data set. The cost function is defined on the entire training data set and is used to calculate the average value of the prediction errors of all training data, so as to better measure the prediction effect of the model. For a general machine learning model, based on the aforementioned cost function, a regular term that measures the complexity of the model can be added as a training objective function, and based on the objective function, the loss value of the entire training data set can be obtained. There are many commonly used loss functions, such as 0-1 loss function, square loss function, absolute loss function, logarithmic loss function, and cross-entropy loss function, which can all be used as the loss function of the machine learning model. Here, they will not be described one by one. In the embodiments of the present application, any one of the loss functions can be selected to determine the loss value of the training. Based on the loss value of the training, the parameters of the model are updated by using a back propagation algorithm, and after several iterations, a trained speech interaction model can be obtained. Specifically, the number of iterations can be preset, or the training can be considered to be completed when the test set reaches the accuracy requirement.

[0082] Step 140: clearing the speech recognition SDK and loading the speech playing SDK to play the speech interaction information.

[0083] In this step, in order to avoid the voice information generated by the work of the speech playing SDK from being input by the speech recognition SDK, resulting in distortion of the text information generated by the speech recognition SDK. It is necessary to clear the speech recognition SDK before loading the speech playing SDK to stop the work of the speech recognition SDK. After clearing the speech recognition SDK, the speech playing SDK can play the speech interaction information generated by the speech generation SDK.

[0084] The voice interaction method disclosed by the embodiment of the application is applied to a smart terminal including but not limited to a smart phone, a palm computer, a tablet computer, a smart television (Smart TV), and the like. The method performs voice interaction based on a voice interaction toolkit of the smart terminal. The method acquires input voice information through the smart terminal. Then, the voice recognition SDK is loaded to recognize the voice information, thereby generating text information corresponding to the voice information. Subsequently, the voice generation SDK is used to generate voice interaction information according to the text information. Then, the voice recognition SDK is cleared, and the voice play SDK is loaded to play the voice interaction information. The voice recognition SDK, the voice generation SDK, and the voice play SDK are integrated into the voice interaction toolkit. The voice interaction is realized through the voice recognition SDK, the voice generation SDK, and the voice play SDK. The calling order of the SDKs is limited. The problem that the work of the SDKs interferes with other SDKs is effectively avoided. Therefore, the recognition rate and the accuracy of voice recognition are improved.

[0085] In addition, the voice interaction method according to the above embodiment of the application can have the following additional technical features.

[0086] With reference to Figure 3 As a further optional embodiment, the step of loading the voice recognition SDK to recognize the voice information and generating text information corresponding to the voice information specifically includes:

[0087] Step 121: Acquire a voiceprint feature of the voice information.

[0088] Step 122: Compare the voiceprint feature with a voiceprint database corresponding to a user to determine whether the voice information is voice information generated by the user.

[0089] Step 123: When the voice information is voice information generated by the user, recognize the voice information through the voice recognition SDK to generate text information corresponding to the voice information.

[0090] Specifically, in the embodiments of the present application, in order to avoid voice interaction with people other than the user, it is necessary to compare the voiceprint features of the voice information first to confirm whether the voice information is voice information generated by the user. Specifically, the voiceprint of the user is collected and stored in the database, and when the voiceprint database is completed, 1:1 identity confirmation and voiceprint recognition can be performed based on the voiceprint database. The voiceprint features of the acquired voice information are compared with the massive voiceprints in the voiceprint library to determine the identity of the unknown voice. After confirming that the voice information is voice information generated by the user, the voice recognition SDK is called to extract the text information in the voice information. When extracting the text information, the voice information needs to be texted first. Automatic speech recognition (ASR) technology can be used to recognize the voice information to obtain the text content of the voice information. Specifically, in the recognition stage, the feature vector of the input voice is compared with each template in the template library of the voice recognition SDK in turn, and the one with the highest similarity is output as the recognition result.

[0091] Optionally, the text content of the voice information is converted into structured data such as a vector through natural language processing technology, so as to take the converted structured data as text feature information. In addition, in some embodiments, a vector index can be set for the vector form of the acoustic feature information to reduce the data operation amount in the matching query process.

[0092] Optionally, the identity information of the user can also be identified, and face recognition can be used for identity verification. Specifically, after obtaining the image data, the image data needs to be feature extracted to obtain the face recognition features. Then, the face recognition features are matched with the target recognition features in the face database of the user to determine whether the collected image information includes the face image information of the user, so as to confirm whether the person who performs the decompression operation is the user himself / herself.

[0093] Referring to Figure 4 As a further optional embodiment, the step of comparing the voiceprint features with the voiceprint database corresponding to the user to determine whether the voice information is voice information generated by the user specifically includes:

[0094] Step 1221: extracting first voiceprint features of the voice information;

[0095] Step 1222: extracting second voiceprint features of the user from the voiceprint database;

[0096] Step 1223: determining the similarity of the first voiceprint features and the second voiceprint features;

[0097] Step 1224: determining whether the similarity is greater than a first preset threshold value;

[0098] Step 1225: when the similarity is greater than the first preset threshold value, the voice information is user-generated voice information.

[0099] Step 1226: when the similarity is less than or equal to the first preset threshold value, the voice information is not user-generated voice information.

[0100] Specifically, the first voiceprint feature here can be a digital feature of the audio spectrum of the voice information. Specifically, some time-frequency points can be selected from the audio spectrum of the voice information according to a predetermined rule, and encoded into a digital sequence, which can be used as the acoustic feature information of the voice information. Of course, the feature extraction algorithm used in the embodiments of the present application can be optional according to needs. For example, in some embodiments, the voice information can be divided into frames and windowed according to a certain frame stacking, to obtain a plurality of audio frames. Specifically, frame division refers to cutting the whole voice information into multiple segments according to a predetermined rule, and each segment is an audio frame. Windowing refers to using a predetermined window function to cut each audio frame, so that the frame-divided voice information is more coherent and exhibits better periodic function characteristics. The window function can be any one of a rectangular window, a Hamming window, or a Gaussian window. Then, the obtained audio frames are subjected to fast Fourier transform (FFT, Fast Fourier Transformation), to obtain the spectrum of each audio frame. The peak signal in each frame spectrum is selected as the digital feature of the spectrum to construct the acoustic feature information of the voice information. In some embodiments, after obtaining the spectrum of the audio frame, the audio frame can also be divided into multiple subbands in the frequency domain, and then the difference value of the energy difference between any two adjacent subbands is calculated. The acoustic feature information of the voice information is constructed based on the obtained difference value. The above-mentioned method generally converts the digital feature into a hash sequence when encoding, i.e., using a hash sequence as the acoustic feature information of the audio data. In some embodiments, the audio digital feature of the voice information can also be extracted as the acoustic feature information through a neural network model, and the data format obtained can be any one of a numerical value, a vector, or a matrix. Of course, the above-mentioned method of extracting acoustic feature information based on the audio spectrum is only used for illustration and does not mean to limit the actual implementation of the present application. For example, the acoustic feature information can also be extracted based on the accuracy of pronunciation, fluency, prosody, signal-to-noise ratio, sound intensity, etc. In some embodiments, the acoustic feature information extracted from multiple dimensions can also be integrated to obtain new acoustic feature information, such as a new vector form of acoustic feature information obtained by splicing multiple vector forms of acoustic feature information.

[0101] In some embodiments, the similarity comparison of the first voiceprint feature and the second voiceprint feature can be completed by a machine learning model that is trained in advance. Specifically, the first voiceprint feature and the second voiceprint feature are input into the comparison model, the similarity of the two is calculated by the comparison model, and according to the size of the similarity, it is determined whether the voice information is the voice information generated by the user. The similarity here is used to represent the degree of similarity between the first voiceprint feature and the second voiceprint feature. In the embodiments of the present application, the similarity can be measured in the form of percentage. At this time, it can be understood that when the value of the similarity is 100%, it can be considered that the first voiceprint feature and the second voiceprint feature are completely identical. It can be understood that since the score result predicted by the machine learning model is not necessarily completely reliable, and there is noise interference in the environment where the voice information is collected, the similarity may have certain errors, therefore, when the value of the similarity exceeds the first preset threshold, it can be considered that the first feature information and the second feature information are completely identical. The first preset threshold can be set according to the actual situation by those skilled in the art.

[0102] Specifically, in determining the similarity between the first voiceprint feature and the second voiceprint feature, in some embodiments, the difference value between the digital features of the first feature information and the second feature information can be determined first, and then the similarity is determined according to the difference value. The greater the difference value, the smaller the similarity, and vice versa. Taking the first feature information and the second feature information as the aforementioned extracted acoustic feature information as an example, the variance between the sequences corresponding to the first feature information and the second feature information can be calculated first, and then the absolute value of the variance is taken as the difference value. The similarity is determined by the difference value and a predetermined function, which makes the similarity and the difference value negatively correlated. It should be understood that there can be many ways to calculate the difference value, such as the difference between numerical values, the Euclidean distance between vectors, the norm difference between matrices, etc.

[0103] In some embodiments, the similarity between the first feature information and the second feature information can also be calculated directly by a similarity algorithm. Taking the first feature information and the second feature information as the aforementioned extracted text feature information, and the data structure of the first feature information and the second feature information as a vector form as an example, the vector corresponding to the first feature information is denoted as a first vector, and the vector corresponding to the second feature information is denoted as a second vector. Then, the similarity can be calculated based on the first vector and the second vector by a cosine similarity algorithm, a Pearson correlation coefficient method, or a Jaccard similarity coefficient method, etc. Specifically, for example, the length of the first vector is first determined and denoted as a first length, and the length of the second vector is determined and denoted as a second length. Then, the product of the first length and the second length is calculated as a first value, and the inner product of the first vector and the second vector is calculated as a second value. Then, the quotient of the first value and the second value is calculated as the similarity between the first vector and the second vector, that is, the similarity between the first feature information and the second feature information. The algorithm formula used in the embodiments of the present application is as follows:

[0104]

[0105] In the formula, a represents the first feature information, b represents the second feature information, and sim(a, b) represents the similarity. represents the first vector. represents the second vector; and “·” represents the inner product between vectors. represents the length of the first vector. represents the length of the second vector.

[0106] With reference to Figure 5 As a further optional embodiment, the step of loading the speech recognition SDK to recognize the speech information and generate text information corresponding to the speech information specifically includes:

[0107] Step 124: Acquiring the volume intensity of the speech information.

[0108] Step 125: When the volume intensity is greater than a second preset threshold, recognizing the speech information by the speech recognition SDK to generate text information corresponding to the speech information.

[0109] In the embodiments, before calling the speech recognition SDK to recognize the speech information, the volume intensity of the speech information is first acquired, and it is determined whether the volume intensity is greater than a second preset threshold. Only when the volume intensity is greater than the preset threshold, the received speech information is processed, so as to avoid the interference of external noise and ensure that the speech information of a specific user is processed. The second preset threshold is the minimum threshold for the speech recognition SDK to start working, which can be set in advance in the voice interaction tool package, and the user can also adjust it according to his own needs.

[0110] Referring to Figure 6 As a further optional embodiment, the step of generating voice interaction information according to the text information by the voice generation SDK specifically comprises:

[0111] Step 131: extracting text feature information of the text information;

[0112] Step 132: inputting the text feature information into a voice interaction model to generate voice interaction information.

[0113] Specifically, in the embodiments of the present application, after the text information of the voice information is extracted, the obtained text content can be first subjected to sentence-level segmentation processing to obtain a plurality of sentences. Then each sentence is subjected to word segmentation processing to obtain word groups constituting the sentence. Here, a variety of word segmentation algorithms can be used, for example, in some embodiments, a dictionary-based word segmentation algorithm can be used, the sentence is first segmented into words according to the dictionary, and then the optimal combination mode of the words is searched; in some embodiments, a word-based word segmentation algorithm can also be used, the sentence is first divided into individual words, and then the words are combined into a word to find the optimal combination mode. After the sentence is subjected to word segmentation processing, the word embedding vector corresponding to each word in the word group can be determined through a pre-established dictionary. Of course, in some embodiments, the word embedding vector can be obtained by mapping the word to a vector space with a uniform lower dimension, and the strategies for generating such mapping include neural networks, dimensionality reduction of word co-occurrence matrices, probabilistic models, and interpretable knowledge base methods, etc. In the embodiments of the present application, these structured data can be used as text feature information. Of course, similarly, the above-mentioned manner of extracting text feature information based on the semantics of the text content is only used for illustration and does not mean to limit the actual implementation of the present application. In the present application, text feature information can also be extracted based on grammatical features, pragmatic features, keyword hit features, etc. dimensions, and the text feature information extracted from multiple dimensions can also be integrated to obtain new text feature information, which will not be described one by one here.

[0114] After obtaining the text feature information of the voice information, it is input into the voice interaction model, and the voice interaction model generates reply content based on the text feature information. The voice interaction model parses the interactive keywords from the text feature information. Specifically, the text feature information is matched with a pre-constructed knowledge graph database, and the interactive keywords are determined according to the matching result. Then, the interactive keywords are matched with the knowledge graph database again to obtain the reply content corresponding to the interactive keywords.

[0115] Referring to Figure 7 As a further optional embodiment, the following steps are further included:

[0116] Step 150: input the voice information into a noise detection model to obtain a noise detection result output by the noise detection model; the noise detection result is used to represent whether the voice information contains noise data;

[0117] Step 160: determine the confidence of the voice interaction information according to the noise detection result.

[0118] In the embodiments of the present application, the noise detection model can be used to detect whether the voice information contains noise data. Specifically, at this time, the acoustic feature information described above can be input into the noise detection model, and the noise detection model processes the acoustic feature information and outputs a noise detection result. Similarly, in some embodiments, the noise detection result can be a classification result, such as a category representing "contains noise" or a category representing "does not contain noise"; in some embodiments, the noise detection result can also be a numerical type result, which can be used to represent the proportion of noise data contained in the voice information, such as 0-100%. Of course, in the embodiments of the present application, the noise detection model can be further subdivided, such as establishing an environmental noise model to detect environmental noise in the voice information, establishing a human voice noise model to detect human voice noise in the voice information, and the like. The noise detection model in the embodiments of the present application can be built based on any machine learning algorithm, such as a basic classification algorithm or a regression algorithm for the noise detection model. After obtaining the noise detection result, a noise score of the voice information can be determined, which can be used to quantify the degree of noise in the voice information data. After determining the noise score of the voice information, the reliability of the voice interaction result obtained according to the voice information data can be effectively quantified, that is, the confidence of the voice interaction information. Specifically, it can be understood that the reliability of the voice interaction information and the noise score of the voice information are negatively correlated, that is, the higher the noise score of the voice information, the less reliable the voice interaction information and the lower the confidence; on the contrary, the lower the noise score of the voice information, the more reliable the voice interaction information and the higher the confidence.

[0119] Referring to Figure 8 , as a further optional embodiment, the following steps are further included:

[0120] Step 170: recognizing the voice information through the voice recognition SDK to generate an operation instruction corresponding to the voice information;

[0121] Step 180: controlling the intelligent terminal according to the operation instruction.

[0122] In this embodiment, the feature information of the voice information can be extracted, and then the feature information is matched with the voice information in the preset database, so as to determine the operation instruction. Specifically, the feature information here can include acoustic feature information of the voice information, or can be text feature information.

[0123] The preset database can include the corresponding relationship between the voice information and the operation instruction. For example, when the user speaks the voice information "open NetEase cloud music", it is determined by the voice recognition model that the voice information "open NetEase cloud music" corresponds to the control instruction, and then the control instruction is executed, when the user speaks the voice information "I want to listen to songs", it is determined by the voice recognition engine that the voice information "listen to songs" corresponds to the control instruction, so as to open the music application program on the smart terminal.

[0124] Optionally, the operation keyword in the voice information can be determined first, and then the feature information of the operation keyword is extracted. For example, the user issues the voice information "I want to open QQ music", the operation keywords "open" and "QQ music" can be extracted from the voice information, and then the feature information of the operation keywords "open" and "QQ music" is extracted, and the operation instruction is recognized by the feature information. The operation instruction includes at least one of the Home key instruction, the open instruction, the back instruction, the return instruction, the forward instruction and the close instruction.

[0125] Optionally, when the voice information cannot be recognized by the voice recognition model, a prompt information can be displayed, the prompt information is used to prompt the user to re-input the voice information, and if the number of times that the voice information cannot be recognized by the voice recognition engine exceeds a preset threshold, the voice control mode can be switched to the manual operation mode, and the user is prompted to manually operate.

[0126] In some embodiments, the smart terminal will make a corresponding response according to the above prompt information, and realize state feedback display through indicator light, sound and display module. The state feedback display mode of the display module can be directly reminded in the form of text in the display screen or the display interface of the APP of the smart terminal, and the text can be Chinese characters or characters of other countries. Optionally, the prompt mode of the prompt information can also be to switch the display color of the preset offline reminder information prompt area from the first color (such as green) to the second color (such as red and the like) in the display screen or the display interface of the APP of the smart terminal.

[0127] Reference Figure 9 The embodiment of the application also provides a voice interaction system, which is based on a voice interaction toolkit of a smart terminal and includes a voice recognition SDK, a voice generation SDK and a voice playback SDK. The system includes the following modules:

[0128] The first module 901 is used to acquire input voice information through a smart terminal;

[0129] The second module 902 is used to load the speech recognition SDK to recognize the speech information and generate text information corresponding to the speech information;

[0130] The third module 903 is used to generate voice interaction information based on the text information using the voice generation SDK;

[0131] The fourth module 904 is used to clear the speech recognition SDK and load the speech playback SDK to play the speech interaction information.

[0132] Understandable Figure 2 The content of the voice interaction method embodiments shown is applicable to the system embodiments of this voice interaction. The specific functions implemented in the system embodiments of this voice interaction are the same as those in the previous embodiments. Figure 2 The voice interaction method shown in the embodiment is the same, and the beneficial effects achieved are the same. Figure 2 The beneficial effects achieved by the illustrated voice interaction method embodiment are also the same.

[0133] Reference Figure 10 This application also discloses a voice interaction device, comprising:

[0134] At least one processor 1001;

[0135] At least one memory 1002 is used to store at least one program;

[0136] When at least one program is executed by at least one processor 1001, causing at least one processor 1001 to perform as follows: Figure 2 The illustrated embodiment is a method for voice interaction in an application.

[0137] It is understandable that, such as Figure 2 The content of the voice interaction method embodiments shown is applicable to the embodiments of this computer device, and the specific functions implemented by the embodiments of this computer device are the same as those shown. Figure 2 The method embodiment for voice interaction shown is the same, and the beneficial effects achieved are the same as those described above. Figure 2 The beneficial effects achieved by the illustrated voice interaction method embodiment are also the same.

[0138] This application also discloses a computer-readable storage medium storing a processor-executable program, which, when executed by a processor, is used to implement, for example... Figure 2 The illustrated embodiment of the voice interaction method.

[0139] It is understandable that, such as Figure 2The contents of the method embodiments of application voice interaction shown are applicable to the computer readable storage medium embodiments, the functions realized by the computer readable storage medium embodiments are the same as the functions realized by the method embodiments of application voice interaction shown, and the beneficial effects achieved are the same as the beneficial effects achieved by the method embodiments of voice interaction shown. Figure 2 The functions / operations mentioned in the block diagram in the computer readable storage medium embodiments are the same as the functions / operations mentioned in the block diagram in the method embodiments of application voice interaction shown, and the beneficial effects achieved are the same as the beneficial effects achieved by the method embodiments of application voice interaction shown. Figure 2 The functions / operations mentioned in the block diagram in the computer readable storage medium embodiments are the same as the functions / operations mentioned in the block diagram in the method embodiments of voice interaction shown, and the beneficial effects achieved are the same as the beneficial effects achieved by the method embodiments of voice interaction shown.

[0140] In some alternative embodiments, the functions / operations mentioned in the block diagram can not occur in the order mentioned in the operation diagram. For example, depending on the functions / operations involved, two blocks shown in succession can actually be executed substantially concurrently or the blocks can sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts in the present application are provided by way of example only. The disclosed methods are not limited to the order in which the operations and logic flows are presented in the present application. Alternative embodiments are contemplated in which the order of various operations is changed and in which sub-operations described as part of a larger operation are independently executed.

[0141] Furthermore, although the present application is described in the context of functional modules, it is to be understood that one or more of the functions and / or features can be integrated in a single physical device and / or software module, or one or more functions and / or features can be implemented in separate physical devices or software modules, unless otherwise specified. It is also to be understood that detailed discussion of the actual implementation of each module is unnecessary to an understanding of the present application. Rather, the properties, functions and internal relationships of the various functional modules disclosed in the devices herein are considered to be within the ordinary skill of an engineer, given the benefit of this disclosure. Therefore, the present application as set forth in the claims is capable of being realized by one of ordinary skill in the art using ordinary skill, without undue experimentation. It is also to be understood that the particular concepts disclosed are illustrative only and not intended to limit the scope of the present application, which is defined by the full scope of the claims appended hereto and their equivalents.

[0142] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0143] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0144] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0145] It should be understood that aspects of the application can be implemented in hardware, software, firmware or a combination of them. In the above embodiments, various steps or methods can be implemented in software or firmware that is stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any of the following technologies, or combinations thereof, can be used: a discrete logic circuit having logic gates for implementing logic functions upon data signals, an application specific integrated circuit having appropriate combinational logic gates, a programmable gate array (PGA), a field programmable gate array (FPGA), and / or the like.

[0146] In the above description of the present specification, the description referring to the terms "one embodiment", "another embodiment" or "certain embodiments" or the like means that a specific feature, structure, material or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. The illustrative expressions of the above terms do not necessarily refer to the same embodiment or example in the present specification. Also, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in an appropriate manner.

[0147] Although the embodiments of the present application have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, alternatives and variations to these embodiments can be made without departing from the principles and spirit of the application, and the scope of the application is defined by the claims and their equivalents.

[0148] The above is a specific description of the preferred embodiments of the present application, but the present application is not limited to the embodiments, and those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present application, and these equivalent modifications or substitutions are included in the scope defined by the claims of the present application

[0149] In the above description of the present specification, the description referring to the terms "one embodiment", "another embodiment" or "certain embodiments" or the like means that a specific feature, structure, material or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. The illustrative expressions of the above terms do not necessarily refer to the same embodiment or example in the present specification. Also, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in an appropriate manner.

[0150] Although the embodiments of the present application have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, alternatives and variations to these embodiments can be made without departing from the principles and spirit of the application, and the scope of the application is defined by the claims and their equivalents.

Claims

1. A method of voice interaction, characterized by, The method is based on the voice interaction of the intelligent terminal voice interaction toolkit, the voice interaction toolkit includes voice recognition SDK, voice generation SDK and voice playback SDK, the method includes the following steps: Obtain the input voice information through the intelligent terminal; Load the voice recognition SDK to recognize the voice information, and generate text information corresponding to the voice information; Generate voice interaction information according to the text information through the voice generation SDK; Clear the voice recognition SDK, and load the voice playback SDK to play the voice interaction information; The step of loading the voice recognition SDK to recognize the voice information and generate text information corresponding to the voice information specifically includes: Obtain the volume intensity of the voice information; When the volume intensity is greater than a second preset threshold, recognize the voice information through the voice recognition SDK to generate text information corresponding to the voice information; The step of generating voice interaction information according to the text information through the voice generation SDK specifically includes: Carry out sentence-level segmentation processing on the text information to obtain a plurality of sentences; Carry out word segmentation processing on each of the sentences to obtain word groups constituting the sentences; Determine the word embedding vector corresponding to each word in the word group through a pre-established dictionary, and obtain text feature information of the text information according to the word embedding vector; Input the text feature information into a voice interaction model to generate voice interaction information.

2. The method of voice interaction of claim 1, wherein, The step of loading the voice recognition SDK to recognize the voice information and generate text information corresponding to the voice information specifically includes: Obtain the voiceprint feature of the voice information; Compare the voiceprint feature with the voiceprint database corresponding to the user to determine whether the voice information is voice information generated by the user; When the voice information is voice information generated by the user, recognize the voice information through the voice recognition SDK to generate text information corresponding to the voice information.

3. The method of voice interaction according to claim 2, wherein, The step of comparing the voiceprint feature with the voiceprint database corresponding to the user to determine whether the voice information is voice information generated by the user specifically includes: Extract the first voiceprint feature of the voice information; Extract the second voiceprint feature of the user from the voiceprint database; Determine the similarity of the first voiceprint feature and the second voiceprint feature; Determine whether the similarity is greater than a first preset threshold; When the similarity is greater than the first preset threshold, the voice information is voice information generated by the user; When the similarity is less than or equal to the first preset threshold, the voice information is not voice information generated by the user.

4. The method of voice interaction of claim 1, wherein, Further comprising the following steps: Input the voice information into a noise detection model to obtain a noise detection result output by the noise detection model; The noise detection result is used to represent whether the voice information contains noise data; According to the noise detection result, determine the confidence degree of the voice interaction information.

5. The method of voice interaction according to any one of claims 1-4, characterized in that, Further comprising the following steps: The voice information is recognized by the voice recognition SDK to generate an operation instruction corresponding to the voice information; The intelligent terminal is controlled according to the operation instruction.

6. A system for voice interaction, characterized by The system performs voice interaction based on a voice interaction toolkit of an intelligent terminal, the voice interaction toolkit including a voice recognition SDK, a voice generation SDK and a voice playback SDK, and the system includes the following modules: A first module for obtaining input voice information through an intelligent terminal; A second module for loading the voice recognition SDK to recognize the voice information and generate text information corresponding to the voice information; A third module for generating voice interaction information according to the text information through the voice generation SDK; A fourth module for clearing the voice recognition SDK and loading the voice playback SDK to play the voice interaction information; The second module is specifically configured to: Obtain the volume intensity of the voice information; When the volume intensity is greater than a second preset threshold, recognize the voice information through the voice recognition SDK to generate text information corresponding to the voice information; The third module is specifically configured to: Perform sentence-level segmentation processing on the text information to obtain a plurality of sentences; Perform word segmentation processing on each of the sentences to obtain word groups constituting the sentences; Determine a word embedding vector corresponding to each word in the word groups through a pre-established dictionary, and obtain text feature information of the text information according to the word embedding vector; Input the text feature information into a voice interaction model to generate voice interaction information.

7. An apparatus for voice interaction, characterized by Comprise: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the voice interaction method of any one of claims 1-5.

8. A computer-readable storage medium having stored therein instructions that are executable by a processor, the instructions comprising: The instructions executable by the processor when executed by the processor are used to implement the voice interaction method of any one of claims 1-5.

Images