Voice interaction recognition method and device

Abstract

The invention discloses a voice interaction recognition method and device, and the method comprises the steps: obtaining first interaction audio data, which is received by equipment for the first time after the equipment is awakened at this time, under the condition that the equipment is awakened; comparing the similarity between the first interactive audio data and the second interactive audio data under the condition that the second interactive audio data which is not received for the first time after the device is awakened this time is obtained; judging whether the second interactive audio data and the first interactive audio data are from the same object or not according to the similarity; and according to a judgment result of whether the second interactive audio data and the first interactive audio data are from the same object, determining whether to perform intelligent voice reply on an identification result of the second interactive audio data. By comparing the similarity of the audio data, whether the current voice input comes from the user who interacts for the first time or not can be accurately judged. The voice of other people is prevented from being mistakenly recognized, and meanwhile the device is prevented from rejecting a correct instruction of a user.

Description

Technical Field

[0001] This application relates to the field of speech recognition, and more specifically, to a method and apparatus for recognizing speech interaction. Background Technology

[0002] With the widespread adoption of smart homes, wearable devices, and mobile smart terminals, voice interaction has become a crucial method of human-computer communication. Users can control devices and perform various functions through simple voice commands, greatly improving the convenience and comfort of daily life.

[0003] However, in related technologies, users are easily interfered with by the surrounding voices when interacting with the device via voice, which can cause the device to respond incorrectly. Summary of the Invention

[0004] This invention provides a voice interaction recognition method and apparatus to at least solve the problem in related technologies where users are easily interfered with by surrounding human voices when interacting with devices via voice, causing the devices to respond erroneously.

[0005] According to one embodiment of the present invention, a voice interaction recognition method is provided, comprising: when a device is woken up, acquiring first interactive audio data first received by the device after the current wake-up; when acquiring second interactive audio data not first received by the device after the current wake-up, comparing the similarity between the first interactive audio data and the second interactive audio data; determining, based on the similarity, whether the second interactive audio data and the first interactive audio data come from the same object; and determining, based on the determination result of whether the second interactive audio data and the first interactive audio data come from the same object, whether to provide an intelligent voice response to the recognition result of the second interactive audio data.

[0006] In one exemplary embodiment, determining whether second interactive audio data and first interactive audio data originate from the same object based on similarity includes: comparing a first similarity between the first interactive audio data and the second interactive audio data with a first similarity threshold, wherein the first similarity is obtained by comparing the first interactive audio data and the second interactive audio data using a first speech recognition model deployed locally on the device; if the first similarity is greater than or equal to the first similarity threshold, uploading the first interactive audio data and the second interactive audio data to the cloud via the device; comparing the first interactive audio data and the second interactive audio data using a second speech recognition model deployed in the cloud to determine a second similarity between the first interactive audio data and the second interactive audio data; if the second similarity is greater than or equal to the second similarity threshold, determining that the second interactive audio data and the first interactive audio data originate from the same object, wherein the accuracy of the second speech recognition model deployed in the cloud is higher than the accuracy of the first speech recognition model deployed locally on the device.

[0007] In one exemplary embodiment, the method further includes: if the first similarity is less than a first similarity threshold, determining that the second interactive audio data and the first interactive audio data come from different objects; and if the second similarity is less than a second similarity threshold, determining that the second interactive audio data and the first interactive audio data come from different objects.

[0008] In one exemplary embodiment, comparing first interactive audio data and second interactive audio data to determine the similarity between the second interactive audio data and the first interactive audio data includes: performing feature extraction on the first interactive audio data and the second interactive audio data respectively to obtain a first feature vector of the first interactive audio data and a second feature vector of the second interactive audio data; calculating the relative distance or relative angle between the first feature vector and the second feature vector; and determining the similarity between the first feature vector and the second feature vector based on the relative distance or relative angle, wherein the similarity between the first feature vector and the second feature vector is used to indicate the similarity between the first interactive audio data and the second interactive audio data.

[0009] In an exemplary embodiment, the method further includes: obtaining a preset test dataset and a preset range of similarity threshold values, wherein the test dataset includes multiple sets of test data, each set of test data includes two audio data points and a corresponding Boolean value, the Boolean value being used to indicate whether the two audio data points in the same set come from the same object; comparing the two audio data points belonging to the same set in the test dataset to obtain the similarity corresponding to each set of test data in the test dataset; and determining a target similarity threshold within the range of similarity threshold values ​​based on the similarity and Boolean value corresponding to each set of test data in the test dataset, wherein the target similarity threshold is a similarity threshold that makes the misclassification rate of each set of test data in the test dataset less than an error threshold.

[0010] In an exemplary embodiment, determining a target similarity threshold within a range of similarity threshold values ​​based on the similarity and Boolean values ​​corresponding to each group of test data in the test dataset includes: judging the similarity of each group of test data in the test dataset using the upper limit of the range to determine a first judgment result set; and judging the similarity of each group of test data in the test dataset using the lower limit of the range to determine a second judgment result set, wherein the first and second judgment result sets include judgment results on whether two audio data points in each group of test data in the test dataset originate from the same object; based on the similarity and Boolean values ​​corresponding to each group of test data in the test dataset... For each set of test data, a Boolean value, a first judgment result set, and a second judgment result set are used to determine the first misclassification rate corresponding to the upper limit and the second misclassification rate corresponding to the lower limit. If the difference between the first misclassification rate and the second misclassification rate is greater than or equal to a set threshold, the range of similarity thresholds is updated, and the process of judging the similarity of each set of test data in the test dataset using the upper limit of the range is repeated to determine the first judgment result set until the difference between the first misclassification rate and the second misclassification rate is less than the set threshold. If the difference between the first misclassification rate and the second misclassification rate is less than the set threshold, any similarity threshold in the current range is used as the target similarity threshold.

[0011] In an exemplary embodiment, the method further includes: when the device accesses a registered voiceprint library, verifying whether the recognition result of the second interactive audio data and the first interactive audio data comes from the same object using the registered voiceprint library, so as to determine the confidence level of the recognition result, wherein the registered voiceprint library includes registered voiceprints of multiple different objects; if the confidence level of the recognition result meets the standard, adding the first interactive audio data, the second interactive audio data and the recognition result as a set of test data to a test dataset, wherein the recognition result is represented by the corresponding Boolean value.

[0012] In one exemplary embodiment, determining whether to provide an intelligent voice response to the recognition result of the second interactive audio data based on the judgment result of whether the second interactive audio data and the first interactive audio data come from the same object includes: ignoring the second interactive audio data when it is determined that the second interactive audio data and the first interactive audio data come from different objects, wherein ignoring the second interactive audio data includes refusing to respond to the second interactive audio data and / or refusing to respond to the operation instructions indicated by the second interactive audio data.

[0013] In one exemplary embodiment, determining whether to provide an intelligent voice response to the recognition result of the second interactive audio data based on the determination result of whether the second interactive audio data and the first interactive audio data come from the same object includes: if it is determined that the second interactive audio data and the first interactive audio data come from the same object, parsing the second interactive audio data to determine semantic content; providing an intelligent voice response in response to the semantic content; and / or executing the operation instructions indicated by the semantic content.

[0014] According to another embodiment of the present invention, a voice interaction recognition device is also provided, comprising:

[0015] The audio acquisition module is used to acquire the first interactive audio data received by the device after this wake-up when the device is woken up.

[0016] The audio comparison module is used to compare the similarity between the first interactive audio data and the second interactive audio data when the device receives the second interactive audio data which is not the first time after this wake-up.

[0017] The judgment module is used to determine whether the second interactive audio data and the first interactive audio data come from the same object based on similarity.

[0018] The recognition module is used to determine whether to provide an intelligent voice response to the recognition result of the second interactive audio data based on the judgment result that the second interactive audio data and the first interactive audio data come from the same object.

[0019] The aforementioned voice interaction recognition method, through precise comparison of voiceprint features, enables smart devices to accurately determine whether subsequent audio data originates from the same user during the initial interaction, reducing false recognition incidents and improving user experience. Ensuring that smart devices only respond to voice commands verified as belonging to a specific user not only enhances device security but also strengthens user trust, improving the overall interactive experience. By combining voiceprint feature capture, audio data feature comparison, threshold judgment, and intelligent response, a complete speaker verification system is constructed, effectively solving the problem of false responses caused by interference from surrounding sounds in traditional voice interaction devices. Smart devices can capture and save the user's voiceprint information during the initial interaction. In subsequent interactions, the device can accurately determine whether the current voice input comes from the user during the initial interaction by extracting features from the audio data and comparing similarity. Preset similarity thresholds provide the basis for judgment, ensuring the security and accuracy of the device when recognizing user voice commands. This method avoids false responses caused by misidentifying other people's voices and also prevents the device from rejecting legitimate user commands due to excessive caution. Ultimately, smart devices are able to effectively respond to various commands from legitimate users, providing a highly secure, personalized, and convenient voice interaction experience. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0021] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of the hardware environment for a voice interaction recognition method according to an embodiment of this application;

[0023] Figure 2 This is a flowchart of a voice interaction recognition method according to an embodiment of the present invention;

[0024] Figure 3 This is a second flowchart of a voice interaction recognition method according to an embodiment of the present invention;

[0025] Figure 4 This is the third flowchart of a voice interaction recognition method according to an embodiment of the present invention;

[0026] Figure 5 This is the fourth flowchart of a voice interaction recognition method according to an embodiment of the present invention;

[0027] Figure 6 This is the fifth flowchart of a voice interaction recognition method according to an embodiment of the present invention;

[0028] Figure 7 This is a flowchart of the voice interaction recognition method according to an embodiment of the present invention;

[0029] Figure 8 This is the seventh flowchart of a voice interaction recognition method according to an embodiment of the present invention;

[0030] Figure 9 This is a structural block diagram of a voice interaction recognition device according to an embodiment of the present invention. Detailed Implementation

[0031] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0032] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0033] According to one aspect of the embodiments of this application, a voice interaction recognition method is provided. This voice interaction recognition method is widely used in whole-house intelligent digital control application scenarios such as smart homes, smart home ecosystems, and intelligence house ecosystems. Optionally, in this embodiment, the above-mentioned smart home device interaction method can be applied to, for example... Figure 1 The hardware environment shown consists of terminal device 102 and server 104. For example... Figure 1As shown, server 104 is connected to terminal device 102 via a network and can be used to provide services (such as application services) to the terminal or clients installed on the terminal. A database can be set up on the server or independently of the server to provide data storage services for server 104. Cloud computing and / or edge computing services can be configured on the server or independently of the server to provide data processing services for server 104.

[0034] The aforementioned network may include, but is not limited to, at least one of the following: wired network, wireless network. The aforementioned wired network may include, but is not limited to, at least one of the following: wide area network, metropolitan area network, local area network. The aforementioned wireless network may include, but is not limited to, at least one of the following: Wi-Fi (Wireless Fidelity), Bluetooth. The terminal device 102 may not be limited to PC, mobile phone, tablet computer, smart air conditioner, smart range hood, smart refrigerator, smart oven, smart stove, smart washing machine, smart water heater, smart washing equipment, smart dishwasher, smart projector, smart TV, smart clothes rack, smart curtains, smart audio-visual equipment, smart socket, smart speaker, smart speaker box, smart fresh air equipment, smart kitchen and bathroom equipment, smart bathroom equipment, smart robot vacuum cleaner, smart window cleaning robot, smart mopping robot, smart air purifier, smart steam oven, smart microwave oven, smart water heater, smart air purifier, smart water dispenser, smart door lock, etc.

[0035] This embodiment provides a voice interaction recognition method, applied to the aforementioned terminal device. Figure 2 This is a flowchart of a voice interaction recognition method according to an embodiment of the present invention, the process including the following steps S200-S230:

[0036] Step S200: When the device is woken up, acquire the first interactive audio data received by the device for the first time after this wake-up.

[0037] The first interactive audio data is audio data that includes human voices.

[0038] Specifically, after the smart device is woken up and begins to receive voice commands, the audio data of the user interacting with the device for the first time is recorded as the basis for subsequent verification.

[0039] For example, the device's microphone array is activated to record an initial audio segment containing the wake word and subsequent commands. To improve the accuracy of voiceprint recognition, the recording continues for a certain period of time to ensure that sufficient audio features are captured.

[0040] By using multiple microphones working in tandem, the accuracy of sound acquisition and its resistance to interference can be improved. A wake word is a specific word or phrase used to trigger a smart device to enter voice recognition mode.

[0041] Step S210: If the device receives second interactive audio data that is not the first time after this wake-up, compare the similarity between the first interactive audio data and the second interactive audio data.

[0042] Specifically, when a smart device receives a second interactive audio data that is not the first time, the device will compare the voiceprint features of the second interactive audio data with the first audio data to calculate the similarity between the two.

[0043] For example, a pre-trained deep learning model is used to extract features from the second interactive audio data to generate a feature vector. Then, cosine similarity or other distance metrics are used to calculate the similarity score between the feature vector of the second interactive audio data and the feature vector of the first audio data.

[0044] Step S220: Based on the similarity, determine whether the second interactive audio data and the first interactive audio data come from the same object.

[0045] Specifically, the device compares the similarity score with a preset similarity threshold to determine whether the second interactive audio data comes from the same speaker in the first interaction.

[0046] For example, a threshold judgment module can be set up inside the device or in the cloud. This module is responsible for determining whether the audio data belongs to the same speaker by comparing the similarity score with a preset threshold. If the similarity score is higher than the threshold, it is considered that the non-first audio data and the first audio data come from the same speaker; otherwise, it is considered that they are different speakers.

[0047] Step S230: Based on the judgment result of whether the second interactive audio data and the first interactive audio data come from the same object, determine whether to provide an intelligent voice response to the recognition result of the second interactive audio data.

[0048] Specifically, if the recognition of the second interactive audio data is successful, the device responds to the instruction indicated by the second interactive audio data.

[0049] Specifically, if the second interactive audio data is verified to be from the same speaker as the first interaction, the smart device will recognize and execute the indicated instructions through the second interactive audio data.

[0050] For example, when the similarity score is higher than a preset threshold, the device's voice recognition engine converts the second interactive audio data into text or instructions, and then performs the corresponding operation, such as adjusting the lights, checking the weather forecast, or playing music.

[0051] In this embodiment, through precise comparison of voiceprint features, the smart device can accurately determine whether non-first-time audio data comes from the same user during the first interaction, reducing false recognition events and improving user experience. Ensuring that the smart device only responds to voice commands verified as belonging to a specific user not only improves device security but also enhances user trust in the device, improving the overall interactive experience. By combining voiceprint feature capture, audio data feature comparison, threshold judgment, and intelligent response, a complete speaker verification system is constructed, effectively solving the problem of false responses caused by interference from surrounding sounds in traditional voice interaction devices. The smart device can capture and save the user's voiceprint information during the first interaction. In subsequent non-first-time interactions, the device can accurately determine whether the current voice input comes from the user during the first interaction by extracting features from the audio data and comparing similarity. The preset similarity threshold provides the basis for judgment, ensuring the security and accuracy of the device when recognizing user voice commands. This method avoids false responses caused by misidentifying other people's voices and also prevents the device from rejecting legitimate user commands due to excessive caution. Ultimately, smart devices are able to effectively respond to various commands from legitimate users, providing a highly secure, personalized, and convenient voice interaction experience.

[0052] In one embodiment, such as Figure 3 As shown, step S220 determines whether the second interactive audio data and the first interactive audio data originate from the same object based on the comparison result of the similarity and a preset similarity threshold. This includes steps S300-S330, wherein:

[0053] Step S300: Compare the first similarity between the first interactive audio data and the second interactive audio data with a first similarity threshold.

[0054] The first similarity is obtained by comparing the first interactive audio data and the second interactive audio data using a first speech recognition model deployed locally on the device.

[0055] Specifically, after the device is woken up and receives the second interactive audio data (which is not the first time), the smart device first uses a locally deployed first speech recognition model to calculate the similarity score between the first audio data and the second interactive audio data, i.e., the first similarity.

[0056] For example, the first speech recognition model built into the smart device is a lightweight deep learning model that runs on the device to reduce the demand for network bandwidth. When the device receives second interactive audio data, the model extracts features from the two audio files and calculates the distance between the feature vectors to obtain a first similarity.

[0057] Step S310: If the first similarity is greater than or equal to the first similarity threshold, the first interactive audio data and the second interactive audio data are uploaded to the cloud via the device.

[0058] Specifically, if the first similarity is greater than or equal to the first similarity threshold, the smart device uploads the first interactive audio data and the second interactive audio data to the cloud for further speaker verification.

[0059] For example, the device transmits audio data to a cloud server via a wireless network. The uploaded data is encrypted to protect user privacy. The cloud server can be a remote data center with powerful computing resources capable of performing complex voice data analysis.

[0060] Step S320: The first interactive audio data and the second interactive audio data are compared using a second speech recognition model deployed in the cloud to determine the second similarity between the first interactive audio data and the second interactive audio data.

[0061] Specifically, the cloud server uses a deployed second speech recognition model to compare the uploaded first and second interactive audio data to determine a second similarity.

[0062] For example, the second speech recognition model in the cloud is typically a more accurate deep learning model with a more complex architecture and more training parameters, capable of providing more accurate speaker verification results. It performs feature extraction and similarity calculation on the uploaded audio data.

[0063] Step S330: If the second similarity is greater than or equal to the second similarity threshold, determine that the second interactive audio data and the first interactive audio data come from the same object.

[0064] Among them, the accuracy of the second speech recognition model deployed in the cloud is higher than that of the first speech recognition model deployed locally on the device.

[0065] Specifically, if the second similarity is greater than or equal to the second similarity threshold, then the system determines that the second interactive audio data and the first audio data do indeed come from the same speaker.

[0066] For example, the cloud server compares the second similarity with a second similarity threshold, and if it is greater than or equal to the threshold, it considers the two audio data to come from the same object.

[0067] The second similarity threshold is the threshold used by the cloud to ultimately determine whether the speakers are the same person. It is usually set higher than the first similarity threshold to ensure higher verification accuracy.

[0068] In this embodiment, the initial judgment of the local model can quickly filter out voice commands that are clearly not intended to wake up the user, reducing the workload of cloud processing and improving the overall system response speed. Data is only uploaded to the cloud when the local model determines that the audio data may come from the same speaker, thus saving network resources and reducing the computational burden on the cloud. The high accuracy of the cloud model provides the final judgment basis, effectively avoiding false responses from the device and improving the accuracy of speaker verification. First, the initial comparison on the device's local side can quickly filter out audio from non-target speakers, reducing unnecessary cloud processing and improving the overall system response speed. Second, the deep model in the cloud provides the final, high-precision speaker verification, ensuring a secure response from the device and avoiding erroneous operations. Finally, the collaborative working mode between the local and cloud systems not only improves system security but also optimizes resource utilization, reduces the burden of network transmission and cloud computing, thereby reducing device response latency and improving the user experience.

[0069] In one embodiment, such as Figure 4 As shown, the method further includes steps S400-S410, wherein:

[0070] Step S400: If the first similarity is less than the first similarity threshold, determine that the second interactive audio data and the first interactive audio data come from different objects.

[0071] Specifically, when the device receives a second interactive audio data that is not the first time, it first calculates the first similarity between the second interactive audio data and the first audio data using a locally preset first speech recognition model. If the first similarity is lower than the first similarity threshold, the device immediately determines that the two audio data come from different objects.

[0072] For example, the processor inside the device runs a first speech recognition model to quickly extract features and calculate similarity from the audio data. Similarity calculations typically use algorithms such as cosine similarity or Euclidean distance. A first similarity threshold is preset for initial local screening.

[0073] Step S410: If the second similarity is less than the second similarity threshold, determine that the second interactive audio data and the first interactive audio data come from different objects.

[0074] Specifically, if the device determines that the first similarity is greater than or equal to the first similarity threshold, it will upload the first and second interactive audio data to the cloud. A second speech recognition model deployed in the cloud will then perform a more accurate similarity calculation to obtain a second similarity. If the second similarity is lower than the second similarity threshold, it will also be determined that the two audio data sets come from different objects.

[0075] For example, the device uploads audio data to a cloud server via a secure network connection and uses a high-precision second speech recognition model to perform a deep comparison of feature vectors. This model typically has more training data and a more complex neural network structure, thus providing a more accurate similarity score.

[0076] In this embodiment, the initial local determination can quickly exclude audio data that is clearly not intended for the user's wake-up, reducing unnecessary cloud processing, improving response speed, and protecting user privacy. It avoids uploading all audio data to the cloud, only uploading audio data that may belong to the same speaker, reducing data transmission costs and server load. Using a high-precision model in the cloud for final similarity comparison significantly improves the accuracy of speaker verification, reduces false identification and rejection events, and enhances device security and user experience. Furthermore, uploading audio data only when initially determined to be likely from the same speaker effectively reduces the amount of data transmitted over the network, lowering communication costs and improving system response efficiency. First, the device's lightweight local model can quickly calculate the similarity of audio data. By determining a first similarity threshold, audio data that is clearly not intended for the user's wake-up is quickly rejected, reducing unnecessary cloud processing, improving system response speed, and lowering the burden on cloud servers and network transmission costs. Second, when a high similarity is initially determined, the audio data is uploaded to the cloud for final verification using a more precise model, ensuring the accuracy of the verification results, effectively preventing false identification and rejection events, and improving device security and user experience.

[0077] In one embodiment, such as Figure 5 As shown, step S210 involves comparing the first and second interactive audio data to determine their similarity when the device receives second interactive audio data that is not the first time it has been received since the current wake-up. This includes steps S500-S520, where:

[0078] Step S500: Extract features from the first interactive audio data and the second interactive audio data respectively to obtain the first feature vector of the first interactive audio data and the second feature vector of the second interactive audio data.

[0079] Specifically, after receiving audio data, the smart device first needs to extract features from the audio data, converting the audio signal into feature vectors that can be analyzed and compared. This step is the core of speaker verification because it transforms audio information into a series of values ​​that reflect the speaker's voiceprint characteristics.

[0080] For example, feature extraction is typically performed by deep learning models such as Convolutional Neural Networks (CNNs) or Recurrent Neural Networks (RNNs). CNNs excel at capturing local features of audio, while RNNs can handle the time-series characteristics of audio. After receiving audio data, the model converts it into a representation such as a spectrogram or Mel-Frequency Cepstral Coefficients (MFCCs), then extracts key feature points, and finally generates a feature vector.

[0081] Step S510: Calculate the relative distance or relative angle between the first feature vector and the second feature vector.

[0082] Specifically, after obtaining the feature vectors of the first and second interactive audio data, the smart device (or cloud server) needs to calculate the relative distance or angle between the two feature vectors to quantify their similarity. The similarity score is a key indicator for determining whether two audio data sets originate from the same speaker.

[0083] For example, relative distance can be calculated using the Euclidean distance method, which calculates the straight-line distance between the endpoints of two vectors. Relative angles, on the other hand, are typically calculated using cosine similarity. The cosine value reflects the directional difference between two vectors; a cosine value closer to 1 indicates that the directions are similar, i.e., high similarity.

[0084] Among them, Euclidean distance is a standard method for measuring the straight-line distance between two points in multidimensional space, suitable for quantifying the distance between feature vectors. Cosine similarity is a metric for measuring the directional similarity between two vectors, based on the cosine of the angle between the two vectors, and is often used for comparing the similarity of text and speech signals.

[0085] Step S520: Determine the similarity between the first feature vector and the second feature vector based on the relative distance or relative angle.

[0086] The similarity between the first feature vector and the second feature vector is used to indicate the similarity between the first interactive audio data and the second interactive audio data.

[0087] Specifically, based on the calculated relative distance or angle, the smart device or cloud system determines the similarity between the first feature vector and the second feature vector. The similarity determination directly affects subsequent voice command processing.

[0088] For example, the system sets a similarity threshold. If the calculated relative distance is less than a certain value (for Euclidean distance) or the cosine value of the calculated relative angle is greater than a certain value (for cosine similarity), then the two feature vectors are considered similar; otherwise, they are considered dissimilar. This threshold setting takes into account the natural variations and noise effects of the speech signal to ensure accurate judgments in various environments.

[0089] In this embodiment, through feature extraction using a deep learning model, the intelligent device can convert audio into feature vectors, facilitating subsequent similarity calculations and improving the accuracy and speed of speaker verification. Calculating similarity scores using relative distance or angle provides a quantitative indicator, allowing the intelligent device or cloud system to determine whether the speakers are the same, thus enhancing the objectivity and operability of the verification. By setting a similarity threshold, the intelligent device can accurately determine whether audio data comes from the same speaker under various environmental conditions, effectively avoiding false recognition or rejection events and improving the user experience and security of voice interaction.

[0090] In one embodiment, such as Figure 6 As shown, the method further includes steps S600-S620, wherein:

[0091] Step S600: Obtain the preset test dataset and the preset range of similarity threshold values.

[0092] Specifically, this step is the initial stage of defining the criteria for judging speech similarity. The main task is to obtain test audio samples and the possible range of similarity thresholds from the preset dataset, which is the basis for determining the final similarity threshold.

[0093] For example, the test dataset should be prepared in advance, containing multiple sets of audio data. Each set consists of two audio clips and a Boolean value to indicate whether the two audio clips are from the same person. The Boolean value is assigned during data annotation to ensure that the dataset comprehensively covers various speech variations and environmental noise, thereby improving the model's generalization ability. The range of similarity threshold values ​​can be initially set based on past experience or statistical methods, providing a starting point for subsequent threshold optimization.

[0094] The test dataset includes multiple sets of test data. Each set of test data includes two audio data points and a corresponding Boolean value. The Boolean value is used to indicate whether the two audio data points in the same set come from the same object.

[0095] Boolean values ​​are binary logic values, typically used to indicate whether a condition is true (TRUE) or false (FALSE). In this scenario, they are used to mark whether two audio clips in the same group come from the same speaker. A similarity threshold is a critical value used to determine audio similarity; values ​​above or below this threshold will determine whether the audio clips are considered to come from the same speaker.

[0096] Step S610: Compare two audio data points belonging to the same group in the test dataset to obtain the similarity of each group of test data in the test dataset.

[0097] Specifically, after obtaining the test dataset, the next task is to calculate the similarity of each group of audio in the dataset and obtain the similarity score between them.

[0098] For example, a pre-trained deep learning model, such as a convolutional neural network (CNN) or a recurrent neural network (RNN), is used to extract features from the audio and convert them into feature vectors. Then, the distance or angle between two feature vectors is calculated to obtain a similarity score. Common calculation methods include Euclidean distance and cosine similarity. By calculating similarity, the degree of similarity between audio files in the test dataset can be quantified, providing a specific numerical reference for subsequent threshold selection.

[0099] Step S620: Determine the target similarity threshold within the range of similarity threshold values ​​based on the similarity and Boolean value corresponding to each group of test data in the test dataset.

[0100] The target similarity threshold is the similarity threshold that makes the misclassification rate of each group of test data in the test dataset less than the error threshold.

[0101] Specifically, based on the similarity score obtained in the previous step and the Boolean value indication of each group of audio, this step aims to find an optimal threshold within the preset similarity threshold range to ensure that the false positive rate is lower than the preset error threshold.

[0102] For example, the algorithm calculates the false positive rate (false rejection rate and false acceptance rate) under different similarity thresholds. The false positive rate is the rate at which a person is judged as "the same person" when they are not (false acceptance), or as "different people" when they are actually the same person (false rejection). The system continuously adjusts the thresholds to find the threshold that minimizes the overall false positive rate, or the threshold at which the rate of specific types of false positives (false rejection or false acceptance) is lower than a preset error threshold. Determining a suitable target similarity threshold allows the system to achieve an optimal balance between speech recognition and speaker verification, avoiding both excessive leniency leading to misoperation and excessive strictness causing frequent false rejections of user commands, thus improving system stability and user interaction experience.

[0103] In this embodiment, firstly, a test dataset containing real speaker information and a range of similarity threshold values ​​are obtained to ensure the scientific validity and rationality of subsequent steps. Next, a deep learning model is used to extract features and calculate similarity from the audio data in the test dataset, quantifying the differences or similarities between audio files. Finally, an optimization algorithm is used to determine a target similarity threshold that effectively controls the false positive rate, achieving the optimal balance between user command recognition and speaker verification.

[0104] In one embodiment, such as Figure 7 As shown, step S620 involves determining the target similarity threshold within the range of similarity threshold values ​​based on the similarity and Boolean values ​​corresponding to each group of test data in the test dataset. This includes steps S700-S750, where:

[0105] Step S700: Use the upper limit of the value range to judge the similarity of each group of test data in the test dataset to determine the first judgment result set.

[0106] Specifically, the intelligent voice system uses the upper limit of the similarity threshold range to determine the similarity of each set of audio data in the saved test dataset, generating a first set of judgment results. This set contains preliminary judgment results on whether each set of audio data comes from the same speaker.

[0107] For example, the system first sets a high similarity threshold as an upper limit. For each pair of audio data in the test dataset, it calculates the distance or angle between their feature vectors (such as Euclidean distance or cosine similarity) and compares the obtained similarity with the preset upper limit. If the similarity score is higher than the upper limit, it is initially determined that the two audio data come from the same object; otherwise, they are considered to come from different objects. This series of judgment results constitutes the first judgment result set.

[0108] And, in step S710, the similarity of each group of test data in the test dataset is judged by using the lower limit of the value range, so as to determine the second judgment result set.

[0109] Specifically, the same test dataset is used to determine similarity using the lower limit of the similarity threshold range, generating a second set of results. Unlike step S700, the threshold here is lower, meaning a wider range for similarity assessment.

[0110] For example, the system uses the lower end of the similarity threshold as a lower limit to calculate and judge the similarity of each set of audio data in the test dataset. If the similarity score is higher than the lower limit, it is initially determined that the audio data comes from the same object; if it is lower than the lower limit, it is determined that they are different objects. The preliminary judgment results obtained in this way constitute the second judgment result set.

[0111] The first set of judgment results and the second set of judgment results include the judgment results of whether two audio data in each group of test data in the test dataset come from the same object.

[0112] Step S720: Based on the Boolean values, the first judgment result set, and the second judgment result set corresponding to each group of test data in the test dataset, determine the first false positive rate corresponding to the upper limit and the second false positive rate corresponding to the lower limit.

[0113] Specifically, the system calculates the misjudgment rate when using the upper and lower limits based on the Boolean labels of the test dataset, the first judgment result set, and the second judgment result set.

[0114] For example, by comparing a preset Boolean value (i.e., a label indicating whether each set of audio data truly comes from the same speaker) with the results in the first and second judgment result sets, the first and second misjudgment rates can be calculated. The misjudgment rate refers to the proportion in which the system incorrectly identifies audio from different speakers as the same speaker, or identifies audio from the same speaker as different speakers.

[0115] Step S730: Determine that the difference between the first misclassification rate and the second misclassification rate is greater than or equal to a set threshold.

[0116] In step S740, if the difference between the first misclassification rate and the second misclassification rate is greater than or equal to a set threshold, the range of values ​​for the similarity threshold is updated, and the process returns to step S700. This continues until the difference between the first misclassification rate and the second misclassification rate is less than the set threshold.

[0117] Specifically, if the difference between the first misjudgment rate and the second misjudgment rate is greater than or equal to a set threshold, the system will update the range of similarity threshold values ​​and recalculate the similarity judgment and misjudgment rate until the difference is lower than the set threshold.

[0118] For example, if the difference between the first and second false positive rates exceeds a set threshold, the system adjusts the range of the similarity threshold, such as narrowing the gap between the upper and lower limits, and repeats the aforementioned steps until the difference meets the condition. This process ensures that a suitable similarity threshold is found to balance false rejections and false acceptances. By iteratively optimizing the similarity threshold, the system can find a threshold that achieves the optimal balance between false rejections and false acceptances, improving the accuracy and reliability of speaker verification.

[0119] Step S750: If the difference between the first misclassification rate and the second misclassification rate is less than a set threshold, then any similarity threshold in the current value range is used as the target similarity threshold.

[0120] Specifically, if the difference between the first misclassification rate and the second misclassification rate satisfies the set threshold, a threshold is selected from the current range of similarity threshold values ​​as the target similarity threshold.

[0121] For example, once the difference in the false positive rate meets the set threshold, the system can select any threshold within the current value range as the target similarity threshold for subsequent speaker verification.

[0122] For example, first, set the minimum and maximum similarity thresholds for calculation. The minimum value can be set to 0, and the maximum value can be set to 1. To prevent infinite iteration during calculation and avoid invalid calculations, set a termination condition for the iteration. The termination condition can be the difference between the first and second false positive rates, or the threshold precision. For example, if the threshold precision is set to 0.01, the iteration will terminate when the difference between the minimum value (0.44) and the maximum value (0.45) is less than or equal to the threshold precision.

[0123] The false recognition rate includes: False rejection rate: This refers to instances where the same person is incorrectly identified as a different person, resulting in a rejection. This can affect normal voice interaction. The calculation method is: number of false rejections / total number of interactions. 100%. False acceptance rate: This indicates instances where different people were mistakenly identified as the same person, resulting in normal interaction despite the message being rejected. This does not affect normal voice interaction, but it impacts the user experience. The calculation method is: number of false acceptances / total number of interactions. 100%.

[0124] The test dataset is run line by line on the model. First, a feature vector is calculated for each audio track, and then the similarity score between two audio tracks is calculated. If the similarity score is less than the similarity threshold, the audio track is rejected. Rejection is determined for both the minimum and maximum thresholds to obtain the false rejection rate and false acceptance rate for the entire test dataset. If the maximum threshold minus the minimum threshold is greater than the threshold precision, the iteration continues; otherwise, the calculation ends. For example, a maximum threshold of 0.5 corresponds to a false rejection rate of 0.8% and a false acceptance rate of 2%; a minimum threshold of 0.4 corresponds to a false rejection rate of 0.7% and a false acceptance rate of 3%. Based on the above logic, the result corresponding to the minimum threshold of 0.4 is better, so a new maximum threshold is set to (0.5 + 0.4) / = 0.45. A new round of calculation is then performed with a minimum threshold of 0.4 and a maximum threshold of 0.45. This process continues until the final result meets the set iteration termination condition, at which point the target similarity threshold is determined.

[0125] The above calculations can be performed on the models deployed on the device and the models deployed in the cloud respectively to obtain the first similarity threshold and the second similarity threshold.

[0126] In this embodiment, through iterative steps, a target similarity threshold is ultimately determined that can accurately distinguish audio data from different speakers in complex environments while effectively avoiding frequent misjudgments. This method not only improves the speaker verification performance of intelligent voice devices and ensures the accurate execution of voice commands, but also balances the two types of misjudgments—false rejection and false acceptance—by reducing the difference in misjudgment rates, thus finding the optimal compromise between protecting user privacy and providing a smooth voice interaction experience.

[0127] In one embodiment, such as Figure 8 As shown, the method further includes steps S800-S810, wherein:

[0128] In step S800, when the device is connected to the registered voiceprint library, the registered voiceprint library is used to verify whether the second interactive audio data and the first interactive audio data come from the same object, so as to determine the confidence level of the recognition result.

[0129] The registered voiceprint library includes registered voiceprints from multiple different individuals.

[0130] Specifically, when a smart device accesses a database containing registered voiceprints from multiple different speakers, the system uses this registered voiceprint database to verify the recognition results obtained through similarity comparisons to determine the confidence level of the results. This process is based on supplementary verification using a high-precision model in the cloud to enhance the accuracy of recognition.

[0131] For example, the system compares the second interactive audio data with the first interactive audio data using a registered voiceprint database, and verifies the recognition result using the voiceprint data in the database. If the recognition result indicates that the two audio recordings may come from the same speaker, the system searches for the voiceprint that best matches these two audio recordings in the registered voiceprint database, and confirms the reliability of the recognition result through further similarity calculation.

[0132] Step S810: If the confidence level of the recognition result meets the standard, add the first interactive audio data, the second interactive audio data, and the recognition result as a set of test data to the test dataset.

[0133] The recognition results are represented by corresponding Boolean values.

[0134] Specifically, if the verification results of the registered voiceprint database show that the confidence level of the recognition result meets the standard, that is, the result is reliable enough, the system will add the recognition case (first interactive audio data, second interactive audio data and recognition result) as a new test case to the test dataset for subsequent similarity threshold optimization.

[0135] For example, based on feedback from the registered voiceprint database, if the confidence level is higher than a preset threshold, the system converts the audio data and recognition result into a Boolean value representing a correctness label, which, along with the audio data, forms a new test data sample. This new sample is then added to the test dataset for future adjustments and optimizations to the similarity threshold.

[0136] Specifically, over time, through the steps described above, the test dataset continuously accumulates new examples. The system periodically analyzes the test dataset, recalculates the similarity threshold, and ensures that it adapts to changes in user behavior and fluctuations in environmental conditions.

[0137] For example, the system performs batch processing on the test dataset, analyzes the similarity score and correctness label of each sample, and finds an optimal similarity threshold through an iterative algorithm (such as binary search) to minimize the false positive rate (including false acceptance rate and false rejection rate). This process may require multiple iterations until the optimal solution is found or a predetermined computational limit is reached.

[0138] In this embodiment, by verifying the registered voiceprint database, the system can improve the accuracy of recognition results, especially when processing audio with similarity close to the threshold boundary. This avoids false positives, enhancing user experience and device responsiveness and security. Including recognition results that meet the confidence threshold in the test dataset not only enriches the dataset but also adds examples from real-world application scenarios. This facilitates model learning and optimization, improves the rationality of similarity threshold settings, and ultimately enhances speaker verification performance and user experience. As the test dataset becomes richer, similarity threshold optimization becomes more targeted, better addressing recognition challenges in various environments and improving verification accuracy and robustness. Through the accumulation of real-world cases with met confidence thresholds, threshold settings become more reasonable, effectively reducing false acceptance and false rejection events, ensuring device security and smooth user interaction.

[0139] In one embodiment, step S230 involves determining whether to provide an intelligent voice response to the recognition result of the second interactive audio data based on the judgment result regarding whether the second interactive audio data and the first interactive audio data originate from the same object. This includes:

[0140] If it is determined that the second interactive audio data and the first interactive audio data come from different objects, the second interactive audio data is ignored.

[0141] Specifically, when the system determines that the second interactive audio data and the first interactive audio data (usually wake-up audio) come from different speakers, a denial mechanism will be activated to prevent unauthorized users' voice commands from being executed by the device. Ignoring unauthorized second interactive audio data can prevent the device from responding to irrelevant or potentially malicious commands, thus protecting the device's operational security and the user's environment.

[0142] For example, if the similarity score calculated by the algorithm is lower than a set threshold, the system will automatically determine that the audio data comes from different objects, thereby triggering a rejection mechanism. A rejection mechanism is a security measure in smart devices or systems used to identify and block the processing of unexpected or unauthorized voice commands. The similarity score is a quantified value calculated by comparing feature vectors, reflecting the degree of similarity between two audio files.

[0143] Ignoring the second interactive audio data includes refusing to respond to it and / or refusing to respond to the operational instructions indicated by it. Once a refusal is determined, the system will not process the second interactive audio data in any way; that is, it will not recognize the audio content or execute any operational instructions it may carry. The system can simply discard the audio data without further parsing its content or attempting to understand the operational instructions, ensuring that unauthorized instructions are not mistakenly executed by the device.

[0144] If it is determined that the second interactive audio data and the first interactive audio data come from the same object, the second interactive audio data is parsed to determine semantic content; intelligent voice response is given in response to the semantic content, and / or the operation instructions indicated by the semantic content are executed.

[0145] Specifically, when the system determines that the second interactive audio data and the first interactive audio data (usually wake-up audio) come from the same speaker, it will perform the normal voice command parsing and response process.

[0146] For example, if the similarity score is higher than a set threshold, the system assumes that the audio data comes from the same object, and then parses and processes the voice commands of the second interactive audio data.

[0147] Specifically, the system performs semantic analysis on the second interactive audio data determined to be from the same object, understands the user's intent, and executes corresponding operation instructions or provides information feedback.

[0148] For example, a device or cloud server converts audio data into text, performs semantic understanding, determines the user's specific needs, and performs corresponding operations, such as adjusting device settings, searching for information, or launching a program.

[0149] In this embodiment, the activation of the denial mechanism can quickly identify and block unauthorized voice command processing, protecting user privacy and device security. When the system confirms that the audio data originates from the same source, the normal response process ensures that the device can accurately understand and execute consecutive commands from legitimate users, providing a seamless and personalized voice interaction experience and improving the usability and user satisfaction of smart devices.

[0150] In one embodiment, the method further includes: extracting emotional feature vectors from the audio data.

[0151] Specifically, a deep learning-based speech emotion recognition model is used to extract emotion feature vectors from audio data. These emotion feature vectors are then combined with voiceprint feature vectors to create a more comprehensive user speech representation. When calculating similarity, both voiceprint and emotion features are considered simultaneously, and a weighted average or other fusion method is used to obtain a comprehensive similarity score.

[0152] It's important to note that while existing speaker verification technologies can distinguish between different individuals' voices, they often overlook the emotional information inherent in speech. Emotional information is a crucial component of personal characteristics, and speech features change under different emotional states, especially in speech recognition and speaker verification. This can affect the accuracy of verification and the personalized experience. For example, a user's speech when happy, sad, or angry may differ significantly from their speech when calm. These differences can mislead the model, leading to misidentification or rejection.

[0153] Determine the preset threshold for emotional similarity.

[0154] Specifically, the model analyzes the emotional distribution in users' historical voice data to identify changes in users' voices under different emotional states. Based on the distribution of users' emotional features, the similarity threshold for each user is adjusted to adapt to their voice characteristics during emotional changes. A basic similarity threshold is set, and then fine-tuned using personalized emotional thresholds based on the user's current emotional state to ensure accurate recognition of voices under different emotional states. When calculating audio similarity, emotional features are integrated, considering not only the physical properties of the sound but also the emotional information conveyed, improving the model's ability to recognize specific user voice features. Personalized emotional thresholds are set for each registered user, and the similarity threshold is adjusted based on the emotional state in the user's historical voice data to ensure accurate speaker identification even when the user's emotions fluctuate.

[0155] We use speech data under different emotional states to train and optimize the emotion recognition model.

[0156] Specifically, we collect user speech data under various emotional states to build an emotionally rich audio database. We then use this data to train an emotion recognition model, such as using a Long Short-Term Memory (LSTM) network or an emotion-aware attention mechanism model. Over time, we continuously collect new user speech data to continuously optimize the emotion recognition model and improve its accuracy in recognizing user emotional states. Finally, we use users' historical speech data, including speech samples from different emotional states, to train an emotion recognition model to identify the user's current emotional state.

[0157] When a user interacts with their voice, an emotion recognition model is run in real time to determine the user's emotional state.

[0158] Specifically, based on the emotion recognition results, the similarity threshold is dynamically adjusted to ensure accurate speech recognition even when the user's emotions fluctuate. A rapid response mechanism is established so that the system immediately updates the similarity threshold when the emotional state changes significantly, in order to avoid potential misidentification.

[0159] In this embodiment, by fusing emotional features, the system can accurately identify users even when their emotions fluctuate, reducing false recognition rates and improving user experience. Personalized emotion threshold settings allow the system to adapt to each user's unique emotional expression, providing more personalized voice interaction services. Real-time emotion monitoring and dynamic adjustment strategies enable the system to quickly respond to changes in the user's emotional state, ensuring stable and effective speaker verification under various circumstances. Continuously collecting and analyzing voice data under the user's emotional state allows for continuous optimization of the emotion recognition model and similarity threshold settings, enabling the system to self-learn and evolve, thus improving long-term performance.

[0160] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0161] Figure 9 This is a structural block diagram of a voice interaction recognition device according to an embodiment of the present invention. Figure 9 As shown, it includes:

[0162] The audio acquisition module 901 is used to acquire the first interactive audio data received by the device after the device is woken up.

[0163] The audio comparison module 902 is used to compare the similarity between the first interactive audio data and the second interactive audio data when the device receives the second interactive audio data after this wake-up, which is not the first time.

[0164] The judgment module 903 is used to determine whether the second interactive audio data and the first interactive audio data come from the same object based on similarity.

[0165] The recognition module 904 is used to determine whether to provide an intelligent voice response to the recognition result of the second interactive audio data based on the judgment result that the second interactive audio data and the first interactive audio data come from the same object.

[0166] In one exemplary embodiment, the device is further configured to: compare a first similarity between first interactive audio data and second interactive audio data with a first similarity threshold, wherein the first similarity is obtained by comparing the first interactive audio data and second interactive audio data using a first speech recognition model deployed locally on the device. If the first similarity is greater than or equal to the first similarity threshold, the device uploads the first interactive audio data and second interactive audio data to the cloud. A second speech recognition model deployed in the cloud is used to compare the first interactive audio data and second interactive audio data to determine a second similarity between the first interactive audio data and second interactive audio data. If the second similarity is greater than or equal to the second similarity threshold, it is determined that the second interactive audio data and the first interactive audio data originate from the same object, wherein the accuracy of the second speech recognition model deployed in the cloud is higher than the accuracy of the first speech recognition model deployed locally on the device.

[0167] In one exemplary embodiment, the apparatus is further configured to: determine that the second interactive audio data and the first interactive audio data originate from different objects if the first similarity is less than a first similarity threshold; and determine that the second interactive audio data and the first interactive audio data originate from different objects if the second similarity is less than a second similarity threshold.

[0168] In one exemplary embodiment, the apparatus is further configured to: extract features from the first interactive audio data and the second interactive audio data respectively, to obtain a first feature vector of the first interactive audio data and a second feature vector of the second interactive audio data; calculate the relative distance or relative angle between the first feature vector and the second feature vector; and determine the similarity between the first feature vector and the second feature vector based on the relative distance or relative angle, wherein the similarity between the first feature vector and the second feature vector is used to indicate the similarity between the first interactive audio data and the second interactive audio data.

[0169] In an exemplary embodiment, the device is further configured to: acquire a preset test dataset and a preset range of similarity threshold values, wherein the test dataset includes multiple sets of test data, each set of test data includes two audio data points and a corresponding Boolean value, the Boolean value indicating whether the two audio data points in the same set originate from the same object; compare the two audio data points belonging to the same set in the test dataset to obtain the similarity corresponding to each set of test data in the test dataset; and determine a target similarity threshold within the range of similarity threshold values ​​based on the similarity and Boolean value corresponding to each set of test data in the test dataset, wherein the target similarity threshold is a similarity threshold that makes the misclassification rate of each set of test data in the test dataset less than an error threshold.

[0170] In an exemplary embodiment, the apparatus is further configured to: determine a first set of judgment results by using an upper limit value of a range to judge the similarity of each set of test data in the test dataset; and determine a second set of judgment results by using a lower limit value of a range to judge the similarity of each set of test data in the test dataset, wherein the first set of judgment results and the second set of judgment results include judgment results on whether two audio data points in each set of test data in the test dataset originate from the same object. Based on the Boolean values ​​corresponding to each set of test data in the test dataset, the first set of judgment results, and the second set of judgment results, a first false positive rate corresponding to the upper limit value and a second false positive rate corresponding to the lower limit value are determined. If the difference between the first false positive rate and the second false positive rate is greater than or equal to a set threshold, the range of the similarity threshold is updated, and the process returns to the step of judging the similarity of each set of test data in the test dataset using the upper limit value of the range to determine the first set of judgment results, until the difference between the first false positive rate and the second false positive rate is less than the set threshold. If the difference between the first misclassification rate and the second misclassification rate is less than a set threshold, any similarity threshold in the current value range shall be used as the target similarity threshold.

[0171] In one exemplary embodiment, the apparatus is further configured to: when the device accesses a registered voiceprint library, verify the recognition result of whether the second interactive audio data and the first interactive audio data originate from the same object using the registered voiceprint library, thereby determining the confidence level of the recognition result, wherein the registered voiceprint library includes registered voiceprints of multiple different objects. If the confidence level of the recognition result meets the standard, the first interactive audio data, the second interactive audio data, and the recognition result are added to a test dataset as a set of test data, wherein the recognition result is represented by a corresponding Boolean value.

[0172] In one exemplary embodiment, the device is further configured to: ignore the second interactive audio data if it is determined that the second interactive audio data and the first interactive audio data originate from different objects, wherein ignoring the second interactive audio data includes refusing to respond to the second interactive audio data and / or refusing to respond to the operation instruction indicated by the second interactive audio data. If it is determined that the second interactive audio data and the first interactive audio data originate from the same object, parse the second interactive audio data to determine semantic content; provide an intelligent voice response in response to the semantic content, and / or execute the operation instruction indicated by the semantic content.

[0173] Embodiments of the present invention also provide a storage medium comprising a stored program, wherein the program, when executed, performs any of the methods described above.

[0174] Optionally, in this embodiment, the storage medium may be configured to store program code for performing the following steps:

[0175] S1, when the device is woken up, acquire the first interactive audio data received by the device after this wake-up.

[0176] S2, if the device receives second interactive audio data that is not the first time after this wake-up, compare the similarity between the first interactive audio data and the second interactive audio data.

[0177] S3. Based on the similarity, determine whether the second interactive audio data and the first interactive audio data come from the same object.

[0178] S4. Based on the judgment result of whether the second interactive audio data and the first interactive audio data come from the same object, determine whether to provide an intelligent voice response to the recognition result of the second interactive audio data.

[0179] Embodiments of the present invention also provide an electronic device including a memory and a processor, the memory storing a computer program and the processor being configured to run the computer program to perform the steps in any of the above method embodiments.

[0180] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0181] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0182] S1, when the device is woken up, acquire the first interactive audio data received by the device after this wake-up.

[0183] S2, if the device receives second interactive audio data that is not the first time after this wake-up, compare the similarity between the first interactive audio data and the second interactive audio data.

[0184] S3. Based on the similarity, determine whether the second interactive audio data and the first interactive audio data come from the same object.

[0185] S4. Based on the judgment result of whether the second interactive audio data and the first interactive audio data come from the same object, determine whether to provide an intelligent voice response to the recognition result of the second interactive audio data.

[0186] Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0187] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0188] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0189] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A voice interaction recognition method, characterized in that, include: When the device is woken up, acquire the first interactive audio data received by the device after this wake-up; If the device receives second interactive audio data that is not the first time it has been activated after this wake-up, compare the similarity between the first interactive audio data and the second interactive audio data. Based on the similarity, determine whether the second interactive audio data and the first interactive audio data come from the same object; Based on the determination result of whether the second interactive audio data and the first interactive audio data come from the same object, it is determined whether to provide an intelligent voice response to the recognition result of the second interactive audio data.

2. The voice interaction recognition method according to claim 1, characterized in that, The step of determining whether the second interactive audio data and the first interactive audio data come from the same object based on the similarity includes: The first similarity between the first interactive audio data and the second interactive audio data is compared with a first similarity threshold, wherein the first similarity is obtained by comparing the first interactive audio data and the second interactive audio data through a first speech recognition model deployed locally on the device; If the first similarity is greater than or equal to the first similarity threshold, the first interactive audio data and the second interactive audio data are uploaded to the cloud via the device; The first interactive audio data and the second interactive audio data are compared using the second speech recognition model deployed in the cloud to determine the second similarity between the first interactive audio data and the second interactive audio data; If the second similarity is greater than or equal to the second similarity threshold, it is determined that the second interactive audio data and the first interactive audio data come from the same object, wherein the accuracy of the second speech recognition model deployed in the cloud is higher than the accuracy of the first speech recognition model deployed locally on the device.

3. The voice interaction recognition method according to claim 2, characterized in that, The method further includes: If the first similarity is less than the first similarity threshold, it is determined that the second interactive audio data and the first interactive audio data come from different objects; If the second similarity is less than the second similarity threshold, it is determined that the second interactive audio data and the first interactive audio data come from different objects.

4. The voice interaction recognition method according to any one of claims 1-3, characterized in that, The comparison of the similarity between the first interactive audio data and the second interactive audio data includes: Feature extraction is performed on the first interactive audio data and the second interactive audio data respectively to obtain the first feature vector of the first interactive audio data and the second feature vector of the second interactive audio data. Calculate the relative distance or relative angle between the first feature vector and the second feature vector; The similarity between the first feature vector and the second feature vector is determined based on the relative distance or the relative angle, wherein the similarity between the first feature vector and the second feature vector is used to indicate the similarity between the first interactive audio data and the second interactive audio data.

5. The voice interaction recognition method according to any one of claims 1-3, characterized in that, The method further includes: Obtain a preset test dataset and a preset similarity threshold range. The test dataset includes multiple sets of test data. Each set of test data includes two audio data points and a corresponding Boolean value. The Boolean value is used to indicate whether the two audio data points in the same set come from the same object. Compare two audio data points belonging to the same group in the test dataset to obtain the similarity of each group of test data in the test dataset. Based on the similarity and Boolean value corresponding to each group of test data in the test dataset, a target similarity threshold is determined within the range of the similarity threshold, wherein the target similarity threshold is a similarity threshold that makes the misclassification rate of each group of test data in the test dataset less than the error threshold.

6. The voice interaction recognition method according to claim 5, characterized in that, The step of determining the target similarity threshold within the range of the similarity threshold based on the similarity and Boolean value corresponding to each group of test data in the test dataset includes: The upper limit of the value range is used to judge the similarity of each group of test data in the test dataset to determine the first judgment result set; Furthermore, the lower limit of the value range is used to judge the similarity of each group of test data in the test dataset to determine the second judgment result set, wherein the first judgment result set and the second judgment result set include the judgment results of whether two audio data in each group of test data in the test dataset come from the same object; Based on the Boolean values ​​corresponding to each group of test data in the test dataset, the first judgment result set, and the second judgment result set, determine the first false positive rate corresponding to the upper limit value and the second false positive rate corresponding to the lower limit value. If the difference between the first misjudgment rate and the second misjudgment rate is greater than or equal to a set threshold, update the range of the similarity threshold and return to the step of using the upper limit of the range to judge the similarity of each group of test data in the test dataset to determine the first judgment result set, until the difference between the first misjudgment rate and the second misjudgment rate is less than the set threshold. If the difference between the first misclassification rate and the second misclassification rate is less than the set threshold, any similarity threshold in the current value range shall be used as the target similarity threshold.

7. The voice interaction recognition method according to claim 5, characterized in that, The method further includes: When the device is connected to the registered voiceprint library, the registered voiceprint library is used to verify whether the second interactive audio data and the first interactive audio data come from the same object, so as to determine the confidence level of the recognition result. The registered voiceprint library includes registered voiceprints of multiple different objects. If the confidence level of the recognition result meets the standard, the first interactive audio data, the second interactive audio data, and the recognition result are added to the test dataset as a set of test data, wherein the recognition result is represented by the corresponding Boolean value.

8. The voice interaction recognition method according to any one of claims 1-3, characterized in that, The step of determining whether to provide an intelligent voice response to the recognition result of the second interactive audio data based on the judgment result of whether the second interactive audio data and the first interactive audio data come from the same object includes: If it is determined that the second interactive audio data and the first interactive audio data come from different objects, the second interactive audio data is ignored, wherein ignoring the second interactive audio data includes refusing to respond to the second interactive audio data and / or refusing to respond to the operation instructions indicated by the second interactive audio data.

9. The voice interaction recognition method according to any one of claims 1-3, characterized in that, The step of determining whether to provide an intelligent voice response to the recognition result of the second interactive audio data based on the judgment result of whether the second interactive audio data and the first interactive audio data come from the same object includes: If it is determined that the second interactive audio data and the first interactive audio data come from the same object, the second interactive audio data is parsed to determine the semantic content; The system responds to the semantic content with intelligent voice and / or executes the operational instructions indicated by the semantic content.

10. A voice interaction recognition device, characterized in that, include: The audio acquisition module is used to acquire the first interactive audio data received by the device after the current wake-up when the device is woken up. An audio comparison module is used to compare the similarity between the first interactive audio data and the second interactive audio data when the device receives second interactive audio data that is not the first time after this wake-up. The judgment module is used to determine whether the second interactive audio data and the first interactive audio data come from the same object based on the similarity. The recognition module is used to determine whether to provide an intelligent voice response to the recognition result of the second interactive audio data based on the judgment result of whether the second interactive audio data and the first interactive audio data come from the same object.

Images