A speech recognition technique based on spatiotemporal clustering information inference

Abstract

The application provides a speech recognition technology based on spatiotemporal clustering information inference to solve the problem of high speech recognition misrecognition rate under special conditions. The speech recognition technology comprises the following processing stages: a data segmentation stage, in which signals are acquired by a sound acquisition sensor, transmitted and stored on a local or cloud server, speech recognition and segmentation are performed, and data are stored in a data structure of "signal (word level)-recognition result (word level)-position-time"; a data clustering stage, in which the saved structural data are clustered in two forms: first-class clustering, in which clustering is performed according to time intervals in the "time" dimension, and second-class clustering, in which clustering is performed according to spatial distances in the "space" dimension; and a speech recognition stage, in which two recognition strategies are adopted according to whether positioning information can be acquired, speech recognition is performed under the condition that positioning is not started by using a first-class clustering result to assist a decision function, and speech recognition is performed under the condition that positioning is started by using a second-class clustering result to assist the decision function.

Description

Technical Field

[0001] This invention relates to the field of computer speech recognition, and in particular to a speech recognition technology based on spatiotemporal clustering information inference. Background Technology

[0002] The accuracy of speech recognition has always been limited by noise interference and pronunciation accuracy. The basic process of speech recognition is as follows: the audio to be recognized is acquired through the mobile device's microphone, uploaded to the cloud, processed in the cloud to obtain the recognized text, and then sent from the cloud to the mobile device.

[0003] However, under special conditions, such as when the collected audio contains strong noise or the pronunciation is unclear / inaccurate, misrecognition or failure to recognize may occur. In these situations, more effective speech recognition technology is needed. Summary of the Invention

[0004] This invention provides a speech recognition technology based on spatiotemporal clustering information inference to solve the problem of high misrecognition rate in speech recognition under special conditions.

[0005] This invention provides a speech recognition technology based on spatiotemporal clustering information inference, comprising the following three stages.

[0006] The first stage is the data segmentation stage. Audio signals are collected through a microphone, transmitted and stored on a local or cloud server, and then the speech is segmented. The speech signal and its recognition result are segmented into words and a corresponding relationship is established. Finally, the data is saved according to the data structure of "signal (word level) - recognition result (word level) - location - time". If the location information is missing, it is stored as "0".

[0007] The second stage, the data clustering stage, involves two forms of clustering on the saved structured data. The first clustering is performed along the "time" dimension, where each speech record is considered a data point. The DBSCAN clustering algorithm is used in the time dimension, with a radius e of 1 hour (meaning the maximum time interval between two speech records in the same cluster is 1 hour) and a minimum number of records (MinPts) of 1 (meaning a cluster contains at least one speech record). For each cluster, all location information is extracted; data with location information of "0" are not included in the clustering. Clustering is then performed based on spatial distance using the DBSCAN algorithm, with a radius e of 2 minutes. The first clustering method uses GPS coordinates, meaning the maximum distance between two voices in the same cluster is ~3.7km, and the minimum number of MinPts is 1, meaning each cluster contains at least one location. After obtaining the clustering results, the number of data entries in all clusters is calculated to obtain the sequence L of the proportion of data entries in each cluster to the total data. The second clustering method clusters based on spatial distance, excluding data with location information of "0". The DBSCAN clustering algorithm is used, with a radius e of 2 (GPS coordinates), meaning the maximum distance between two voices in the same cluster is ~3.7km, and the minimum number of MinPts is 1, meaning each cluster contains at least one location.

[0008] The third stage is the speech recognition stage, which has two scenarios:

[0009] Scenario 1: Speech recognition without location services enabled. In this case, firstly, based on the current time, find the corresponding cluster (i.e., a cluster from the second-stage clustering results) and obtain the sequence L representing the percentage of data in each category. When max(L) > 75%, use the DTW algorithm to calculate the distance between the word signal in the largest category and the word signal in the currently recognized signal. Take the reciprocal of this distance as a score and add it to the recognition decision function.

[0010] S word =S online +S distance (1)

[0011] Among them, S word The decision score for its final identification result, S online S represents the decision score given by the cloud for the current word recognition result. If the cloud does not recognize this word, then S... online S is 0 online The distance between the word signal in the largest category and the word signal of the current recognition signal is the reciprocal. When max(L) <= 75%, the DTW algorithm is used to calculate the distance between all word signals in the cluster and the word signal of the current recognition signal, and the reciprocal of the distance is taken as the score and added to the recognition decision function, i.e., formula (1).

[0012] Scenario 2: Speech recognition is performed under the condition of location enabled. First, based on the location information, the DTW algorithm is used to calculate the distance between the word signal in the category with the largest number of samples and the word signal of the current recognition signal. The reciprocal of the distance is taken as the score and added to the recognition decision function, i.e., formula (1);

[0013] Once the user confirms the recognition result, both the input text and voice will be uploaded to the cloud for storage, and the data structure will be updated in the cloud as "signal (word level) - recognition result (word level) - location - time". Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a flowchart of a speech recognition technology based on spatiotemporal clustering information inference provided by an embodiment of the present invention. Detailed Implementation

[0016] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of the invention. However, those skilled in the art will understand that the invention can be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of the invention with unnecessary detail.

[0017] To illustrate the technical solution of the present invention, specific embodiments are described below.

[0018] Example 1

[0019] like Figure 1 As shown, this embodiment provides a speech recognition technology based on spatiotemporal clustering information inference, which includes the following three stages:

[0020] The first stage is the data segmentation stage. Audio signals are collected through a microphone, transmitted and stored on a local or cloud server, and then the speech is segmented. The speech signal and its recognition result are segmented into words and a corresponding relationship is established. Finally, the data is saved according to the data structure of "signal (word level) - recognition result (word level) - location - time". If the location information is missing, it is stored as "0".

[0021] The second stage, the data clustering stage, involves two forms of clustering on the saved structured data. The first clustering is performed along the "time" dimension, where each speech record is considered a data point. The DBSCAN clustering algorithm is used in the time dimension, with a radius e of 1 hour (meaning the maximum time interval between two speech records in the same cluster is 1 hour) and a minimum number of records (MinPts) of 1 (meaning a cluster contains at least one speech record). For each cluster, all location information is extracted; data with location information of "0" are not included in the clustering. Clustering is then performed based on spatial distance using the DBSCAN algorithm, with a radius e of 2 minutes. The first clustering method uses GPS coordinates, meaning the maximum distance between two voices in the same cluster is ~3.7km, and the minimum number of MinPts is 1, meaning each cluster contains at least one location. After obtaining the clustering results, the number of data entries in all clusters is calculated to obtain the sequence L of the proportion of data entries in each cluster to the total data. The second clustering method clusters based on spatial distance, excluding data with location information of "0". The DBSCAN clustering algorithm is used, with a radius e of 2 (GPS coordinates), meaning the maximum distance between two voices in the same cluster is ~3.7km, and the minimum number of MinPts is 1, meaning each cluster contains at least one location.

[0022] The third stage is the speech recognition stage, which has two scenarios:

[0023] Scenario 1: Speech recognition without location services enabled. In this case, firstly, based on the current time, find the corresponding cluster (i.e., a cluster from the second-stage clustering results) and obtain the sequence L representing the percentage of data in each category. When max(L) > 75%, use the DTW algorithm to calculate the distance between the word signal in the largest category and the word signal in the currently recognized signal. Take the reciprocal of this distance as a score and add it to the recognition decision function.

[0024] S word =S online +S distance (2)

[0025] Among them, S word The decision score for its final identification result, S online S represents the decision score given by the cloud for the current word recognition result. If the cloud does not recognize this word, then S... online S is 0 online The distance between the word signal in the largest category and the word signal of the current recognition signal is the reciprocal. When max(L) <= 75%, the DTW algorithm is used to calculate the distance between all word signals in the cluster and the word signal of the current recognition signal, and the reciprocal of the distance is taken as the score and added to the recognition decision function, i.e., formula (2).

[0026] Scenario 2: Speech recognition is performed under the condition of location enabled. First, based on the location information, the DTW algorithm is used to calculate the distance between the word signal in the category with the largest number of samples and the word signal of the current recognition signal. The reciprocal of the distance is taken as the score and added to the recognition decision function, i.e., formula (2);

[0027] Once the user confirms the recognition result, both the input text and voice will be uploaded to the cloud for storage, and the data structure will be updated in the cloud as "signal (word level) - recognition result (word level) - location - time".

Claims

1. A speech recognition technology based on spatiotemporal clustering information inference, characterized in that, The processing includes the following stages: Data segmentation stage, where signals are acquired through sound acquisition sensors, transmitted and stored on local or cloud servers, and speech recognition and segmentation are performed. The data is stored in a data structure of "signal (word level) - recognition result (word level) - location - time". In the data clustering stage, the saved structured data is clustered in two ways: the first type of clustering is based on time intervals in the "time" dimension, and the second type of clustering is based on spatial distance in the "space" dimension. In the speech recognition stage, depending on whether location information is available, there are two recognition strategies: when location is not enabled, the first type of clustering results are used to assist the decision function for speech recognition; when location is enabled, the second type of clustering results are used to assist the decision function for speech recognition.

2. The speech recognition technology based on spatiotemporal clustering information inference as described in claim 1, characterized in that, The first and second clustering in the data clustering stage are specifically as follows: The first type of clustering: Clustering is performed along the "time" dimension. Each speech record is a data sample. For all samples, the DBSCAN clustering algorithm is used in the time dimension, with a radius e of 1 hour (meaning the maximum time interval between two speech records in the same cluster is 1 hour) and a minimum number of MinPts of 1 (meaning a cluster contains at least one speech record). For each cluster, all location information is extracted and clustered according to spatial distance. Data with location information of "0" are not included in the clustering. The DBSCAN clustering algorithm is used, with a radius e of 2 minutes (GPS coordinates), meaning the maximum distance between two speech records in the same cluster is ~3.7 km and a minimum number of MinPts of 1 (meaning a cluster contains at least one location). After obtaining the clustering results, the number of data samples contained in each cluster is calculated to obtain the sequence L, which is the percentage of data in each category relative to the total data. The second type of clustering: clustering is based on spatial distance. Data with location information of "0" does not participate in clustering. The DBSCAN clustering algorithm is used, with a radius e of 2 minutes (GPS coordinates). This means that the maximum distance between two voices in the same cluster is ~3.7km, and the minimum number of MinPts is 1, meaning that a cluster contains at least one location.

3. The speech recognition technology based on spatiotemporal clustering information inference as described in claim 1, characterized in that, The specific steps for speech recognition using the first type of clustering results to assist the decision function when positioning is not enabled are as follows: First, based on the current time, find the corresponding cluster, which is a cluster in the clustering results of the second stage, and obtain the sequence L of the proportion of data contained in each category to the total data. When max(L) > 75%, the current cluster is the largest category. Using the DTW algorithm, calculate the distance between the word signal in the largest category and the word signal in the currently recognized signal, and take the reciprocal of the distance as the score and add it to the recognition decision function, that is: S word =S online +S distance , Among them, S word The decision score for its final identification result, S online S represents the decision score given by the cloud for the current word recognition result. If the cloud does not recognize this word, then S... online S is 0 distance The distance between the word signal in the largest category and the word signal of the currently recognized signal is the reciprocal. When max(L) <= 75%, the DTW algorithm is used to calculate the distance between all word signals in the cluster and the word signals of the currently recognized signal, and the reciprocal of the distance is taken as the score and added to the recognition decision function.

4. The speech recognition technology based on spatiotemporal clustering information inference as described in claim 1, characterized in that, The specific steps for using the second type of clustering results to assist speech recognition under the condition of location activation are as follows: When performing speech recognition with location services enabled, the first step is to use the DTW algorithm, based on location information, to calculate the distance between the word signal in the category with the largest number of samples and the word signal in the currently recognized signal. The reciprocal of this distance is then used as a score and added to the recognition decision function, i.e., formula S. word =S online +S distance , of which S word The decision score for its final identification result, S online S represents the decision score given by the cloud for the current word recognition result. If the cloud does not recognize this word, then S... online S is 0 distance It is the reciprocal of the distance between the word signal in the largest category and the word signal of the currently recognized signal.

Images