Fast intelligent voice assistant response method based on approximate nearest neighbor retrieval algorithm

Abstract

The invention relates to the technical field of smart home, in particular to a quick intelligent voice assistant response method based on an approximate nearest neighbor retrieval algorithm. The method comprises the following steps that historical voice instruction information of a user is collected to construct a cloud database, voice feature extraction is carried out on data in the cloud database, and the voice feature extraction is stored in a cloud vector database; carrying out clustering layering on the voice feature vectors in the cloud vector database, and introducing a multi-stage balance degree adjustment strategy to carry out balance degree adjustment; the clustering center is managed by using an IVF-HNSW framework; migrating data in the voice feature extraction model, extracting a user voice instruction feature vector and historical voice instruction information of the user as auxiliary information, and transmitting the auxiliary information to the cloud; and querying the user voice instruction feature vector through the IVF-HNSW index structure to obtain a key value of a matched voice feature, and returning the instruction to the local intelligent voice terminal. According to the method, efficient index construction and approximate nearest neighbor retrieval can be realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of smart home, and in particular to a fast intelligent voice assistant response method based on an approximate nearest neighbor search algorithm. BACKGROUND

[0002] With the improvement of living standards, more and more families use intelligent voice assistants to control smart home appliances, and the response speed of the intelligent voice assistant and the accuracy of the smart home appliance control are important links affecting user experience. Due to differences in voiceprint information such as user speaking style, accent, and environmental noise, the same operation instruction may produce different voice data in the use process of the intelligent terminal. Therefore, in order to recognize the voice operation instruction of a specific user for the smart home appliance, a large-scale database needs to be maintained, which can be used to train a voice feature extraction model on the one hand, and can more accurately match specific voice information with high similarity in the database on the other hand. The two aspects complement each other to improve the voice assistant's ability to recognize sound signals. In addition, with the increase in smart home appliances, the same instruction may control multiple intelligent terminals. How to effectively and quickly feed back the operation instruction and realize the quasi-coordinated operation of multiple terminals is a major challenge faced by intelligent voice assistants.

[0003] Approximate Nearest Neighbor Search (ANNS) is a vector similarity calculation method that has emerged in recent years. This method greatly improves the retrieval efficiency while ensuring high retrieval accuracy, and has attracted widespread attention from researchers. According to the difference in index structure, the existing ANNS algorithm can be roughly divided into: IVF-Based, Graph-Based and Tree-Based. Compared with other methods, IVF-Based can achieve efficient compression of large-scale data sets, and under the premise of ensuring retrieval accuracy, the index structure is loaded into limited high-speed memory at one time, thereby improving retrieval efficiency. The construction process of IVF-Based index is first to use a clustering algorithm to divide the data set into clusters and generate the corresponding cluster centers. The resulting cluster centers are used as index heads to construct an inverted table structure. Then, a quantization algorithm is used to compress the vectors in the data set into short codes, and the short codes are added to the corresponding index heads to form an inverted index. In the retrieval process, the query vector will first locate to the nearest index head, then query the vectors associated with the index head, and get the final query result. This method narrows down the query range and avoids global vector queries, thereby improving query efficiency. However, the core clustering algorithm of the inverted index-based method faces the following two challenges when facing large-scale retrieval tasks: Clustering time affects index building efficiency: While ensuring retrieval accuracy, how to reduce the time for selecting the initial cluster centers and the time complexity of the clustering process to improve index building efficiency is very selective; Clustering balance affects retrieval efficiency: How to adjust the clustering process to achieve relatively balanced clustering results and improve retrieval efficiency is also challenging.

[0004] Therefore, the present invention provides a fast intelligent voice assistant response method based on an approximate nearest neighbor retrieval algorithm to solve the above problems. Summary of the Invention

[0005] In response to the shortcomings of the existing technology, the present invention develops a fast intelligent voice assistant response method based on the approximate nearest neighbor retrieval algorithm. The main purpose is to apply the approximate nearest neighbor method based on inverted index to the intelligent voice assistant and improve the core clustering algorithm for large-scale data.

[0006] The technical solution to the technical problem solved by the present invention is a fast intelligent voice assistant response method based on an approximate nearest neighbor retrieval algorithm, which is as follows: S1. Collect the user's historical voice command information to build a cloud database, then input the voice commands in the cloud database into the voice feature extraction model for voice feature extraction. The voice feature is in the form of <voice feature vector, command pair>, and then save the voice feature vector to the cloud vector database. ; S2. Cluster the speech feature vectors in the cloud vector database and calculate the cluster center set for each layer. Then, select the initial cluster centers for each cluster layer, and make the number of initial cluster centers the same as the number of cluster centers in the cluster center set. Then, introduce a multi-level balance adjustment strategy to adjust the balance. The multi-level balance adjustment strategy includes an intra-layer balance adjustment strategy and an inter-layer balance adjustment strategy. S3, using the IVF-HNSW framework to manage the speech feature vectors and the calculated cluster centers; S4. Migrate the data in the voice feature extraction model, extract the user's voice command feature vector, and transmit it to the cloud together with the user's historical voice command information as auxiliary information; S5. After the cloud receives the user query vector and voice information, it queries the user's voice command feature vector through the IVF-HNSW index structure, queries and outputs similar voice vectors stored in the cloud database, matches the vector number in the query result to the corresponding <voice feature vector, command> key value, and returns the command to the local intelligent voice terminal.

[0007] S1 is as follows: The historical voice instruction information in the cloud database is input into a voice feature extraction module to obtain voice features, and then the voice features are saved to a cloud vector database, and the voice features are represented as , , represents any one of the historical voice instruction information in the cloud database, represents the voice feature vector obtained according to the historical voice instruction information , represents the instruction pair obtained according to the historical voice instruction information , and respectively represent the object of the instruction action and the instruction required action obtained according to the historical voice instruction information , , represents the first voice feature vector in the cloud vector database, , , .

[0008] S2 is specifically as follows: S2.1, clustering and layering the voice feature vector: The voice feature vectors in the cloud vector database are subjected to hierarchical clustering of layers, the number of hierarchical clustering layers is , and the number of clustering centers of each layer after clustering is , represents the number of clustering centers of the layer, The final clustering center output after clustering the data in the cloud vector database is the clustering center of the last layer, and the number of clustering centers of the last layer is , and satisfy ; When , the number of clustering centers of each layer and the number of clustering centers of the next layer satisfy the following conditions: , wherein, , , , , , , respectively represent the first, second, third, …, , … The number of cluster centers of the layer; when When using two-level clustering, , at this time the number of cluster centers in the first layer is , the number of cluster centers in the second layer is .

[0009] The clustering and stratification process is as follows: S2.1.1. Before each layer of clustering, the position of the cluster center of that layer is initialized. The number of cluster centers in the first layer is , cloud vector database to be clustered Total First, the cloud vector database Divide into disjoint subsets , Indicates the disjoint subsets, , calculated by the k-means algorithm The number of representative nodes in the first layer , the calculation formula is as follows: , in, Indicates the intra-layer balance adjustment parameter, set ; After all subsets in the first layer are clustered, the representative nodes obtained from each subset are combined into a representative node set. The size of the representative node set is , ; Then, the k-means++ clustering algorithm is used to select the representative nodes from the set. Nodes are used as the initial cluster center set of the first layer clustering , Indicates the first layer Initial cluster center vectors, ; Similarly, we get Initial cluster center set for layer clustering , Indicates the Tier Initial cluster center vectors, , iterate multiple times until the Initial cluster center set for layer clustering , Indicates the Tier Initial cluster center vectors, ; S2.1.2, the initial cluster center set of the first layer clustering is clustered by the minibatch k-means clustering algorithm and the cloud vector database Multiple clustering iterations are performed, and the minibatch k-means is loaded each time The vectors are clustered, and the clustering result is checked every 10 iterations. When the clustering converges or reaches the maximum number of iterations, the first layer clustering is completed. Similarly, the initial cluster center set of the first layer clustering is clustered by the minibatch k-means clustering algorithm and the cloud vector database Multiple clustering iterations are performed until the first layer clustering is completed, and multiple iterations are performed to complete the first layer clustering; In any layer clustering, every 10 iterations is an iteration unit, and each layer clustering is composed of multiple iteration units. The cluster center within the layer is adjusted according to the result of each iteration unit. For any iteration unit, the input is the cluster center output after the previous iteration unit is completed, and the output clustering result is the output cluster center set. For any iteration unit, the input cluster center set is , and the vector representation of the first input cluster center, respectively; the output cluster center set is , and the vector representation of the first output cluster center, respectively.

[0010] S2.2, introduce a multi-level balance adjustment strategy to adjust the balance of the iteration unit: The balance adjustment includes intra-layer balance adjustment and inter-layer balance adjustment; S2.2.1, intra-layer balance adjustment: For any layer clustering, the process of intra-layer balance adjustment of each iteration unit includes three steps: (1) complete the iteration to obtain temporary clusters; (2) calculate the average cluster size and define large clusters and small clusters; (3) split the large clusters and eliminate the small clusters. The specific process is as follows: First, in the iteration unit, after 10 iterations are completed, the temporary cluster center set , and the vector representation of the first temporary cluster center, respectively, the cloud vector database is assigned to the nearest temporary cluster center to obtain temporary clusters , and the vector representation of the first The temporary clusters corresponding to the vectors of the temporary cluster centers, each temporary cluster represents a vector set; The second step is to calculate the average size of each temporary cluster. The calculation process is as follows: , in, Indicates the Average size of temporary clusters of layers; when When , the temporary cluster is a large cluster. When , the temporary cluster is a small cluster; The third step is to split the large cluster into In each subset, the average value of all vectors in the subset is calculated, and the obtained average value is used as the cluster center of the subset. Then the temporary cluster center corresponding to the large cluster is From the cluster center collection Delete it and add the newly calculated new cluster center to Newly added The cluster center ratio from The number of cluster centers deleted is large, in order to maintain , , The number of cluster centers is consistent to eliminate small clusters; Calculate the number of additional cluster centers , the calculation process is as follows: , After the large cluster split operation, Sort the cluster centers in the cluster, arrange them in descending order according to the corresponding cluster size, and then delete The smallest cluster size The cluster centers corresponding to the clusters form the final output .

[0011] S2.2.2. Adjustment of inter-layer balance: After clustering at any layer, the cluster center of that layer is obtained , , Respectively Tier The vector representation of the cluster center is converted into a cloud vector database. Assigned to The cluster centers of the first layer are obtained Clusters of hierarchical clustering , Respectively Clusters corresponding to cluster centers, each cluster represents a vector set; According to And the formula Get the next layer of cluster center number , by Get the average size of the cluster size corresponding to each cluster center , calculate the first Layer cluster Each cluster in the first Layer clustering is the number of temporary cluster centers to be assigned , the calculation formula is as follows: , , According to the first Layer cluster Cluster to calculate the temporary cluster center of the first Layer and for , the calculation process is as follows: , At this time, , the first Layer Temporary cluster center is assigned, and the first Layer Cluster According to the order from large to small, then add the first Layer sorted cluster to the first Layer, and the first Layer In turn, add a cluster in the first Layer in order to each cluster, and the updated As the final cluster center assignment result of the first Layer.

[0012] S3 is as follows: Through the PQ quantization algorithm, the cloud vector database is compressed to obtain the quantized cloud vector database , according to And the last layer of cluster center obtained by iteration in step S2 Build IVF-HNSW index structure, Indicates the first Layer in the first Cluster center; Specifically, the quantized cloud vector database Is assigned to the nearest cluster center to form a cluster structure, and the cluster center is used to represent the cluster, so as to realize the construction of IVF inverted index, and then all cluster centers On the basis of the above, the HNSW algorithm is used to construct a graph index to manage the cluster centers, and the IVF-HNSW framework structure is used to manage the speech feature vectors and the calculated cluster centers, wherein the vector and cluster center organization management strategy is IVF, and the cluster center organization management strategy is HNSW.

[0013] S4 is specifically as follows: The data in the speech feature extraction model is migrated to the small terminal model through mature model scaling technology, the small terminal model is deployed to the intelligent speech terminal to extract the user speech instruction feature vector, the extracted user speech instruction feature vector is used as a query vector, the query vector is sent to the cloud through WIFI / 5G, and the historical speech instruction information of the user is transmitted to the cloud as auxiliary information.

[0014] The effects provided in the summary are only the effects of the embodiments, not all the effects of the application, and the above technical solutions have the following advantages or beneficial effects: The application uses the approximate nearest neighbor ANNS technology, combines the cloud vector database and the terminal lightweight feature extraction deep model, improves the recognition efficiency of complex semantic instructions, and improves the recognition accuracy of user purposes. In the aspect of the ANNS technology based on the inverted index, on the one hand, the hierarchical clustering framework is used to replace the original single-layer clustering in the inverted index, and the minibatch k-means method is used in the hierarchical clustering framework, so that the time complexity of the clustering operation in the index construction process is greatly reduced, thereby improving the inverted index construction efficiency; on the other hand, in each layer of clustering, the initialization point selection strategy of divide and conquer is used, which reduces the initial point selection time of the clustering algorithm while ensuring the clustering accuracy, optimizes the overall clustering time, and further shortens the index construction time. The application proposes an intra / inter-layer balance adjustment strategy to realize multi-level cluster size balance adjustment. The intra-layer balance adjustment effectively controls the cluster size in the clustering iteration process, so that the cluster size will not be too large or too small; the inter-layer balance adjustment strategy allocates the cluster centers to each cluster in a proportional manner, avoids the underfitting of large clusters and the overfitting of small clusters, and the two strategies are matched with each other to ensure that the cluster sizes are roughly equal, thereby improving the inverted index retrieval efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0015] The accompanying drawings are used to provide a further understanding of the application, and constitute a part of the specification, together with the embodiments of the application, to explain the application, and do not constitute a limitation on the application.

[0016] Figure 1 It is a method flowchart of the application. DETAILED DESCRIPTION

[0017] In order to clearly illustrate the technical features of this solution, the present invention is described in detail below through specific implementation methods and in conjunction with the accompanying drawings.

[0018] Example 1 like Figure 1 As shown in FIG, a fast intelligent voice assistant response method based on an approximate nearest neighbor retrieval algorithm is as follows: S1. Collect the user's historical voice command information to build a cloud database, then input the voice commands in the cloud database into the voice feature extraction model for voice feature extraction. The voice feature is in the form of <voice feature vector, command pair>, and then save the voice feature vector to the cloud vector database. ; S2. Cluster the speech feature vectors in the cloud vector database and calculate the cluster center set for each layer. Then, select the initial cluster centers for each cluster layer, and make the number of initial cluster centers the same as the number of cluster centers in the cluster center set. Then, introduce a multi-level balance adjustment strategy to adjust the balance. The multi-level balance adjustment strategy includes an intra-layer balance adjustment strategy and an inter-layer balance adjustment strategy. S3, using the IVF-HNSW framework to manage the speech feature vectors and the calculated cluster centers; S4. Migrate the data in the voice feature extraction model, extract the user's voice command feature vector, and transmit it to the cloud together with the user's historical voice command information as auxiliary information; S5. After the cloud receives the user query vector and voice information, it queries the user's voice command feature vector through the IVF-HNSW index structure, queries and outputs similar voice vectors stored in the cloud database, matches the vector number in the query result to the corresponding <voice feature vector, command> key value, and returns the command to the local intelligent voice terminal.

[0019] S1 is as follows: The historical voice command information in the cloud database Input into the speech feature extraction module to obtain speech features, and then save the speech features to the cloud vector database. The speech features are represented as , Represents any historical voice command information in the cloud database. Indicates historical voice command information The obtained speech feature vector is Indicates historical voice command information The obtained instruction pair, and Respectively represent historical voice command information The object of the obtained instruction and the instruction required action, the cloud vector database is expressed as , represents the first voice feature vector in the cloud vector library, , .

[0020] S2 is as follows: S2.1, clustering and layering the voice feature vectors: The voice feature vectors in the cloud vector database are subjected to layer hierarchical clustering, the number of hierarchical clustering layers is , and the number of cluster centers of each layer after clustering is , represents the number of cluster centers of the layer, , and the final cluster center output after clustering the data in the cloud vector database is the cluster center of the last layer, and the number of cluster centers of the last layer is , and satisfy ; When , the number of cluster centers of each layer and the number of cluster centers of the next layer satisfy the following conditions: , wherein , , , …, , , …, , represent the number of cluster centers of the 1st, 2nd, 3rd, …, , … layer, respectively; When , two-layer clustering is used, , and at this time, the number of cluster centers of the first layer is , and the number of cluster centers of the second layer is .

[0021] The clustering and layering process is as follows: S2.1.1, before clustering at each layer, the cluster center position of the layer is initialized, the number of cluster centers of the first layer is , and the cloud vector database to be clustered has The cloud vector database is first divided into non-intersecting subsets , , representing the first non-intersecting subset, , the number of representative nodes of the first layer is calculated by the k-means algorithm , and the calculation formula is as follows: , wherein, represents a layer balance adjustment parameter, and is set to ; After clustering of all subsets of the first layer is completed, the representative nodes obtained by each subset are combined into a representative node set, and the size of the representative node set is , ; Then, the k-means++ clustering algorithm is used to select nodes from the representative node set as an initial cluster center set of the first layer clustering , representing the first initial cluster center vector of the first layer, ; Similarly, an initial cluster center set of the first layer clustering is obtained , representing the first initial cluster center vector of the first layer, , multiple iterations are performed until an initial cluster center set of the first layer clustering is obtained , representing the first initial cluster center vector of the first layer, ; S2.1.2, the initial cluster center set of the first layer clustering and the cloud vector database are clustered multiple times by the minibatch k-means clustering algorithm, the minibatch k-means loads vectors for clustering each time, and the clustering result is checked every 10 iterations, and the first layer clustering is completed when the clustering converges or the maximum number of iterations is reached; Similarly, the initial cluster center set of the first layer clustering is clustered by the minibatch k-means clustering algorithm And cloud vector data Multiple clustering iterations are performed until the completion of the layer clustering, multiple iterations, the completion of the layer clustering; In any layer clustering, every 10 iterations is an iteration unit, and each layer clustering is composed of multiple iteration units. The cluster centers in the layer are adjusted according to the results of each iteration unit. For any iteration unit, the input is the cluster center output after the completion of the previous iteration unit, and the output cluster result is the output cluster center set. For any iteration unit, the input cluster center set is , The vector representation of the first input cluster center is respectively; and the output cluster center set is , The vector representation of the first output cluster center is respectively.

[0022] S2.2, introduce multi-level balance adjustment strategy to adjust the balance of the iteration unit: The balance adjustment includes intra-layer balance adjustment and inter-layer balance adjustment; S2.2.1, intra-layer balance adjustment: For any layer clustering, the process of intra-layer balance adjustment of each iteration unit includes three steps: (1) complete the iteration to obtain temporary clusters; (2) calculate the average cluster size and define large clusters and small clusters; (3) split the large clusters and eliminate the small clusters, the specific process is as follows: First, in the iteration unit, after completing 10 iterations, a temporary cluster center set , The vector representation of the first temporary cluster center is respectively, and the cloud vector database is assigned to the nearest temporary cluster center to obtain temporary clusters , The temporary cluster corresponding to the vector of the first temporary cluster center, and each temporary cluster represents a vector set. Second, calculate the average size of each temporary cluster size as follows: , Wherein, Indicates the average size of the layer temporary cluster; When , the temporary cluster is a large cluster, , the temporary cluster is a small cluster; Third, for large clusters, split the large clusters into a subset, and calculate the average value of all vectors in each subset, take the average value as the cluster center of the subset, and then add the temporary cluster center corresponding to the large cluster to from the cluster center set and add the newly calculated new cluster center to The number of cluster centers added to is more than the number of cluster centers deleted from To keep , , The number of cluster centers in is consistent, and the small cluster is eliminated; The number of extra cluster centers is calculated as follows: , Sort the cluster centers in obtained after the large cluster splitting operation, arrange them in descending order of corresponding cluster size, then delete the cluster centers corresponding to the last clusters with the smallest cluster size, and form the final output . .

[0023] S2.2.2, interlayer balance adjustment: After any layer completes clustering, the cluster center of the layer is obtained , , The vector representation of the th cluster center of the th layer, respectively, and the cloud vector database is assigned to the th layer cluster center to obtain the cluster of the th layer clustering , The th cluster center corresponds to a cluster, and each cluster represents a vector set. According to and formula , the number of cluster centers of the next layer is obtained , the average size of the cluster size corresponding to each cluster center is obtained through , and the th cluster in the th layer is calculated Each cluster in the th layer clustering is assigned to the temporary cluster center number of the th layer, and the calculation formula is as follows: , According to the first layer cluster , the temporary clustering centers of the first layer are calculated and are , and the calculation process is as follows: , At this time, , the first layer temporary clustering centers are assigned, the first layer clusters are arranged in descending order, and then the first layer sorted clusters are added to the first layer, and the first layer is added to each cluster in turn one of the first layer arranged clusters, and the updated is used as the final clustering center assignment result of the first layer.

[0024] S3 is as follows: The cloud vector database is compressed by the PQ quantization algorithm to obtain a quantized cloud vector database , and the last layer clustering center obtained by iteration in step S2 is used to construct an IVF-HNSW index structure, , wherein represents the first layer cluster center; The quantized cloud vector database is assigned to the nearest cluster center to form a cluster structure, and the cluster center is used to represent the cluster, thereby realizing the construction of the IVF inverted index. Then, based on all the clustering centers , the HNSW algorithm is used to construct a graph index to manage the clustering centers, and the voice feature vectors and the calculated clustering centers are managed from the IVF-HNSW framework structure, wherein the vector and clustering center organization management strategy is IVF, and the clustering center organization management strategy is HNSW.

[0025] S4 is as follows: The data in the voice feature extraction model is migrated to a small terminal model by a mature model scaling technology, the small terminal model is deployed to an intelligent voice terminal to extract a user voice instruction feature vector, the extracted user voice instruction feature vector is used as a query vector, the query vector is sent to the cloud through WIFI / 5G, and the historical voice instruction information of the user is transmitted to the cloud as auxiliary information.

[0026] Example 2 A large number of performance comparison experiments are carried out on open source benchmark data, and the experimental results can prove that the clustering framework provided by the application can realize efficient index construction and approximate nearest neighbor search.

[0027] Specifically as follows: This embodiment details the application of the method in the application to a plurality of internationally advanced inverted index algorithms and the performance thereof, as shown in Table 1, the method in the application can be integrated into the following two types of inverted index algorithm architectures, the first type is a common IVF structure combined with different quantization algorithms, wherein the quantization algorithms include four types, namely PQfs (product quantization), RQ (residual quantization), LSQ (local scalar quantization), and SQ (scalar quantization), thereby forming four different IVF+quantization index structures, IVF-PQfs, IVF-RQ, IVF-LSQ, and IVF-SQ; the method in the application can also be combined with another type of IVF structure, namely an IVF+graph index structure, which has two representative implementation manners, IVF-HNSW and IVF-NSG, IVF-HNSW is an Inverted File System (IVF) with Hierarchical Navigable Small Worlds (HNSW), namely an index structure combining an inverted index and an approximate nearest neighbor algorithm, and IVF-NSG is an Inverted File System (IVF) with Navigating Spreading-out Graph (NSG), namely an index structure combining an inverted index and an approximate nearest neighbor search.

[0028] By the method in the application and the commonly used clustering algorithms in the above two types of six different IVF structures, the effectiveness of the application can be proved.

[0029] This embodiment selects two commonly used comparison algorithms, namely K-means (K-means clustering algorithm) and Mini-Batch K-means (a variant of the K-means clustering algorithm), compares the data sets, and selects internationally general multi-modal vector data sets, which include four 1B data sets BIGANN-1B, DEEP-1B, MSTuring-1B, and MSSPACEV-1B, and four 10M data sets BIGANN-10M, DEEP-10M, MSTuring-10M, and MSSPACEV-10M. Among them, the BIGANN (Big Artificial Neural Network) dataset is a large-scale artificial neural network dataset for evaluating and comparing different approximate nearest neighbor search algorithms; DEEP-1B is a large-scale vector dataset released by Yandex, which contains descriptor data composed of the projection and normalized output of the last fully connected layer of the GoogLeNet model; the MSTuring dataset may refer to the vector dataset released by the Microsoft Turing team; Microsoft SPACEV-1B is a dataset related to web search released by Microsoft Bing, which is composed of document and query vectors encoded by the Microsoft SpaceV Superior model, aiming to capture general intent representations.

[0030] Table 2 compares the clustering efficiency on different scale datasets, and the clustering efficiency is shown by the clustering time, the shorter the clustering time, the higher the efficiency, the embodiment is set in two groups, the first group of experiments is to realize the clustering of 65,536, 262,144, 1,048,576 clustering centers on four 10 billion scale datasets respectively, on the four datasets, compared with K-means and Mini-Batch K-means, the method in the application has a large efficiency improvement, and the clustering time is greatly shortened; for example, on the BIGANN-1B, to complete the clustering of 1,048,576 clustering centers, the K-means algorithm needs 1,252.92 seconds, the Mini-Batch K-means needs 1,665.25 seconds, and the method in the application only needs 956.57 seconds; The second group of experiments realizes the clustering of 4,096, 16,384, 65,536 clustering centers on four 10 million scale datasets respectively, which is consistent with the 10 billion scale, on the four datasets, compared with K-means and Mini-Batch K-means, the method in the application has a large efficiency improvement, and the clustering time is greatly shortened; for example, on the BIGANN-1B, to complete the clustering of 65,536 clustering centers, the K-means algorithm needs 41.98 seconds, the Mini-Batch K-means needs 43.36 seconds, and the method in the application only needs 36.49 seconds.

[0031] Table 3 compares the effect of applying the method in the application to different inverted indexes, the effect is measured by evaluation indexes Recall@10 and QPS, wherein Recall@10 is the retrieval accuracy, the calculation method is the percentage of the coincidence of the TOP-10 nearest neighbors output by the algorithm and the real TOP-10 nearest neighbors, and QPS (Queries Per Second) represents the number of query vectors processed per second.

[0032] The embodiment is set in two groups, the first group of experiments compares the effects of different clustering methods on 6 IVF indexes on four 100 million scale data sets respectively, and the evaluation results show that the method in the application is significantly better than the traditional K-means and Mini-batch K-means in retrieval efficiency, while maintaining high retrieval accuracy, the number of query vectors processed per second is significantly improved; for example, on BIGANN-10M, when the IVF-LSQ index is constructed, the Recall@10 obtained by using kmeans clustering is 0.829, and the QPS is 4,711.86; the Recall@10 is 0.875 when using Mini-batch K-means, and the QPS is 5,034.87; and the Recall@10 obtained by using the method in the application is 0.877, and the QPS is 5,162.29; The second group of experiments compares the effects of different clustering methods on 3 IVF indexes on four 1 billion scale data sets respectively, and the evaluation results show that the method in the application is significantly better than the traditional K-means and Mini-batch K-means in retrieval efficiency, while maintaining high retrieval accuracy, the number of query vectors processed per second is significantly improved; for example, on DEEP-1B, when the IVF-PQfs index is constructed, the Recall@10 obtained by using K-means clustering is 0.664, and the QPS is 767.56; the Recall@10 is 0.673 when using Mini-batch K-means, and the QPS is 789.32; and the Recall@10 obtained by using the method in the application is 0.674, and the QPS is 866.05.

[0033] Table 1 Comparison algorithm introduction table of the application Table 2 Comparison results of index construction efficiency of the application Table 3 Comparison results of retrieval accuracy and query efficiency of the application Therefore, the hierarchical clustering framework with the initial point selection strategy of fusion and divide and conquer and the intra / inter layer balance adjustment strategy can be applied to various inverted index structures, has universality, and after the clustering center is obtained through the clustering framework, the data set can be compressed in combination with various quantization modes, the index can be directly loaded in limited memory for a large-scale vector data set (billion scale), and therefore the retrieval efficiency of the index is improved by using high-speed memory.

[0034] Although the specific embodiments of the application are described above with reference to the drawings, the description is not a limitation on the protection scope of the application, and various modifications or changes made by those skilled in the art on the basis of the technical solutions of the application without creative labor are still within the protection scope of the application.

Claims

1. A fast intelligent voice assistant response method based on an approximate nearest neighbor retrieval algorithm, characterized in that: The following steps are involved: S1. Collect the user's historical voice command information to build a cloud database, then input the voice commands in the cloud database into the voice feature extraction model for voice feature extraction. The voice feature is in the form of <voice feature vector, command pair>, and then save the voice feature vector to the cloud vector database. ; S2. Cluster the speech feature vectors in the cloud vector database and calculate the cluster center set for each layer. Then, select the initial cluster centers for each cluster layer, and make the number of initial cluster centers the same as the number of cluster centers in the cluster center set. Then, introduce a multi-level balance adjustment strategy to adjust the balance. The multi-level balance adjustment strategy includes an intra-layer balance adjustment strategy and an inter-layer balance adjustment strategy. S3, using the IVF-HNSW framework to manage the speech feature vectors and the calculated cluster centers; S4. Migrate the data in the voice feature extraction model, extract the user's voice command feature vector, and transmit it to the cloud together with the user's historical voice command information as auxiliary information; S5. After the cloud receives the user query vector and voice information, it queries the user's voice command feature vector through the IVF-HNSW index structure, queries and outputs similar voice vectors stored in the cloud database, matches the vector number in the query result to the corresponding <voice feature vector, command> key value, and returns the command to the local intelligent voice terminal.

2. The fast intelligent voice assistant response method based on the approximate nearest neighbor retrieval algorithm according to claim 1 is characterized in that: S1 is as follows: The historical voice command information in the cloud database Input into the speech feature extraction module to obtain speech features, and then save the speech features to the cloud vector database. The speech features are represented as , Represents any historical voice command information in the cloud database. Indicates historical voice command information The obtained speech feature vector is Indicates historical voice command information The obtained instruction pair, and Respectively represent historical voice command information The object of the instruction and the action required by the instruction are obtained, and the cloud vector database is represented as , Indicates the first speech feature vectors, , .

3. The fast intelligent voice assistant response method based on the approximate nearest neighbor retrieval algorithm according to claim 2 is characterized in that: S2 is as follows: S2.

1. Clustering and stratifying speech feature vectors: Cloud vector database in Speech feature vectors Hierarchical clustering of layers, number of hierarchical clustering layers , the number of cluster centers in each layer after clustering is , Indicates the The number of cluster centers of the layer, , for cloud vector database The final cluster center output after clustering the data is the cluster center of the last layer. The number of cluster centers in the last layer is , and Satisfaction between ; when When , the number of cluster centers in each layer and the number of cluster centers in the next layer meet the following conditions: ， in, 、 、 、……、 、 、……、 、 Respectively represent the 1st, 2nd, 3rd, ..., 、 … The number of cluster centers of the layer; when When using two-level clustering, , at this time the number of cluster centers in the first layer is , the number of cluster centers in the second layer is .

4. The fast intelligent voice assistant response method based on the approximate nearest neighbor retrieval algorithm according to claim 3 is characterized in that: The clustering and stratification process is as follows: S2.1.

1. Before each layer of clustering, the position of the cluster center of that layer is initialized. The number of cluster centers in the first layer is , cloud vector database to be clustered Total First, the cloud vector database Divide into disjoint subsets , Indicates the disjoint subsets, , calculated by the k-means algorithm The number of representative nodes in the first layer , the calculation formula is as follows: ， in, Indicates the intra-layer balance adjustment parameter, set ; After all subsets in the first layer are clustered, the representative nodes obtained from each subset are combined into a representative node set. The size of the representative node set is , ; Then, the k-means++ clustering algorithm is used to select the representative nodes from the set. Nodes are used as the initial cluster center set of the first layer clustering , Indicates the first layer Initial cluster center vectors, ; Similarly, we get Initial cluster center set for layer clustering , Indicates the Tier Initial cluster center vectors, , iterate multiple times until the Initial cluster center set for layer clustering , Indicates the Tier Initial cluster center vectors, ; S2.1.

2. Initial cluster center set of the first layer clustering using minibatch k-means clustering algorithm and cloud-based vector databases Perform multiple clustering iterations, loading minibatch k-means in each iteration The vectors are clustered and the clustering results are checked every 10 iterations. When the clustering converges or the maximum number of iterations is reached, the first layer of clustering is completed. Similarly, the minibatch k-means clustering algorithm is used to cluster the Initial cluster center set for layer clustering and cloud vector data Perform multiple clustering iterations until the Layer clustering, multiple iterations, complete the Hierarchical clustering; In any layer clustering, every 10 iterations is an iteration unit. Each layer clustering consists of multiple iteration units. The cluster centers in the layer are adjusted according to the results of each iteration unit. For any iteration unit, the input is the cluster center output after the previous iteration unit is completed. The output clustering result is the output cluster center set. For any iteration unit, the input cluster center set is , Respectively The vector representation of the input cluster centers; the set of cluster centers output is , Respectively The vector representation of the output cluster centers.

5. The fast intelligent voice assistant response method based on the approximate nearest neighbor retrieval algorithm according to claim 4 is characterized in that: S2 is as follows: S2.

2. Introduce a multi-level balance adjustment strategy to adjust the balance of the iteration unit: Balance adjustment includes intra-layer balance adjustment and inter-layer balance adjustment; S2.2.

1. Adjustment of intra-layer balance: For any layer of clustering, the process of adjusting the intra-layer balance of each iteration unit includes three steps: (1) completing the iteration and obtaining temporary clusters; (2) calculating the average cluster size and defining large clusters and small clusters; (3) splitting the large clusters and eliminating the small clusters. The specific process is as follows; The first step is to obtain a temporary cluster center set after completing 10 iterations within the iteration unit. , Respectively The vector representation of the temporary cluster center is converted into a cloud vector database. Assigned to the nearest temporary cluster center, the temporary cluster is , Respectively The temporary clusters corresponding to the vectors of the temporary cluster centers, each temporary cluster represents a vector set; The second step is to calculate the average size of each temporary cluster. The calculation process is as follows: ， in, Indicates the Average size of temporary clusters of layers; when When , the temporary cluster is a large cluster. When , the temporary cluster is a small cluster; The third step is to split the large cluster into In each subset, the average value of all vectors in the subset is calculated, and the obtained average value is used as the cluster center of the subset. Then the temporary cluster center corresponding to the large cluster is From the cluster center collection Delete it and add the newly calculated new cluster center to Newly added The cluster center ratio from The number of cluster centers deleted is large, in order to maintain , , The number of cluster centers is consistent to eliminate small clusters; Calculate the number of additional cluster centers , the calculation process is as follows: ， After the large cluster split operation, Sort the cluster centers in the cluster, arrange them in descending order according to the corresponding cluster size, and then delete The smallest cluster size The cluster centers corresponding to the clusters form the final output .

6. The fast intelligent voice assistant response method based on the approximate nearest neighbor retrieval algorithm according to claim 5 is characterized in that: S2 is as follows: S2.2.

2. Adjustment of inter-layer balance: After clustering at any layer, the cluster center of that layer is obtained , , Respectively Tier The vector representation of the cluster center is converted into a cloud vector database. Assigned to The cluster center of the first layer is obtained Clusters of hierarchical clustering , Respectively Clusters corresponding to cluster centers, each cluster represents a vector set; according to And the formula Get the number of cluster centers in the next layer ,pass Get the average size of the cluster corresponding to each cluster center , calculate the Layer Cluster Each cluster in In the Layer clustering is the number of temporary cluster centers that should be assigned , the calculation formula is as follows: ， According to Layer Cluster The clusters are calculated to get The temporary cluster centers of the layer are , the calculation process is as follows: ， at this time, , then layer The temporary cluster centers are allocated. layer Clusters Arrange them in order from largest to smallest, and then The sorted clusters are added to the In the layer, Layer Add a A cluster in the layer after sorting, the updated As the first The final cluster center assignment result of the layer.

7. The fast intelligent voice assistant response method based on the approximate nearest neighbor retrieval algorithm according to claim 6 is characterized in that: S3 is as follows: The cloud vector database is compressed using the PQ quantization algorithm to obtain the quantized cloud vector database. ,according to The cluster center of the last layer obtained by iterating in step S2 Construct IVF-HNSW graph index structure, Indicates the Layer cluster centers; Specifically, the quantized cloud vector database Assign to the nearest cluster center to form a cluster structure, use the cluster center to represent the cluster, thereby realizing the construction of IVF inverted index, and then in all cluster centers Based on this, the HNSW algorithm is used to build a graph index to manage the cluster centers. The speech feature vectors and the calculated cluster centers are managed based on the IVF-HNSW framework structure. The organization and management strategy of the vectors and cluster centers is IVF, and the organization and management strategy of the cluster centers is HNSW.

8. The fast intelligent voice assistant response method based on the approximate nearest neighbor retrieval algorithm according to claim 7 is characterized in that: S4 is as follows: Through mature model scaling technology, the data in the voice feature extraction model is migrated to the small terminal model, and the small terminal model is deployed to the intelligent voice terminal to extract the user voice command feature vector. The extracted user voice command feature vector is used as the query vector, and the query vector is sent to the cloud via WIFI / 5G. The user's historical voice command information is also transmitted to the cloud as auxiliary information.

Images