Voice tag generation method and device, electronic equipment and storage medium

Abstract

The invention relates to a voice tag generation method and device, electronic equipment and a storage medium, and belongs to the technical field of computers, and the method comprises the steps: carrying out the feature matrix decomposition of a plurality of to-be-processed first voice feature matrixes, and obtaining a decomposed first matrix and a decomposed second matrix; performing hierarchical clustering on column vectors in the first matrix to obtain a tag tree of the first matrix; performing projection processing on the second voice according to the second matrix to obtain a voice feature representation vector of which the dimension is the same as the column number of the first matrix; querying a tag tree, and determining a feature tag of the second voice; wherein the feature tag of the second voice is determined according to the clustering tag in the tag tree mapped by the voice feature representation vector. According to the embodiment of the invention, the construction and maintenance cost of a voice feature tag generation system can be remarkably reduced, and the automation degree and robustness of tag generation are improved.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a method and apparatus for generating voice tags, an electronic device, and a storage medium. Background Technology

[0002] Content understanding systems are technologies that process and analyze various forms of content (such as text, images, and audio) to understand their meaning and context. Combining Natural Language Processing (NLP), machine learning, and data mining, they aim to automatically identify and extract key information from content, thereby providing more intelligent services. Content understanding systems play a crucial role in recommendation systems, search systems, and intelligent customer service systems, improving the intelligence level and user experience of these systems by understanding and processing content.

[0003] For audio content understanding, it is often necessary to classify and label the input speech. Tagging systems play a crucial role in speech content understanding systems, improving retrieval efficiency, enabling accurate recommendations, and enhancing semantic understanding, thereby improving the system's functionality and user experience. However, in existing speech content understanding systems, constructing speech feature tagging systems is complex, and manual annotation is costly. Summary of the Invention

[0004] This disclosure presents a method, apparatus, electronic device, and storage medium for generating voice tags.

[0005] According to one aspect of this disclosure, a method for generating speech tags is provided, comprising: performing feature matrix decomposition on feature matrices of multiple first speech samples to be processed, obtaining a decomposed first matrix and a second matrix, wherein each row of the feature matrix corresponds to a feature vector of a different first speech sample, the number of rows in the first matrix is ​​the same as the number of first speech samples, and the number of columns in the first matrix is ​​the same as the number of rows in the second matrix; performing hierarchical clustering on the column vectors in the first matrix to obtain a tag tree of the first matrix, the tag tree having multiple layers; each layer including clustering labels obtained based on the hierarchical clustering; performing projection processing on the second speech sample according to the second matrix to obtain a speech feature representation vector with the same dimension as the number of columns in the first matrix; querying the tag tree to determine the feature tags of the second speech sample; wherein the feature tags of the second speech sample are determined based on the clustering labels in the tag tree mapped from the speech feature representation vectors.

[0006] In one possible implementation, hierarchical clustering is performed on the column vectors in the first matrix to obtain a label tree for the first matrix, including: performing clustering processing on each column vector in the first matrix to determine the clustering result for each column vector, wherein the clustering result includes a clustering label vector and clustering model parameters corresponding to each column vector; determining the first column vector with the best clustering quality from multiple column vectors in the first matrix based on the quality score of the clustering result of each column vector in the first matrix; using the column number of the first column vector as a target feature index; and determining the clustering label of the first layer in the label tree based on the target feature index and the clustering model parameters corresponding to the first column vector.

[0007] In one possible implementation, hierarchical clustering is performed on the column vectors in the first matrix to obtain a label tree for the first matrix. This includes: performing clustering processing on each column vector in the sample subset corresponding to the clusters obtained from the previous layer of clustering processing, determining the clustering result for each column vector, wherein the clustering result includes a cluster label vector and clustering model parameters corresponding to each column vector, and the clusters obtained from the previous layer of clustering processing are determined by the cluster label vector corresponding to the column vector with the best clustering quality from the previous layer; determining the second column vector with the best clustering quality from multiple column vectors in the sample subset corresponding to the cluster based on the quality score of the clustering result of each column vector in the sample subset corresponding to the cluster; using the column number of the second column vector as a target feature index; and determining the cluster label of the current layer in the label tree based on the target feature index and the clustering model parameters corresponding to the second column vector.

[0008] In one possible implementation, clustering is performed on each column vector in the sample subset corresponding to the cluster obtained by the previous layer clustering process to determine the clustering result of each column vector. This includes: if the maximum value of the quality score of the clustering result of each column vector in the previous layer clustering process is greater than or equal to a first preset threshold, or if the sample size in the cluster obtained by the previous layer clustering process is greater than or equal to a second preset threshold, then clustering is performed on each column vector in the cluster obtained by the previous layer clustering process to determine the clustering result of each column vector.

[0009] In one possible implementation, the quality score is determined by a scoring function that includes a silhouette coefficient and / or normalized information entropy. The silhouette coefficient measures the intra-cluster compactness and inter-cluster separation of the clustering process, and the normalized information entropy measures the balance of cluster size.

[0010] In one possible implementation, querying the tag tree to determine the feature labels of the second speech includes: extracting target elements from the speech feature representation vector at target positions for the cluster labels of the i-th layer in the tag tree, in order from low to high levels, where the target position is determined by the target feature index in the i-th layer cluster labels, and the i-th layer indicates that any layer i in the tag tree is a positive integer less than or equal to P; where P is the total number of layers in the tag tree; inputting the target element into a target clustering model to obtain the target labels of the i-th layer, where the target clustering model is determined by the clustering model parameters of the i-th layer cluster labels; and using the concatenation result of the target labels of each layer in the tag tree as the feature labels of the second speech.

[0011] In one possible implementation, the method for obtaining the feature matrix of the plurality of first speech samples includes: preprocessing the plurality of first speech samples to be processed to obtain a plurality of preprocessed speech samples; segmenting each preprocessed speech sample into sub-speech samples according to a preset speech activity detection model; encoding the sub-speech samples of each preprocessed speech sample to obtain the encoding vector of each sub-speech sample; and aggregating the encoding vectors of the plurality of sub-speech samples to obtain the feature matrix.

[0012] In one possible implementation, the feature matrix is ​​obtained by aggregating the encoding vectors of multiple sub-speech segments, including: pooling the encoding vectors of multiple sub-speech segments in each preprocessed speech segment to obtain a first aggregated vector corresponding to each preprocessed speech segment, wherein the pooling aggregation includes at least one of mean pooling aggregation, max pooling aggregation, minimum pooling aggregation, and attention pooling aggregation; and generating the feature matrix based on the first aggregated vector.

[0013] According to one aspect of this disclosure, a speech tag generation apparatus is provided, comprising: a decomposition module, configured to decompose feature matrices of multiple first speech samples to be processed, obtaining a decomposed first matrix and a second matrix, wherein each row of the feature matrix corresponds to a feature vector of a different first speech sample, the number of rows of the first matrix is ​​the same as the number of first speech samples, and the number of columns of the first matrix is ​​the same as the number of rows of the second matrix; a clustering module, configured to perform hierarchical clustering on the column vectors in the first matrix to obtain a tag tree of the first matrix, the tag tree having multiple layers; each layer including clustering labels obtained based on the hierarchical clustering; a projection module, configured to project the second speech samples according to the second matrix to obtain speech feature representation vectors with the same dimension as the number of columns of the first matrix; and a query module, configured to query the tag tree to determine the feature tags of the second speech samples; wherein the feature tags of the second speech samples are determined based on the clustering labels in the tag tree mapped from the speech feature representation vectors.

[0014] In one possible implementation, the clustering module is configured to: perform clustering processing on each column vector in the first matrix to determine the clustering result of each column vector, the clustering result including the clustering label vector and clustering model parameters corresponding to each column vector; determine the first column vector with the best clustering quality from multiple column vectors in the first matrix based on the quality score of the clustering result of each column vector in the first matrix; use the column number of the first column vector as the target feature index; and determine the clustering label of the first layer in the label tree based on the target feature index and the clustering model parameters corresponding to the first column vector.

[0015] In one possible implementation, the clustering module is configured to: perform clustering processing on each column vector in the sample subset corresponding to the clusters obtained from the previous layer clustering process, determine the clustering result of each column vector, the clustering result including the cluster label vector and clustering model parameters corresponding to each column vector, wherein the clusters obtained from the previous layer clustering process are determined by the cluster label vector corresponding to the column vector with the best clustering quality from the previous layer; determine the second column vector with the best clustering quality from multiple column vectors in the sample subset corresponding to the clusters based on the quality score of the clustering result of each column vector in the sample subset corresponding to the clusters; use the column number of the second column vector as the target feature index; and determine the clustering label of the current layer in the label tree based on the target feature index and the clustering model parameters corresponding to the second column vector.

[0016] In one possible implementation, clustering is performed on each column vector in the sample subset corresponding to the cluster obtained by the previous layer clustering process to determine the clustering result of each column vector. This includes: if the maximum value of the quality score of the clustering result of each column vector in the previous layer clustering process is greater than or equal to a first preset threshold, or if the sample size in the cluster obtained by the previous layer clustering process is greater than or equal to a second preset threshold, then clustering is performed on each column vector in the cluster obtained by the previous layer clustering process to determine the clustering result of each column vector.

[0017] In one possible implementation, the quality score is determined by a scoring function that includes a silhouette coefficient and / or normalized information entropy. The silhouette coefficient measures the intra-cluster compactness and inter-cluster separation of the clustering process, and the normalized information entropy measures the balance of cluster size.

[0018] In one possible implementation, the query module is configured to: extract target elements from the speech feature representation vector at target positions for the cluster labels of the i-th layer in the label tree, in order from low to high levels, wherein the target position is determined by the target feature index in the i-th layer cluster labels, and the i-th layer indicates that any layer i in the label tree is a positive integer less than or equal to P; where P is the total number of layers in the label tree; input the target element into a target clustering model to obtain the target label of the i-th layer, wherein the target clustering model is determined by the clustering model parameters of the i-th layer cluster labels; and use the concatenation result of the speech feature representation vector of the target labels in each layer of the label tree as the feature label of the second speech.

[0019] In one possible implementation, the device further includes an acquisition module for acquiring the feature matrix of the plurality of first speech samples, comprising: preprocessing the plurality of first speech samples to be processed to obtain a plurality of preprocessed speech samples; segmenting each preprocessed speech sample into sub-speech samples according to a preset speech activity detection model; encoding the sub-speech samples of each preprocessed speech sample to obtain an encoding vector for each sub-speech sample; and aggregating the encoding vectors of the plurality of sub-speech samples to obtain the feature matrix.

[0020] In one possible implementation, the feature matrix is ​​obtained by aggregating the encoding vectors of multiple sub-speech segments, including: pooling the encoding vectors of multiple sub-speech segments in each preprocessed speech segment to obtain a first aggregated vector corresponding to each preprocessed speech segment, wherein the pooling aggregation includes at least one of mean pooling aggregation, max pooling aggregation, minimum pooling aggregation, and attention pooling aggregation; and generating the feature matrix based on the first aggregated vector.

[0021] According to one aspect of this disclosure, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to perform the method described above.

[0022] According to one aspect of this disclosure, a computer-readable storage medium is provided that stores computer program instructions thereon, which, when executed by a processor, implement the above-described method.

[0023] In this embodiment of the disclosure, feature matrix decomposition is performed on the feature matrices of multiple first speech samples to be processed, resulting in a first matrix and a second matrix. Each row in the feature matrix corresponds to a feature vector of a different first speech sample. The number of rows in the first matrix is ​​the same as the number of first speech samples, and the number of columns in the first matrix is ​​the same as the number of rows in the second matrix. Hierarchical clustering is performed on the column vectors in the first matrix to obtain a label tree of the first matrix. The label tree has multiple layers. Each layer includes clustering labels obtained based on the hierarchical clustering. The second speech samples are projected onto the second matrix to obtain speech feature representation vectors with the same dimension as the number of columns in the first matrix. The label tree is queried to determine the feature labels of the second speech samples. The feature labels of the second speech samples are determined based on the clustering labels in the label tree mapped from the speech feature representation vectors.

[0024] The speech tag generation method disclosed in this embodiment employs a multi-level tag tree automatically generated using hierarchical clustering. The clustering object is the first matrix after feature matrix decomposition (since the feature matrix is ​​composed of feature vectors from different first speech samples, the feature vectors of each first speech sample can be extracted using a pre-trained model, eliminating the need for manual annotation). This eliminates the need for manually designed tag systems and reduces reliance on manual annotation. Furthermore, this tag tree can be used to generate feature tags for new second speech samples. The new second speech samples can be inferred after projection into the existing second matrix space without rerunning the full clustering. In addition, leveraging the structural characteristics of the tag tree itself, when a new speech style emerges in the content ecosystem, incremental clustering can be performed in local subtrees and concatenated to the original tag tree, ensuring the stability of old tags and zero-intrusion deployment of new tags. Therefore, this approach significantly reduces the construction and maintenance costs of the speech feature tag generation system, improves the automation and robustness of tag generation, and possesses good versatility, interpretability, and evolvability. It can be widely applied to various speech content understanding systems, providing strong technical support for improving recommendation quality, retrieval efficiency, and user experience.

[0025] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0026] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the specification, serve to illustrate the technical solutions of this disclosure.

[0027] Figure 1 A flowchart illustrating a method for generating voice tags according to an embodiment of the present disclosure is shown.

[0028] Figure 2 A schematic diagram of a tag tree according to an embodiment of the present disclosure is shown.

[0029] Figure 3 A block diagram of a voice tag generation apparatus according to an embodiment of the present disclosure is shown.

[0030] Figure 4 A block diagram of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0031] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0032] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0033] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0034] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0035] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, data stored, data displayed, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant regions.

[0036] In voice-centric scenarios, the tagging system for content understanding of voice content will differ significantly from the content tagging systems used in personalized recommendations and search systems. For example, in an e-commerce personalized recommendation system, the content understanding tagging system primarily describes aspects such as product category, brand, and characteristics. However, in scenarios where audio content is the primary focus (such as an audio website primarily featuring podcasts and ebooks), if a content understanding system wants to make personalized recommendations based on the speaker's voice characteristics, the tags could include: speech rate tags: slow / medium / fast; tone tags: high / mid / low; timbre tags: soft / clear / steady / hoarse; emotional expression tags: enthusiastic / calm / emotional / humorous; tone tags: friendly / professional / casual / formal; language and accent tags: standard Mandarin / regional accent / foreign language (such as English, French, etc.) / foreign accent (such as British English, American English); speaking style tags: storytelling / discussion / educational / entertainment; interactivity tags: high interactivity / medium interactivity / low interactivity; speech clarity tags: high clarity / medium clarity / low clarity; rhythm tags: distinct rhythm / slow rhythm / variable rhythm; background noise tags: no background noise / light background music / obvious background sound effects. These tags allow the content understanding system to more precisely characterize the voice features of each podcast, enabling personalized recommendations based on user preferences. For example, a user who prefers slower speech, a softer tone, and richer emotional expression might be recommended podcasts or audiobooks featuring those voice characteristics. This not only enhances the user experience but also increases user dependence on the platform and satisfaction.

[0037] To address the aforementioned need for voice feature labeling, an automated voice feature labeling system is required for large-scale voice content scenarios. However, defining a comprehensive labeling system for large-scale voice content scenarios presents significant challenges due to the large number and diverse categories of voice features to be described. For example, the labels in a voice content understanding system require varying levels of granularity across different application scenarios, necessitating definition based on the characteristics of each application. Furthermore, the voice labeling system needs dynamic updates; as the content ecosystem of the application scenario changes, new, previously undefined labels may be added, requiring maintenance by professional content operators, which is costly. In addition, voice data annotation is challenging; accurate annotation demands high levels of expertise, and providing sufficient annotation data for each voice label also incurs substantial costs.

[0038] For example, a typical approach involves training a speech model based on a large amount of labeled data to automatically identify and classify speech features. For instance, speech feature annotation schemes in related technologies require defining all available labels, manually annotating the speech data, and labeling each label with a sufficient number of speech samples. Then, a speech model is trained based on these sufficient speech samples to label the entire existing data; this speech model is a supervised speech labeling model.

[0039] In some cases, the annotation of speech models may have one or more of the following problems:

[0040] First, the maintenance cost of voice feature labels is extremely high: as the content ecosystem of application scenarios changes, it is necessary to dynamically add or delete corresponding labels. In this case, the entire data needs to be re-labeled, which generates a large workload.

[0041] Second, imbalanced label data affects annotation accuracy: For different application scenarios, the distribution of different label content in the same label system varies greatly. For example, on some audiobook websites, there is more "story-related" label content and less "entertainment" label content, which leads to an imbalance between data and samples, and subsequently, accuracy deviations are likely to occur in annotation.

[0042] Third, manual annotation is costly and requires a large amount of data: Training models with manually annotated data is extremely costly. Detailed annotation of speech features based on a vast labeling system, involving complex features such as pitch, timbre, and emotion, is time-consuming and requires specialized knowledge. Furthermore, achieving high accuracy and robustness requires a large amount of labeled data, which poses a significant challenge for audio platforms with abundant existing content. The large workload and low efficiency of annotation lead to lengthy initial data preparation phases.

[0043] Therefore, in large-scale voice content scenarios (such as podcasts, audiobooks, voice customer service, etc.), there is a heavy reliance on manual annotation and fixed tag sets, which leads to high costs, low efficiency, difficulty in updating and maintaining, and low scalability of the tag system when processing massive heterogeneous voice data.

[0044] In view of this, embodiments of the present disclosure provide a method for generating speech tags, which involves decomposing feature matrices of multiple first speech samples to obtain a first matrix and a second matrix, wherein each row of the feature matrix corresponds to a feature vector of a different first speech sample, the number of rows in the first matrix is ​​the same as the number of first speech samples, and the number of columns in the first matrix is ​​the same as the number of rows in the second matrix; performing hierarchical clustering on the column vectors in the first matrix to obtain a tag tree of the first matrix, the tag tree having multiple layers; each layer including clustering labels obtained based on the hierarchical clustering; projecting the second speech sample onto the second matrix to obtain a speech feature representation vector with the same dimension as the number of columns in the first matrix; querying the tag tree to determine the feature tags of the second speech sample; wherein the feature tags of the second speech sample are determined based on the clustering labels in the tag tree mapped from the speech feature representation vector.

[0045] The speech tag generation method disclosed in this embodiment employs a multi-level tag tree automatically generated using hierarchical clustering. The clustering object is the first matrix after feature matrix decomposition (since the feature matrix is ​​composed of feature vectors from different first speech samples, the feature vectors of each first speech sample can be extracted using a pre-trained model, eliminating the need for manual annotation). This eliminates the need for manually designed tag systems and reduces reliance on manual annotation. Furthermore, this tag tree can be used to generate feature tags for new second speech samples. The new second speech samples can be inferred after projection into the existing second matrix space without rerunning the full clustering. In addition, leveraging the structural characteristics of the tag tree itself, when a new speech style emerges in the content ecosystem, incremental clustering can be performed in local subtrees and concatenated to the original tag tree, ensuring the stability of old tags and zero-intrusion deployment of new tags. Therefore, this approach significantly reduces the construction and maintenance costs of the speech feature tag generation system, improves the automation and robustness of tag generation, and possesses good versatility, interpretability, and evolvability. It can be widely applied to various speech content understanding systems, providing strong technical support for improving recommendation quality, retrieval efficiency, and user experience.

[0046] Figure 1 A flowchart illustrating a method for generating voice tags according to an embodiment of the present disclosure is shown, such as... Figure 1 As shown, the method for generating this voice tag may include:

[0047] In step S11, the feature matrices of the multiple first speech samples to be processed are decomposed to obtain the decomposed first matrix and second matrix. Each row in the feature matrix corresponds to the feature vector of a different first speech sample. The number of rows in the first matrix is ​​the same as the number of first speech samples, and the number of columns in the first matrix is ​​the same as the number of rows in the second matrix.

[0048] In step S12, hierarchical clustering is performed on the column vectors in the first matrix to obtain a label tree of the first matrix, wherein the label tree has multiple layers; each layer includes clustering labels obtained based on the hierarchical clustering.

[0049] In step S13, the second speech is projected according to the second matrix to obtain a speech feature representation vector with the same dimension as the number of columns in the first matrix;

[0050] In step S14, the label tree is queried to determine the feature label of the second speech; wherein the feature label of the second speech is determined based on the clustering label in the label tree mapped by the speech feature representation vector.

[0051] In one possible implementation, the voice tag generation method of this disclosure embodiment can be executed by an electronic device such as a terminal device or a server. The terminal device can be a user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc. The method can be implemented by a processor calling computer-readable instructions stored in memory. Alternatively, the method can be executed by a server.

[0052] In one possible implementation, in step S11, M representative voice data content units can be sampled from the target application scenario, such as podcasts, audiobooks, intelligent customer service systems, and voice social platforms, as M first voices, where M is an integer greater than 1. Different sampling mechanisms can be set based on factors such as audio duration, topic diversity, voice style (e.g., speech rate, tone, emotional expression), and language diversity. Specific sampling standards and methods can be flexibly adjusted and optimized according to the characteristics of the actual application scenario, and are not limited here.

[0053] Features can be extracted from M first speech samples to obtain a feature matrix composed of the feature vectors of the M first speech samples. The number of rows in the feature matrix is ​​the same as the number of first speech samples, and each row in the feature matrix corresponds to a feature vector of a first speech sample.

[0054] To better analyze the feature matrices of the M first speech sounds, we can perform feature matrix decomposition to obtain the decomposed first and second matrices. For example, assuming the feature matrix G0 has dimensions (M, 4×D), performing feature matrix decomposition on G0 will decompose it into two smaller matrices, W and H. The feature matrix G0 is the product of the first matrix W and the second matrix H, i.e., G0 ≈ W·H. The first matrix W has dimensions (M, K) and represents the weights or coefficients of the M first speech sounds (i.e., the M row vectors in feature matrix G0) in the new feature space. The second matrix H has dimensions (K, 4×D) and represents the basis vectors of K independent features. These basis vectors can reconstruct the original speech features, and different basis vectors can correspond to different phonemes or acoustic features. The second matrix H can be retained as a feature basis matrix so that the feature vectors of new audio data (such as the second speech) can be projected into the system in the future, so that the newly added audio data (such as the second speech) can reuse the same feature basis, thus ensuring the stability and consistency of speech tag generation.

[0055] The methods for characteristic matrix decomposition may include: Independent Component Analysis (ICA), Principal Component Analysis (PCA), Nonnegative Matrix Factorization (NMF), etc. The embodiments of this disclosure do not limit the methods for characteristic matrix decomposition.

[0056] For example, principal component analysis transforms the feature matrix into orthogonal principal component directions through eigenvalue decomposition of the covariance matrix. This transforms the original high-dimensional feature matrix into a low-dimensional product of a first matrix and a second matrix. The first matrix represents the weights or coefficients of the variance contribution of each component, and the second matrix represents the fundamental vector of the principal component directions.

[0057] For example, Independent Component Analysis (ICA) achieves the separation and dimensionality reduction of the feature matrix by maximizing the non-Gaussianity of the independent components. ICA may include processes such as centering and whitening the data, selecting a non-Gaussianity metric, solving for the independent components, and transforming the data. It transforms the original high-dimensional feature matrix into a low-dimensional product of a first matrix and a second matrix. The first matrix represents the weights or coefficients of the variance contributions of each independent component, and the second matrix represents the fundamental vectors of the independent component directions.

[0058] For example, nonnegative matrix factorization (NMF) achieves the decomposition of the eigenvalue matrix by constraining the result of matrix factorization to be nonnegative. NMF may include processes such as initializing the nonnegative matrix, iterative optimization, and calculating the reconstruction error, transforming the original high-dimensional eigenvalue matrix into a low-dimensional product of a first and a second matrix. The first matrix represents the weights or coefficients of the original eigenvectors in the new eigenspace, and the second matrix represents the fundamental vectors of the independent features. Comparing principal component analysis (PCA) and independent component analysis (ICA), both the first and second matrices obtained by NMF are nonnegative.

[0059] After obtaining the first matrix after feature matrix decomposition in step S11, hierarchical clustering can be performed on the column vectors in the first matrix in step S12 to obtain the label tree of the first matrix. The label tree is a hierarchical structure defined by branching relationships, and has multiple layers. Each layer may include clustering labels obtained based on hierarchical clustering.

[0060] Figure 2 A schematic diagram of a tag tree according to an embodiment of the present disclosure is shown, such as Figure 2 As shown, the node at level n in the tag tree This represents the cluster label of a cluster at level n, for example, node L at level n. 1 _1、L 1 _2、L 1 _3 represents different clusters in the first level, and the node L in the second level 2 _1_1、L 2 _1_2、L 2 _1_3 represents cluster 1 (which can correspond to a subset of samples G). j1 Different clusters in the second level under the branch, the nodes of the second level L 2 _2_1 represents cluster 2 (which can correspond to a subset of samples G). j2 The clusters in the second level under the branch, the nodes in the second level L 2 _3_1、L 2 _3_2 represents cluster 3 (which can correspond to a subset of samples G). j3 Different clusters in the second level under the branch, and nodes L in the third level. 3 _3_1_1、L 2 _3_1_2 represents different clusters in the third level under the third branch of cluster cluster 1 under the third branch of cluster cluster 3.

[0061] In the label tree, if a node has no further subdivisions (e.g., the clusters at the level above the node satisfy the level termination condition), that node can be considered a leaf node, and the cluster labels of the clusters at that node are the leaf labels. It should be understood that in practical applications, cluster labels can be represented as paths, such as L... 1 _3 / L2 _3_1 / L 2 _3_1_2, The query process of the tag tree is also a path traversal from the lower level to the higher level.

[0062] Hierarchical clustering, also known as recursive hierarchical clustering, involves recursively splitting the first matrix into rows based on the optimal classification of the columns (e.g., some rows belong to one cluster, and others to another). This process is repeated for each row within each cluster, and each cluster is further split into rows until the resulting clusters meet the hierarchical termination condition. This recursive splitting of the first matrix constructs a tree-like clustering structure, or label tree, which can be used to generate feature labels for new audio (e.g., second speech).

[0063] Each clustering label in the label tree can include a target feature index and the corresponding clustering model parameters. The target feature index represents the number of the optimal column in each clustering layer. The optimal column is the column with the highest quality score in the quality score of each column in the first matrix (or the clusters obtained from the previous layer's clustering process). The clustering model parameters are determined by the clustering results of the optimal column at that layer. For example, the clustering model can be used to fit the element distribution of the optimal column to determine the clustering model parameters.

[0064] Each clustering layer can employ clustering models such as the Dirichlet Process Gaussian Mixture Model (DP-GMM) or the Gaussian Mixture Model (GMM). The embodiments of this disclosure do not limit the specific type of clustering model used in each layer, and can be set according to the actual application scenario.

[0065] In step S13, the second speech can be projected according to the second matrix, and the components of the second speech in the direction of the second matrix can be calculated to obtain a speech feature representation vector with the same dimension as the number of columns of the first matrix, so as to improve the stability and consistency of subsequent feature label generation.

[0066] For example, for any new input second speech X, the projection of the second speech X onto the second matrix H can represent the component of the second speech X along the direction of the second matrix H. The result is a vector in the same direction as the second matrix H, that is, the speech feature representation vector W. new = (X·H) / |H|.

[0067] In step S13, a speech feature representation vector is obtained. In step S14, the speech feature representation vector is recursively calculated along the label tree from the root node downwards, and the clustering label of the speech feature representation vector of the second speech at each layer is calculated. Finally, the clustering label of the complete path of the speech feature representation vector of the second speech in the label tree is output as the feature label of the second speech. The clustering label of each layer is determined based on the clustering label of that layer.

[0068] The second matrix serves as the feature basis matrix after matrix eigenvalue decomposition. Subsequent new second speech only needs to be projected onto the space of the second matrix and the feature labels of the second speech can be inferred using the label tree. There is no need to add annotations or retrain the multi-path classifier, which helps to significantly reduce the amount of manual annotation and avoid retraining the model every time the label system is adjusted, thus reducing computing power and labor costs.

[0069] In this way, the embodiments of this disclosure provide a highly automated, easy-to-maintain, and scalable voice feature tag generation technology, which significantly reduces the time, manpower, and computing power costs of building and operating large-scale voice content understanding systems, improves the automation and robustness of tag generation, and has good versatility, interpretability, and evolvability. It can be widely applied to various voice content understanding systems, providing strong technical support for improving recommendation quality, retrieval efficiency, and user experience.

[0070] The method for generating voice tags according to embodiments of this disclosure will now be described in detail.

[0071] In one possible implementation, in step S11, multiple feature matrices of first speech can be obtained first. The method for obtaining the feature matrices of multiple first speech includes: preprocessing the multiple first speech to be processed to obtain multiple preprocessed speech; segmenting each preprocessed speech into sub-speech according to a preset speech activity detection model; encoding the sub-speech of each preprocessed speech to obtain the encoding vector of each sub-speech; and aggregating the encoding vectors of multiple sub-speech to obtain the feature matrix.

[0072] For example, M representative voice data content units can be sampled from the target application scenario (such as podcasts, audiobooks, intelligent customer service systems, etc.) as M first voices. Here, M is an integer greater than 1, which can be set according to the actual application scenario; the embodiments of this disclosure do not limit the specific value of M. The target application scenario is determined based on actual business needs and the specific requirements of the content understanding scenario; the embodiments of this disclosure do not limit the target application scenario.

[0073] The sampling method can take into account factors such as audio duration, topic diversity, speech style (e.g., speech rate, tone, emotional expression), and language diversity to ensure that the M sampled first speech samples are rich and representative. Specific sampling standards and methods can be flexibly adjusted and optimized according to the characteristics of the actual application scenario, and are not limited here.

[0074] M first speech samples are sampled from the target application scenario. These M first speech samples can be preprocessed, for example, by performing sampling rate normalization (e.g., to 16kHz), volume normalization (e.g., to -23LUFS), and audio denoising, to obtain M preprocessed speech samples. Audio denoising can employ spectral subtraction or a deep learning-based denoising model; the embodiments disclosed herein are not limited in this regard.

[0075] Then, Voice Activity Detection (VAD) can be performed on the M preprocessed speech samples. Using a pre-trained VAD model (e.g., a deep learning model), each preprocessed speech sample can be segmented into sub-speech segments. For example, the preprocessed speech (long audio) can be segmented into short, semantically meaningful clauses as sub-speech segments, so that the segmented sentences are as semantically complete as possible. Here, a short clause refers to a sentence composed of a single word or a very simple structure.

[0076] A general-purpose speech understanding pre-trained model (such as Whisper or Wav2Vec 2.0) can be used to encode sub-speech in each pre-processed speech, obtaining an encoding vector for each sub-speech. The selection criteria for the speech understanding model include: public availability, generalization ability to understand audio content, and validation performance on large-scale data. The embodiments of this disclosure do not limit the specific type of speech understanding pre-trained model.

[0077] After obtaining the encoding vector of each sub-speech in the M preprocessed speech, the encoding vectors of multiple sub-speech can be aggregated to obtain the feature matrix.

[0078] It should be understood that the preprocessing, clause segmentation, and speech content vectorization methods, specifically the standardization (such as sampling rate standardization and volume standardization), denoising algorithms (such as spectral subtraction or deep learning-based denoising models), speech activity detection algorithms, and speech understanding pre-trained models used for feature extraction or encoding, are all optional embodiments of this solution. The specific algorithm and model selected depends on factors such as the performance requirements of the actual system, the data scale, and computing resources, and the embodiments disclosed herein do not limit this.

[0079] In this way, silence segments and background noise interference can be effectively removed, focusing on sub-speech segments containing effective speech information. The sub-speech segments are encoded to generate vector representations. This fine-grained processing method is conducive to capturing richer acoustic features and semantic information, thereby improving the quality of the feature matrix.

[0080] In one possible implementation, the encoding vectors of multiple sub-speech segments in each preprocessed speech segment are pooled and aggregated to obtain a first aggregated vector corresponding to each preprocessed speech segment. The pooling and aggregation includes at least one of mean pooling, max pooling, minimum pooling, and attention pooling. The feature matrix is ​​generated based on the first aggregated vector.

[0081] Among them, mean pooling is used to capture the average features of preprocessed speech; max pooling is used to capture the most salient features of preprocessed speech; min pooling is used to capture the least salient but potentially meaningful features of preprocessed speech; and attention pooling, by introducing a self-attention mechanism, more accurately emphasizes clauses that are more critical to the overall content.

[0082] For example, mean pooling can be performed on the encoding vectors of multiple sub-speech segments in each preprocessed speech segment to obtain a mean pooling aggregated vector (i.e., the first aggregated vector) corresponding to each preprocessed speech segment, with dimensions (1, D). The mean pooling aggregated vectors corresponding to the M preprocessed speech segments are then combined into a feature matrix of (M, D). This feature matrix contains the average features of the preprocessed speech segments.

[0083] For example, the encoded vectors of multiple sub-speech segments in each preprocessed speech segment can be aggregated using max pooling to obtain a max pooling aggregated vector (i.e., the first aggregated vector) corresponding to each preprocessed speech segment, with dimensions (1, D). The max pooling aggregated vectors corresponding to the M preprocessed speech segments are then combined into a feature matrix of (M, D). This feature matrix contains the most salient features of the preprocessed speech segment.

[0084] For example, the encoded vectors of multiple sub-speech segments in each preprocessed speech segment can be aggregated using minimum pooling to obtain a minimum pooling aggregated vector (i.e., the first aggregated vector) corresponding to each preprocessed speech segment, with dimensions (1, D). The minimum pooling aggregated vectors corresponding to the M preprocessed speech segments are then combined into a feature matrix of (M, D). This feature matrix contains the least significant but potentially meaningful features from the preprocessed speech segments.

[0085] For example, attention pooling can be performed on the encoding vectors of multiple sub-speech units in each preprocessed speech to obtain an attention pooling aggregation vector (i.e., the first aggregation vector) corresponding to each preprocessed speech, with dimensions (1, D). The attention pooling aggregation vectors corresponding to the M preprocessed speech units are then merged into a feature matrix of (M, D). This feature matrix can more accurately emphasize clauses that are more critical to the overall content.

[0086] For example, the encoded vectors of multiple sub-speech units in each preprocessed speech unit can be aggregated using mean pooling, max pooling, min pooling, and attention pooling, respectively. This yields a first aggregated vector for each preprocessed speech unit, concatenated using the four pooling methods, with dimensions (1, 4×D). These first aggregated vectors from the M preprocessed speech units are then combined into a (M, 4×D) feature matrix, where different values ​​of D represent pooling vectors obtained using different pooling methods. This feature matrix includes the average features of the preprocessed speech, the most salient features, the least salient but potentially significant features, and clauses that more accurately emphasize the overall content.

[0087] It should be understood that the four aggregation methods—mean pooling, max pooling, min pooling, and attention pooling—can capture the average features, most salient features, least salient features, and key content features of speech, respectively. These four aggregation methods can be used individually or in combination, and other aggregation methods can be added or removed according to actual needs; no restrictions are placed here.

[0088] In this way, the feature representation of the audio itself can be obtained by combining multiple aggregation methods. The multi-pooling aggregation strategy fully preserves multi-dimensional signals such as speech rate, pitch, and emotion, thereby obtaining a higher quality feature matrix.

[0089] For example, mean pooling, max pooling, min pooling, and attention pooling can be performed on the encoding vectors of multiple sub-speech segments in each preprocessed speech, respectively, to obtain a one-dimensional vector of dimension D. Specifically, mean pooling yields the mean pooling vector for each preprocessed speech segment; max pooling yields the max pooling vector; min pooling yields the min pooling vector; and attention pooling yields the attention pooling vector.

[0090] The mean pooling vector, max pooling vector, minimum pooling vector, and attention pooling vector corresponding to each preprocessed speech are concatenated to form a final speech content vector representation with a dimension of 4×D, which serves as the first aggregation vector. Here, different D values ​​represent the content vectors obtained by different pooling methods.

[0091] This study introduces mean pooling, max pooling, min pooling, and attention pooling aggregation into the encoding vector. The mean pooling vector, max pooling vector, min pooling vector, and attention pooling vector corresponding to each preprocessed speech are concatenated. This multi-pooling aggregation strategy can fully preserve fine-grained features such as speech rate, intonation, and emotion, providing information-rich speech vectors while maintaining high accuracy and robustness on long-tailed and imbalanced data. It provides information-rich and robust input vectors for subsequent unsupervised clustering, significantly alleviating the problem of class dilution due to few samples, and improving clustering quality and label accuracy.

[0092] Aggregate the M first aggregation vectors with dimensions 4×D to obtain a feature matrix with dimensions (M, 4×D), where each row of the feature matrix represents a feature vector of a speech data content unit (e.g., including the first speech collected from the target application scenario).

[0093] To improve the effectiveness and stability of subsequent clustering algorithms, feature standardization or normalization is performed on the feature matrix to make the contributions of different feature dimensions to the clustering analysis more balanced. Standardization methods may include standard deviation standardization, zero-mean standardization, etc., while normalization methods may include average normalization, nonlinear normalization, min-max normalization, etc. The embodiments of this disclosure do not limit the specific standardization or normalization methods used.

[0094] The standardized feature matrix forms the initial cluster candidate set, labeled as feature matrix G0, which can be used for subsequent hierarchical cluster analysis.

[0095] Feature matrix decomposition can be performed on a feature matrix G0 of dimension (M, 4×D), including non-negative matrix factorization (NFF), which decomposes the feature matrix G0 into two non-negative matrices W and H, i.e.: G0 ≈ W·H, where W is a first matrix of dimension (M, K), representing the weights or coefficients of the original audio content units (e.g., including M first speech samples collected from the target application scenario) in the new feature space; H is a second matrix of dimension (K, 4×D), representing the basis vectors of K independent features, which are used for the reproduction and projection of feature vectors of subsequent new audio data.

[0096] The second matrix H can be retained as a feature basis matrix so that the feature vectors of new audio data can be projected and reproduced in the future system, so that newly added audio content units can reuse the same feature basis vectors, ensuring the stability and consistency of tag generation.

[0097] In step S11, the first matrix and the second matrix after feature matrix decomposition are obtained. In step S12, hierarchical clustering is performed on the column vectors in the first matrix to obtain the label tree of the first matrix.

[0098] In one possible implementation, step S12 may include: performing clustering processing on each column vector in the first matrix to determine the clustering result of each column vector, the clustering result including the clustering label vector and clustering model parameters corresponding to each column vector; determining the first column vector with the best clustering quality from multiple column vectors in the first matrix based on the quality score of the clustering result of each column vector in the first matrix; using the column number of the first column vector as the target feature index; and determining the clustering label of the first layer in the label tree based on the target feature index and the clustering model parameters corresponding to the first column vector.

[0099] For example, for each column vector W of the first matrix W with dimensions (M, K), k For k∈{1,2,…,K}, perform clustering sequentially to determine each column vector W. k The clustering results are shown. Here, column vector W1 represents the first column vector of the first matrix W, column vector W2 represents the second column vector of the first matrix W, and so on, with column vectors W... K Let K represent the Kth column vector of the first matrix W.

[0100] Optionally, a Bayesian variant of the Gaussian Mixture Model (GMM), namely the Dirichlet Process Gaussian Mixture Model (DP-GMM), can be used to automatically infer the optimal number of clusters for clustering. Among them, the optimal number of clusters Cmax is an integer less than K, where K represents the total number of columns in the first matrix W. The optimal number of clusters is calculated. This can reduce the burden of manual parameter selection.

[0101] Then, based on the optimal number of clusters For each column vector W in the first matrix W k Clustering is performed on the M rows of elements to obtain each column vector W. k The clustering results, which may include each column vector W kThe corresponding clustering label vector C k and clustering model parameters θ k = {mean μc, variance σc², cluster weights} }, where the cluster label vector C k The dimension is (M, 1), and the column vector is W. k Each element in the M rows of elements can be derived from the clustering label vector C. k The corresponding labels were found in the data, where μc represents the mean of the clustering model, σc² represents the variance of the clustering model, and πc represents the cluster weights of the clustering model. This represents the optimal number of clusters.

[0102] For each column vector W of the first matrix W k The clustering results are used to evaluate and score the clustering quality, based on each column vector W in the first matrix W. k The quality score of the clustering results is derived from multiple column vectors W in the first matrix W. k The first column vector with the best clustering quality is determined in the middle. .

[0103] In one possible implementation, the quality score is determined by a scoring function that includes a silhouette coefficient and / or normalized information entropy. The silhouette coefficient measures the intra-cluster compactness and inter-cluster separation of the clustering process, and the normalized information entropy measures the balance of cluster size.

[0104] By combining silhouette coefficients and normalized information entropy, the optimal feature dimension with "high separation and balanced cluster size" can be automatically selected on a single feature dimension. Compared with the drawbacks of clustering relying on manual experience for dimension selection, this method can improve the automation of hierarchical clustering and the interpretability of labels, while maintaining high accuracy and robustness on long-tailed and imbalanced data.

[0105] For example, each column vector W can be computed. k The silhouette score and normalized information entropy of the clustering results are used to determine the W of each column vector. k The quality score of the clustering results.

[0106] The silhouette coefficient s(k) measures intra-class compactness and inter-class separation. It is calculated based on the average distance a(i) between samples within a class and the average distance b(i) from a sample to the nearest other class. The silhouette coefficient s(k) is obtained by the formula (b(i)-a(i)) / max(a(i),b(i)), where max represents the maximum value function, and s(k) takes values ​​in the range [-1,1]. When s(k) approaches 1, it indicates that the sample clustering effect is better.

[0107] The normalized information entropy e(k) is used to measure the uniformity of cluster size and can be expressed as:

[0108] (1)

[0109] Where e(k) represents the normalized information entropy, and k represents any column in the first matrix W, k∈{1,2,…,K}. This represents the optimal number of clusters, which is the total number of clusters in the clustering process.

[0110] The scoring function Q(k) can be used to determine each column vector W. k The quality score of the clustering results, i.e.:

[0111] (2)

[0112] Where s(k) represents the silhouette coefficient, and e(k) represents the normalized information entropy. As a balance coefficient, it is recommended that the value of λ be in the range of 0.3 to 0.5. However, the embodiments of this disclosure do not impose any restrictions on the specific value of λ.

[0113] It should be understood that the selection of specific evaluation indicators and methods can also be changed according to actual needs. For example, other clustering quality evaluation methods such as the Kalinsky-Hallabus index (CH index) can be added, but this is not limited here.

[0114] A higher score in the scoring function Q(k) indicates better cluster separation without excessive imbalance. This can be determined based on each column vector W in the first matrix W. k The quality score Q(k) of the clustering results is used to find the index of the optimal column vector (i.e., the first column vector). The argmax function is used to find the K column vectors W. k Column numbering of the maximum mass fraction Q(k) .

[0115] Number the columns of the first column vector As the index of the target feature in the first-level cluster labels of the label tree, the number will be... Indicating column vector The cluster label vector in the corresponding clustering results The first-level cluster label vector is denoted as... Number Indicating column vector Clustering model parameters in the corresponding clustering results , which serves as the clustering model parameter in the first-level clustering labels of the label tree.

[0116] For example, persistent clustering model parameters ,Will Save this as the system's first-level clustering configuration Profile_1 for subsequent processing of newly added audio vectors W. new After projection, the clustering model parameters are directly used. Calculate its cluster labels This ensures the consistency and scalability of the labeling system.

[0117] In one possible implementation, hierarchical clustering is performed on the column vectors in the first matrix to obtain a label tree for the first matrix. This includes: performing clustering processing on each column vector in the sample subset corresponding to the clusters obtained from the previous layer of clustering processing, determining the clustering result for each column vector, wherein the clustering result includes a cluster label vector and clustering model parameters corresponding to each column vector, and the clusters obtained from the previous layer of clustering processing are determined by the cluster label vector corresponding to the column vector with the best clustering quality from the previous layer; determining the second column vector with the best clustering quality from multiple column vectors in the sample subset corresponding to the cluster based on the quality score of the clustering result of each column vector in the sample subset corresponding to the cluster; using the column number of the second column vector as a target feature index; and determining the cluster label of the current layer in the label tree based on the target feature index and the clustering model parameters corresponding to the second column vector.

[0118] For example, consider the column vectors W in the first matrix W. k The process of hierarchical clustering is to continuously search for the optimal sub-labels at the next lower level after obtaining the clustering results of the first level.

[0119] In one possible implementation, clustering is performed on each column vector in the cluster obtained from the previous layer clustering process to determine the clustering result of each column vector. This includes: if the maximum value of the quality score of the clustering result of each column vector in the previous layer clustering process is greater than or equal to a first preset threshold, or if the sample size in the cluster obtained from the previous layer clustering process is greater than or equal to a second preset threshold, then clustering is performed on each column vector in the cluster obtained from the previous layer clustering process to determine the clustering result of each column vector.

[0120] For example, a comprehensive score threshold Q_thresh (e.g., 0.35) can be set as the first preset threshold, and a minimum sample size threshold Mmin can be set as the second preset threshold. The clustering operation of each layer can be determined based on the first preset threshold Q_thresh and the second preset threshold Mmin to determine whether the layer termination condition is met.

[0121] If the maximum value of the quality score of the clustering result of each column vector in the previous clustering process, max(Q(k)), is greater than or equal to the first preset threshold Q_thresh, or if the sample subset G corresponding to the clusters obtained in the previous clustering process is... j If the sample size in the current cluster is greater than or equal to the second preset threshold Mmin, and the hierarchical termination condition is not met, then the sample subset G of the current cluster can be processed. j The process continues recursively to the next level of clustering, that is, it continues to process the clusters obtained from the previous level to obtain the sample subset G corresponding to the clusters. j Each column vector W in k(j) Perform clustering to determine each column vector W k(j) The clustering results.

[0122] If the maximum quality score max(Q(k)) of the clustering result of each column vector in the previous clustering process is less than the first preset threshold Q_thresh, and the sample subset G corresponding to the clusters obtained by the previous clustering process is... j If the sample size is less than the second preset threshold Mmin, it means that the hierarchical termination condition is met. The clustering results of the previous layer will not continue to be recursively clustered. The sub-labels of its clustering results are the leaf nodes of the entire hierarchical label tree.

[0123] It should be understood that the termination conditions and the specific number of clustering layers in recursive clustering can be flexibly adjusted according to the application scenario, and the specific termination conditions and the selection method of the layer depth are not limited here.

[0124] By setting hierarchical termination conditions, excessive subdivision during the hierarchical clustering process can be reduced.

[0125] For example, if the clustering result of the previous level does not meet the hierarchical termination condition, each cluster S obtained at the previous level can be further processed. j A subset of samples (j = 1, 2, ..., Nc-prev) Repeat the first-level clustering operation. The clusters S obtained from the previous level of clustering are... j It is the column vector with the best clustering quality in the previous layer. Corresponding cluster label vector It's confirmed.

[0126] For example, suppose the first matrix W in the first level is The target feature index at this level is also the number of the optimal column. Column vector W2 = [D12 D22 D32 D42] T The corresponding cluster label vector C2=[1 2 2 1] TTherefore, the first level yields cluster S1 with label 1 and cluster S2 with label 2. The sample subset G1 corresponding to cluster S1 is... The sample subset G2 corresponding to cluster S2 is It should be understood that the embodiments disclosed herein are merely examples and do not limit the specific form of the first matrix W.

[0127] For example, G can be re-evaluated. j Each column vector W in k(j) Run the Dirichlet Process Gaussian Mixture Model (DP-GMM) to determine the sample subset G. j The optimal number of clusters.

[0128] Then, based on the optimal number of clusters, the clusters S obtained from the previous layer of clustering can be processed. j The corresponding sample subset G j Each column vector W in k(j) Perform clustering to determine each column vector W k(j) The clustering results, which may include each column vector W k(j) The corresponding clustering label vector C j and clustering model parameters θ k(j) .

[0129] Formula (2) can be used to analyze the cluster S. j The corresponding sample subset G j Each column vector W k(j) The clustering results are used to evaluate and score the clustering quality, and a new quality score Q(k) is calculated so that the clusters S can be analyzed. j The corresponding sample subset G j Determine the second column vector with the best clustering quality from the K column vectors. , the second column vector Column number The second column vector serves as the index of the target feature in the cluster labels of the current layer (i.e., the layer below the previous layer) in the label tree. The cluster label vector in the corresponding clustering results As the clustering label vector of the current layer (i.e., the layer below the previous layer) in the label tree, the second column vector Clustering model parameters in the clustering results As a clustering model parameter in the clustering labels of the current layer (i.e., the layer below the previous layer) in the label tree.

[0130] The clustering labels generated recursively during hierarchical clustering, including the target feature index of each layer and the corresponding clustering model parameters, can be organized into a path-named label tree structure. Each clustering model can be persistently stored as a model dictionary `Profiles = { path → (k, θ)}`, and bound to the clustering label of each node. Here, `path` represents the path, `k` represents the target feature index, and `θ` represents the clustering model parameters.

[0131] In this way, the optimal number of clusters can be automatically determined based on recursive hierarchical clustering and a scoring function. Within each cluster, recursive clustering continues until the hierarchical termination condition is met, forming a scalable multi-level labeling system, i.e., a label tree. This eliminates the need for manual label design and the reliance on manually preset label boundaries, facilitating a low-cost, dynamically evolving speech labeling system. For example, when business scenarios change or new speech styles emerge, there is no need to redesign the label set. The structural characteristics of the label tree can be used to adaptively subdivide or merge labels, avoiding repeated manual sorting and large-scale relabeling.

[0132] Optionally, in step S13, the second input speech can be projected onto the second matrix H to obtain a speech feature representation vector W with the same dimension and number of columns as the first matrix W. new For example, the preprocessing and feature vectorization process described above can be encapsulated into a standardized preprocessing module. This module supports mapping any input second speech to a 4×D vector representation, and further projecting it onto a second matrix H to obtain a K-dimensional speech feature representation vector W. new .

[0133] For example, when a new second speech segment (such as a podcast audio clip) is acquired, it can be preprocessed and vectorized. For instance, the second speech first undergoes sampling rate and volume normalization, as well as noise reduction. Then, it is segmented into several semantically complete clauses using Voice Activity Detection (VAD). Each clause is encoded into a clause vector using a speech understanding model (such as Whisper). These clause vectors are then combined using four aggregation methods (e.g., mean, max, minimum, and attention pooling) to form a 4×D whole speech vector. This speech vector can then be projected using a second matrix H to obtain a K-dimensional audio feature vector W. new .

[0134] In step S13, the speech feature representation vector W is obtained. new In step S14, the speech feature representation vector W can be... new The clustering labels at each layer of the label tree serve as feature labels for the second speech.

[0135] In one possible implementation, step S14 may include: extracting target elements from the speech feature representation vector at target positions for the cluster labels of the i-th layer in the label tree, in order from low to high levels, wherein the target position is determined by the target feature index in the i-th layer cluster labels, and the i-th layer indicates that any layer i in the label tree is a positive integer less than or equal to P; where P is the total number of layers in the label tree; inputting the target element into a target clustering model to obtain the target labels of the i-th layer, wherein the target clustering model is determined by the clustering model parameters of the i-th layer cluster labels; and using the concatenation result of the speech feature representation vector of the target labels in each layer of the label tree as the feature labels of the second speech.

[0136] For example, starting from the root node of the label tree, the first-level cluster labels can be read. ,in, This represents the target feature index at the first level. This represents the parameters of the first-level clustering model. The extracted audio feature vector W... new The Middle Target element at dimensional position Substitute the target clustering model to obtain the target label L¹ of the first level; then enter the child node corresponding to L¹ in the label tree, repeat the process, and calculate the second level, the third level, etc., until a leaf node is reached or the level termination condition is met.

[0137] The target labels obtained from each layer of reasoning can be concatenated into a complete path-like label representation, which can be used as the feature labels of the second speech, for example: L¹_3 / L²_3_1 / L³_3_1_2.

[0138] In this way, the target label of the second speech can be recursively calculated from the root node down the label tree. The complete path label of the second speech in the label tree is output as the feature label of the second speech. The path can represent the complete label chain to which the second speech belongs in the label tree structure, and can describe the high-order speech content features such as the core speech style, rhythm characteristics, and expression mode of the second speech.

[0139] Optionally, the methods described above can be encapsulated into components and assigned a unified version number, such as: feature extraction model version (e.g., Whisper-v2, NMF configuration), clustering algorithm and label tree version (TagTree_vX.Y), training dataset version, and generation date, for management and traceability. The construction and encapsulation of the label tree inference system may include the encapsulation of a standardized preprocessing module, the integration of the label tree model structure, and the implementation of the inference execution module; these are all optional embodiments of this solution. Specific encapsulation methods, modular design, and version control management methods can be flexibly selected according to the needs of the actual application system and software architecture; this disclosure does not limit these aspects.

[0140] It should be understood that the order and method of the steps involved in generating and reasoning the feature labels for the new second speech are preprocessing and vectorization, feature projection, label tree reasoning, and feature label output, which are part of the best practices for implementing this solution. However, in practice, the execution order, parallel method, or distributed implementation method of these steps can be flexibly adjusted according to the performance requirements and computing resources of the actual application scenario, and the embodiments disclosed herein do not limit this.

[0141] In summary, compared to related technologies, which require defining a complete labeling system, performing large-scale manual annotation, and training multi-path classification models, the entire process often takes weeks to months. The speech label generation method of this disclosure automatically generates a multi-level label tree through recursive hierarchical clustering. Only one offline modeling step is needed to support real-time labeling of subsequent new audio, shortening the time to launch a single new label from weeks to within hours, and significantly reducing the feature label generation cycle. Furthermore, compared to related technologies, which rely on hundreds or thousands of hours of manually annotated corpora, re-collection or re-annotation is required when the labeling system is expanded or modified. The speech label generation method of this disclosure can utilize unsupervised or weakly supervised clustering methods to automatically discover label boundaries and maintain consistency in the feature base space (e.g., the space where the second matrix is ​​located), saving a significant amount of manual annotation work and greatly reducing the manpower and computing power required for label maintenance and model retraining.

[0142] Furthermore, the multi-pooling aggregation strategy fully preserves multi-dimensional signals such as speech rate, tone, and emotion. Each layer can adaptively determine the optimal number of clusters using a clustering model (such as the DP-GMM model), and the optimal feature dimension is selected by combining the silhouette coefficient and information entropy as dual indicators. In podcast and audiobook testing, this scheme significantly improves label accuracy and robustness compared to purely supervised model schemes. For example, in actual testing, the average label consistency rate of this scheme is significantly improved (e.g., 8-15 percentage points), with particularly noticeable improvement in scores for long-tail categories.

[0143] Moreover, the second matrix of the feature matrix and the label tree can be stored with version numbers. New audio or new business domains (such as the second voice) only need to be inferred after projection into the space where the existing second matrix is ​​located, without having to rerun the full clustering. When a new voice style appears in the content ecosystem, incremental clustering can be performed in a local subtree and spliced ​​to the original label tree, ensuring the stability of old labels and the zero-intrusion deployment of new labels, thus enhancing the system's scalability and evolution capabilities.

[0144] Previously, since preprocessing, vectorization, and inference could all be implemented in streaming or batch parallelism, the end-to-end tag latency was very small in the multi-core GPU test environment, meeting the real-time requirements of online intelligent customer service scenarios and greatly improving throughput.

[0145] In summary, the embodiments of this disclosure provide a highly automated, easy-to-maintain, and scalable speech feature tag generation technology, which significantly reduces the time, manpower, and computing power costs of building and operating large-scale speech content understanding systems, while improving tag quality and system real-time response capabilities, demonstrating outstanding technological advancement and industrial application value.

[0146] It is understood that the various method embodiments mentioned above in this disclosure can be combined with each other to form combined embodiments without violating the principle and logic. Due to space limitations, this disclosure will not elaborate further. Those skilled in the art will understand that in the above methods of specific implementation, the specific execution order of each step should be determined by its function and possible internal logic.

[0147] In addition, this disclosure also provides a voice tag generation apparatus, electronic device, computer-readable storage medium, and program, all of which can be used to implement any of the voice tag generation methods provided in this disclosure. The corresponding technical solutions and descriptions are described in the corresponding descriptions in the method section and will not be repeated here.

[0148] Figure 3 A block diagram of a voice tag generation apparatus according to an embodiment of the present disclosure is shown, such as Figure 3 As shown, the device includes:

[0149] The decomposition module 31 is used to decompose the feature matrices of the multiple first speech to be processed to obtain the decomposed first matrix and second matrix. Each row in the feature matrix corresponds to the feature vector of different first speech. The number of rows in the first matrix is ​​the same as the number of first speech, and the number of columns in the first matrix is ​​the same as the number of rows in the second matrix.

[0150] Clustering module 32 is used to perform hierarchical clustering on the column vectors in the first matrix to obtain a label tree of the first matrix, wherein the label tree has multiple layers; each layer includes clustering labels obtained based on the hierarchical clustering.

[0151] Projection module 33 is used to project the second speech according to the second matrix to obtain a speech feature representation vector with the same dimension as the number of columns of the first matrix;

[0152] The query module 34 is used to query the tag tree to determine the feature label of the second speech; wherein the feature label of the second speech is determined based on the clustering label in the tag tree mapped by the speech feature representation vector.

[0153] In one possible implementation, the clustering module 32 is configured to: perform clustering processing on each column vector in the first matrix to determine the clustering result of each column vector, the clustering result including the clustering label vector and clustering model parameters corresponding to each column vector; determine the first column vector with the best clustering quality from multiple column vectors in the first matrix based on the quality score of the clustering result of each column vector in the first matrix; use the column number of the first column vector as the target feature index; and determine the clustering label of the first layer in the label tree based on the target feature index and the clustering model parameters corresponding to the first column vector.

[0154] In one possible implementation, the clustering module 32 is configured to: perform clustering processing on each column vector in the sample subset corresponding to the cluster obtained from the previous layer clustering processing, determine the clustering result of each column vector, the clustering result including the cluster label vector and clustering model parameters corresponding to each column vector, wherein the cluster obtained from the previous layer clustering processing is determined by the cluster label vector corresponding to the column vector with the best clustering quality from the previous layer; determine the second column vector with the best clustering quality from multiple column vectors in the sample subset corresponding to the cluster based on the quality score of the clustering result of each column vector in the sample subset corresponding to the cluster; use the column number of the second column vector as the target feature index; and determine the cluster label of the current layer in the label tree based on the target feature index and the clustering model parameters corresponding to the second column vector.

[0155] In one possible implementation, clustering is performed on each column vector in the sample subset corresponding to the cluster obtained by the previous layer clustering process to determine the clustering result of each column vector. This includes: if the maximum value of the quality score of the clustering result of each column vector in the previous layer clustering process is greater than or equal to a first preset threshold, or if the sample size in the cluster obtained by the previous layer clustering process is greater than or equal to a second preset threshold, then clustering is performed on each column vector in the cluster obtained by the previous layer clustering process to determine the clustering result of each column vector.

[0156] In one possible implementation, the quality score is determined by a scoring function that includes a silhouette coefficient and / or normalized information entropy. The silhouette coefficient measures the intra-cluster compactness and inter-cluster separation of the clustering process, and the normalized information entropy measures the balance of cluster size.

[0157] In one possible implementation, the query module 34 is configured to: extract target elements from the speech feature representation vector at target positions for the cluster labels of the i-th layer in the label tree, in order from low to high levels, wherein the target position is determined by the target feature index in the i-th layer cluster labels, and the i-th layer indicates that any layer i in the label tree is a positive integer less than or equal to P; wherein P is the total number of layers in the label tree; input the target element into a target clustering model to obtain the target label of the i-th layer, wherein the target clustering model is determined by the clustering model parameters of the i-th layer cluster labels; and use the concatenation result of the speech feature representation vector of the target labels in each layer of the label tree as the feature label of the second speech.

[0158] In one possible implementation, the device further includes an acquisition module for acquiring the feature matrix of the plurality of first speech samples, comprising: preprocessing the plurality of first speech samples to be processed to obtain a plurality of preprocessed speech samples; segmenting each preprocessed speech sample into sub-speech samples according to a preset speech activity detection model; encoding the sub-speech samples of each preprocessed speech sample to obtain an encoding vector for each sub-speech sample; and aggregating the encoding vectors of the plurality of sub-speech samples to obtain the feature matrix.

[0159] In one possible implementation, the feature matrix is ​​obtained by aggregating the encoding vectors of multiple sub-speech segments, including: pooling the encoding vectors of multiple sub-speech segments in each preprocessed speech segment to obtain a first aggregated vector corresponding to each preprocessed speech segment, wherein the pooling aggregation includes at least one of mean pooling aggregation, max pooling aggregation, minimum pooling aggregation, and attention pooling aggregation; and generating the feature matrix based on the first aggregated vector.

[0160] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0161] This disclosure also proposes a computer-readable storage medium storing computer program instructions that, when executed by a processor, implement the above-described method. The computer-readable storage medium can be volatile or non-volatile.

[0162] This disclosure also proposes an electronic device, including: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to execute the above-described method.

[0163] This disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device performs the above-described method.

[0164] Electronic devices can be provided as terminals, servers, or other forms of devices.

[0165] Figure 4 A block diagram of an electronic device 1900 according to an embodiment of the present disclosure is shown. For example, the electronic device 1900 may be provided as a server or a terminal device. (Refer to...) Figure 4 The electronic device 1900 includes a processing component 1922, which further includes one or more processors, and memory resources represented by memory 1932 for storing instructions, such as application programs, that can be executed by the processing component 1922. The application programs stored in memory 1932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 1922 is configured to execute instructions to perform the methods described above.

[0166] Electronic device 1900 may also include a power supply component 1926 configured to perform power management of electronic device 1900, a wired or wireless network interface 1950 configured to connect electronic device 1900 to a network, and an input / output interface 1958. Electronic device 1900 can operate on an operating system stored in memory 1932, such as a Microsoft Server operating system (Windows Server). TM Apple's graphical user interface-based operating system (Mac OS X) TM ), a multi-user, multi-process computer operating system (Unix) TM Linux is a free and open-source Unix-like operating system. TM ), an open-source Unix-like operating system (FreeBSD) TM (or similar.)

[0167] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by a processing component 1922 of an electronic device 1900 to perform the above-described method.

[0168] This disclosure can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this disclosure.

[0169] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, (but not limited to) electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0170] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0171] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0172] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0173] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0174] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0175] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0176] The computer program product can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0177] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0178] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0179] If the technical solution of this application involves personal information, the product using this technical solution has clearly informed the user of the personal information processing rules and obtained the user's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using this technical solution has obtained the user's separate consent before processing the sensitive personal information, and also meets the requirement of "express consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set up to inform users that they have entered the scope of personal information collection and that personal information will be collected. If an individual voluntarily enters the collection scope, it is deemed that they have agreed to the collection of their personal information; or on the personal information processing device, with clear signs / information informing users of the personal information processing rules, authorization is obtained from the individual through pop-up information or by asking the individual to upload their personal information; wherein, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

[0180] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for generating voice tags, characterized in that, The method includes: The feature matrices of the multiple first speech samples to be processed are decomposed into a first matrix and a second matrix. Each row in the feature matrix corresponds to a feature vector of a different first speech sample. The number of rows in the first matrix is ​​the same as the number of first speech samples, and the number of columns in the first matrix is ​​the same as the number of rows in the second matrix. Hierarchical clustering is performed on the column vectors in the first matrix to obtain a label tree of the first matrix, the label tree having multiple layers; each layer includes clustering labels obtained based on the hierarchical clustering; The second speech is projected onto the second matrix to obtain a speech feature representation vector with the same dimension as the number of columns in the first matrix. The feature labels of the second speech are determined by querying the label tree; wherein the feature labels of the second speech are determined by clustering labels in the label tree mapped from the speech feature representation vector.

2. The method according to claim 1, characterized in that, Hierarchical clustering is performed on the column vectors in the first matrix to obtain the label tree of the first matrix, including: Clustering is performed on each column vector in the first matrix to determine the clustering result for each column vector. The clustering result includes the clustering label vector and clustering model parameters corresponding to each column vector. Based on the quality score of the clustering results of each column vector in the first matrix, determine the first column vector with the best clustering quality from multiple column vectors in the first matrix; Use the column number of the first column vector as the target feature index; Based on the target feature index and the clustering model parameters corresponding to the first column vector, the clustering labels of the first layer in the label tree are determined.

3. The method according to claim 1, characterized in that, Hierarchical clustering is performed on the column vectors in the first matrix to obtain the label tree of the first matrix, including: Clustering is performed on each column vector in the sample subset corresponding to the cluster obtained from the previous layer clustering process to determine the clustering result of each column vector. The clustering result includes the cluster label vector and clustering model parameters corresponding to each column vector. The cluster obtained from the previous layer clustering process is determined by the cluster label vector corresponding to the column vector with the best clustering quality from the previous layer. Based on the quality score of the clustering result of each column vector in the sample subset corresponding to the cluster, determine the second column vector with the best clustering quality from multiple column vectors in the sample subset corresponding to the cluster; Use the column number of the second column vector as the target feature index; Based on the target feature index and the clustering model parameters corresponding to the second column vector, the clustering label of the current layer in the label tree is determined.

4. The method according to claim 3, characterized in that, Clustering is performed on each column vector in the sample subset corresponding to the clusters obtained from the previous layer of clustering to determine the clustering result for each column vector, including: If the maximum quality score of the clustering result of each column vector in the previous layer of clustering is greater than or equal to the first preset threshold, or if the sample size in the cluster obtained by the previous layer of clustering is greater than or equal to the second preset threshold, then each column vector in the cluster obtained by the previous layer of clustering is subjected to clustering to determine the clustering result of each column vector.

5. The method according to any one of claims 2 to 4, characterized in that, The quality score is determined by a scoring function, which includes a silhouette coefficient and / or normalized information entropy. The silhouette coefficient is used to measure the intra-cluster compactness and inter-cluster separation of the clustering process, and the normalized information entropy is used to measure the balance of cluster size.

6. The method according to claim 2 or 3, characterized in that, Querying the label tree to determine the feature labels of the second speech includes: Following the order from low to high levels, for the cluster label of the i-th layer in the label tree, the target element is extracted from the target position in the speech feature representation vector. The target position is determined by the target feature index in the cluster label of the i-th layer. The i-th layer means that any layer i in the label tree is a positive integer less than or equal to P. P is the total number of layers in the label tree. The target element is input into the target clustering model to obtain the target label of the i-th layer. The target clustering model is determined by the clustering model parameters of the i-th layer clustering label. The concatenation result of the speech feature representation vector at each layer of the label tree is used as the feature label of the second speech.

7. The method according to claim 1, characterized in that, The methods for obtaining the feature matrices of the multiple first speech sounds include: Multiple first-speech samples to be processed are preprocessed to obtain multiple preprocessed speech samples; Each pre-processed speech is segmented into sub-speech based on a pre-defined speech activity detection model; Encode each sub-speech of the preprocessed speech to obtain the encoding vector of each sub-speech; The feature matrix is ​​obtained by aggregating the encoded vectors of multiple sub-speech words.

8. The method according to claim 7, characterized in that, The feature matrix is ​​obtained by aggregating the encoded vectors of multiple sub-speech words, including: Pooling is performed on the encoding vectors of multiple sub-speech in each preprocessed speech to obtain the first aggregated vector corresponding to each preprocessed speech. The pooling aggregation includes at least one of mean pooling aggregation, maximum pooling aggregation, minimum pooling aggregation, and attention pooling aggregation. The feature matrix is ​​generated based on the first aggregation vector.

9. A device for generating voice tags, characterized in that, include: The decomposition module is used to decompose the feature matrices of multiple first speech samples to be processed, and obtain the decomposed first matrix and second matrix. Each row in the feature matrix corresponds to the feature vector of different first speech samples. The number of rows in the first matrix is ​​the same as the number of first speech samples, and the number of columns in the first matrix is ​​the same as the number of rows in the second matrix. A clustering module is used to perform hierarchical clustering on the column vectors in the first matrix to obtain a label tree of the first matrix, wherein the label tree has multiple layers; each layer includes clustering labels obtained based on the hierarchical clustering. The projection module is used to project the second speech according to the second matrix to obtain a speech feature representation vector with the same dimension as the number of columns of the first matrix; The query module is used to query the tag tree to determine the feature labels of the second speech; wherein the feature labels of the second speech are determined based on the clustering labels in the tag tree mapped by the speech feature representation vector.

10. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 8.

Images