A speech classification method and device for group level mapping network

Abstract

The present invention discloses a speech classification method and device of a group-level mapping network. By utilizing kernel density estimation and sparse coding technology, multi-perspective sparse distribution features are extracted from the original speech signal, and the intrinsic structure and characteristics of the speech signal are comprehensively described according to the multi-perspective sparse distribution features. In combination with multi-perspective sparse distribution and cross-sample learning methods, a group-level mapping network is constructed to further extract a speechomics feature set, improve the group representativeness and prediction performance of the features, enhance the generalization ability and prediction accuracy of the features, and at the same time, select a suitable machine learning model according to a specific classification task to ensure the matching of the selected model with the classification task, thereby improving the accuracy and efficiency of classification, so as to complete the classification task of speech acoustic features by utilizing the extracted speechomics feature set and the matching machine learning model, and improve the classification accuracy of speech acoustic features.

Description

Technical Field

[0001] The present invention relates to the field of speech processing technology, and in particular to a speech classification method and device of a group level mapping network. Background Art

[0002] Human speech production is the result of fine-tuning and feedback at three cascade levels: the brain's language centers, peripheral neuromuscular effectors, and the vocal tract. This not only gives speech the potential to be a potential digital biomarker, but also suggests that we need a hyperdimensional decomposition method to identify the smallest components embedded in the "brain-neuromuscular-vocal tract" axis that can represent a wide range of paralinguistic features.

[0003] With the continuous development of science and technology and the gradual maturity of digital medical technology, the use of telemedicine and digital biomarkers has become more and more common. In this context, telemedicine, as a new form of health care, has been widely used in the global public health crisis. Providing health care through remote devices such as event monitors, smart devices and wearable devices has brought many benefits in the health management of various diseases including hypertension and heart failure. In recent years, satisfactory progress has been made in extracting biomarkers from voice and using machine learning technology for prediction and monitoring to assist medical analysis. Voice biomarkers have potential in providing health care remotely. By analyzing the voice characteristics of individuals, auxiliary information can be provided to medical professionals to help them detect, progress and make treatment decisions for diseases more accurately.

[0004] Existing speech signal feature extraction and conversion methods are mostly limited to paralinguistic representation at the individual level. The annotation of individual states is mostly concentrated on a single influencing factor, while ignoring other states that potentially affect the comprehensive paralinguistic features outside of this influencing factor. As a result, some speech features may reflect multiple influencing factors at the same time, but it is impossible to decompose the features to obtain the smallest component representing the broad paralinguistic features, resulting in inaccurate speech feature classification. Summary of the invention

[0005] The present invention provides a speech classification method and device of a group level mapping network to improve the accuracy of speech acoustic feature classification.

[0006] In order to solve the above technical problems, an embodiment of the present invention provides a speech classification method of a group level mapping network, comprising:

[0007] Acquire a speech signal, and construct a multi-view sparse distribution of the speech signal based on kernel density estimation and sparse coding;

[0008] Constructing a group-level mapping network based on the multi-view sparse distribution and cross-sample learning, and acquiring a phonemic feature set based on the group-level mapping network;

[0009] A classification task is obtained, a machine learning model is matched based on the classification task, and speech acoustic feature classification is performed based on the machine learning model and the speechomics feature set.

[0010] This scheme uses kernel density estimation and sparse coding technology to extract multi-perspective sparse distribution features from the original speech signal, and comprehensively describes the intrinsic structure and characteristics of the speech signal based on the multi-perspective sparse distribution features. In combination with multi-perspective sparse distribution and cross-sample learning methods, a group-level mapping network is constructed to further extract the speech omics feature set, improve the group representativeness and prediction performance of the features, and enhance the generalization ability and prediction accuracy of the features. At the same time, a suitable machine learning model is selected according to the specific classification task to ensure the matching of the selected model with the classification task, thereby improving the accuracy and efficiency of classification, so as to complete the classification task of speech acoustic features using the extracted speech omics feature set and the matching machine learning model, and improve the classification accuracy of speech acoustic features.

[0011] Furthermore, constructing the multi-view sparse distribution of the speech signal based on kernel density estimation and sparse coding includes:

[0012] Extracting features from the speech signal based on preset parameters to obtain a speech feature set;

[0013] The speech feature set is screened based on the kernel density estimation and the vowel formant to obtain the first layer of speech features;

[0014] Iteratively operate the first layer of speech features based on preset acoustic indicators to obtain second layer of speech features;

[0015] A multi-view sparse distribution is acquired based on a preset feature range, a preset activation strength, the first-layer speech features, and the second-layer speech features.

[0016] The present invention extracts initial speech features from the original speech signal, and further optimizes the feature set through kernel density estimation and resonance peak screening, reduces noise and redundant information, improves the purity and representativeness of the features, removes abnormal data that does not conform to the resonance peak theoretical range, and improves the quality of the features; at the same time, based on the optimized first-layer speech features, higher-level features are generated through iterative operations to increase the diversity and complexity of the features, and sparse distributed representation is generated through fixed feature range and activation strength calculation, thereby enhancing the generalization ability and classification accuracy of the model and providing support for subsequent machine learning tasks.

[0017] Furthermore, the step of acquiring multi-view sparse distribution based on a preset feature range, a preset activation strength, the first-layer speech features, and the second-layer speech features includes:

[0018] Acquire a plurality of continuous sub-feature spaces based on the feature range, and screen the number of features of target features in each sub-feature space; the target features include first-layer speech features or second-layer speech features;

[0019] Normalizing the sub-feature space based on the number of features to obtain activation strength of each sub-feature space;

[0020] Acquire a first SDR form of the first layer of speech features and a second SDR form of the second layer of speech features based on the sub-feature space and the activation strength;

[0021] The first SDR form and the second SDR form are spliced ​​to generate the multi-view sparse distribution.

[0022] The present invention optimizes the feature set, reduces noise and redundant information, improves the purity and representativeness of features, removes abnormal data that does not meet the intensity threshold, and improves the quality of features through sub-feature space division and feature screening. The activation intensity of different sub-feature spaces is made comparable according to normalization processing to generate sparse distributed representations of the first-layer speech features and the second-layer speech features, optimize the feature structure, improve the expressiveness of features and the performance of the model, and generate multi-perspective sparse distribution by splicing features at different levels to provide optimized feature input for subsequent machine learning tasks.

[0023] Furthermore, constructing a group-level mapping network based on the multi-view sparse distribution and cross-sample learning, and acquiring a speechomics feature set based on the group-level mapping network, includes:

[0024] Construct a group-level mapping network based on cross-sample learning using principal component analysis;

[0025] The principal component matrix of the multi-view sparse distribution is obtained based on the group-level mapping network, and the phonemic feature set is obtained based on the principal component matrix.

[0026] The present invention uses the principal component analysis (PCA) algorithm to perform cross-sample learning on the speech features in the SDR (Sparse Distributed Representation) form to achieve the purpose of dimensionality reduction, and extracts the key features of each speaker by quantifying the speech omics features.

[0027] Further, the obtaining of the classification task and matching the machine learning model based on the classification task include:

[0028] Obtaining a classification task, and obtaining target speech omics features based on the classification task and preset evaluation indicators;

[0029] Determining a task type based on the classification task, and matching a machine learning model based on the task goal;

[0030] Speech acoustic feature classification is performed based on the machine learning model and the target speech omics features.

[0031] The present invention removes redundant features based on classification tasks and preset evaluation indicators, screens out the most valuable speech omics features for classification tasks, improves the efficiency and accuracy of the model, selects the most appropriate machine learning model according to the characteristics and requirements of the classification task, improves the performance and applicability of the model, thereby realizing speech acoustic feature classification and completing the goal of the classification task.

[0032] Furthermore, the step of obtaining target speech omics features based on the classification task and preset evaluation indicators includes:

[0033] Acquire feature redundancy of each phonomic feature in the phonomic feature set based on a graph theory analysis method and the classification task;

[0034] Calculate the feature contribution of each speech-omics feature in the speech-omics feature set based on the Shapley value analysis method and the classification task;

[0035] Calculate the feature importance of each phonomics feature in the phonomics feature set based on the feature importance analysis method and the classification task;

[0036] The target phonetic features are obtained based on the feature redundancy, feature contribution and feature importance.

[0037] The present invention comprehensively evaluates the feature redundancy, contribution and importance of the speech omics feature set based on graph theory analysis, Shapley value analysis and feature importance analysis, thereby screening out the target speech omics features, which helps to build a more efficient and accurate machine learning model and improve the accuracy of classification tasks.

[0038] Further, determining the task type based on the classification task and matching the machine learning model based on the task goal includes:

[0039] Determine a task type based on the classification task, wherein the task type includes a screening task and a prediction task;

[0040] When the task type is a screening task, the machine learning model includes a vector machine model and a random forest model;

[0041] When the task type is a prediction task, the machine learning model includes a neural network model and a linear regression model.

[0042] The present invention refines the classification tasks, clarifies the task types, and matches the most appropriate machine learning model according to the task types, thereby ensuring the consistency of the model performance and task objectives, ensuring that the model can effectively solve the classification tasks, and improving the accuracy and efficiency of classification.

[0043] In a second aspect, the present invention provides a speech classification device of a group level mapping network, comprising: a sparse distribution construction module, a feature dimension reduction module and a classification module;

[0044] The sparse distribution construction module is used to obtain a speech signal and construct a multi-view sparse distribution of the speech signal based on kernel density estimation and sparse coding;

[0045] The feature dimension reduction module is used to construct a group-level mapping network based on the multi-view sparse distribution and cross-sample learning, and obtain a phonemic feature set based on the group-level mapping network;

[0046] The classification module is used to obtain a classification task, match a machine learning model based on the classification task, and perform speech acoustic feature classification based on the machine learning model and the speech omics feature set.

[0047] Furthermore, the sparse distribution building module is used to:

[0048] Extracting features from the speech signal based on preset parameters to obtain a speech feature set;

[0049] The speech feature set is screened based on the kernel density estimation and the vowel formant to obtain the first layer of speech features;

[0050] Iteratively operate the first layer of speech features based on preset acoustic indicators to obtain second layer of speech features;

[0051] A multi-view sparse distribution is acquired based on a preset feature range, a preset activation strength, the first-layer speech features, and the second-layer speech features.

[0052] Furthermore, the sparse distribution building module is used to:

[0053] Acquire a plurality of continuous sub-feature spaces based on the feature range, and screen the number of features of target features in each sub-feature space; the target features include first-layer speech features or second-layer speech features;

[0054] Normalizing the sub-feature space based on the number of features to obtain activation strength of each sub-feature space;

[0055] Acquire a first SDR form of the first layer of speech features and a second SDR form of the second layer of speech features based on the sub-feature space and the activation strength;

[0056] The first SDR form and the second SDR form are spliced ​​to generate the multi-view sparse distribution. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 A flow chart of a speech classification method using a group level mapping network provided by an embodiment of the present invention

[0058] Figure 2 A feature system structure diagram of a speech feature provided by an embodiment of the present invention;

[0059] Figure 3 A schematic diagram of an SDR form provided by an embodiment of the present invention;

[0060] Figure 4 A schematic diagram of multi-view sparse distribution provided by an embodiment of the present invention;

[0061] Figure 5 A schematic diagram of a visual analysis of speech omics features provided by an embodiment of the present invention;

[0062] Figure 6 A schematic diagram of redundancy analysis of speech omics features provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0063] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0064] The terms "first" and "second" and the like in the specification, claims and drawings of this application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products or devices.

[0065] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0066] Example 1

[0067] See also Figure 1 , Figure 1 A flow chart of a speech classification method of a group level mapping network provided by an embodiment of the present invention. The embodiment of the present invention provides a speech classification method of a group level mapping network, including steps 101 to 103, which are as follows:

[0068] Step 101: Acquire a speech signal, and construct a multi-view sparse distribution of the speech signal based on kernel density estimation and sparse coding;

[0069] In this embodiment, constructing the multi-view sparse distribution of the speech signal based on kernel density estimation and sparse coding includes:

[0070] Extracting features from the speech signal based on preset parameters to obtain a speech feature set;

[0071] The speech feature set is screened based on the kernel density estimation and the vowel formant to obtain the first layer of speech features;

[0072] Iteratively operate the first layer of speech features based on preset acoustic indicators to obtain second layer of speech features;

[0073] A multi-view sparse distribution is acquired based on a preset feature range, a preset activation strength, the first-layer speech features, and the second-layer speech features.

[0074] In this embodiment, the paralinguistic information related to the demand feature in the speech signal is obtained by performing hyper-dimensional decomposition on the speech signal.

[0075] In this embodiment, single vowels are relatively easy to record in clinical applications, and almost all languages ​​in the world can produce these three corner vowels, so the three single vowels with the weakest semantic attributes ( / a / , / i / , / u / ) are selected as the starting application corpus. The semantic attributes of these vowels are relatively weak, but they can reflect the functional state of the vocal tract.

[0076] In this embodiment, the parameters for feature extraction are preset, including the frame length and the overlap time of two adjacent frames, and the first three resonance peaks of the speech signal and their corresponding bandwidths are extracted based on the parameters. The frame length is set to 25ms, and the overlap time between two adjacent frames is 10ms, and feature measurement values ​​with time series labels, i.e., speech feature sets, are generated.

[0077] In this embodiment, the first and second formants (F1 and F2) are considered to be the main acoustic clues for identifying the identity of vowels. The third formant (F3) helps to identify whether the vowel is rounded or nasalized. Therefore, the technical solution extracts the first three formants (F1, F2, F3) of / a / , / i / , and / u / and their corresponding bandwidths (B1, B2, B3). Through the acoustic feature extraction script in the PRAAT speech analysis software, the original acoustic features of the speech signal are extracted frame by frame based on preset parameters. The frame length is set to 25ms, and the overlap between adjacent frames is 10ms. A feature measurement value (XFi) with a time sequence label (i.e., which frame) is generated instead of a single measurement value.

[0078] In this embodiment, initial speech features are extracted from the original speech signal, and the feature set is further optimized through kernel density estimation and resonance peak screening, noise and redundant information are reduced, the purity and representativeness of the features are improved, abnormal data that does not conform to the theoretical range of resonance peaks are removed, and the quality of the features is improved; at the same time, based on the optimized first-layer speech features, higher-level features are generated through iterative operations to increase the diversity and complexity of the features, and sparse distributed representations are generated through fixed feature ranges and activation strength calculations, thereby enhancing the generalization ability and classification accuracy of the model and providing support for subsequent machine learning tasks.

[0079] Please refer to Figure 2 , Figure 2 A feature system architecture diagram of a speech feature provided by an embodiment of the present invention.

[0080] In this embodiment, a three-layer feature system is formed by the first layer of speech features, the second layer of speech features and the third layer of speech features to form a multi-perspective sparse distribution. Among them, the first layer of speech features includes F1, F2, F3 and their corresponding B1, B2, B3 extracted from the three corner vowels, namely F1a, F2a, F3a, B1a, B2a, B3a, F1i, F2i, F3i, B1i, B2i, B3i, F1u, F2u, F3u, B1u, B2u, B3u; the second layer of speech features includes derived features: VSA (Vowel Space Area), FCR (Formant Covariance Ratio), F2R (F2 Range); the third layer of speech features includes all the features of the first two layers.

[0081] In this embodiment, the third-layer speech feature is multi-view sparse distribution.

[0082] In this embodiment, the speech feature set is screened based on the kernel density estimation and the vowel resonance peaks to obtain the first layer of speech features, including: a preset theoretical value screening range table, and based on the theoretical value screening range table, all first layer acoustic feature measurement values ​​of the frame where the measurement values ​​that are not within the theoretical range are deleted.

[0083] Table 1

[0084]

[0085] As a specific example of an embodiment of the present invention, please refer to Table 1, which is a theoretical value screening range table provided by an embodiment of the present invention. In this embodiment, based on the vowel formant range of normal people and the situation that head and neck cancer patients may have obvious deviations due to the influence of lesions, we set theoretical value screening ranges for F1a, F2a, F1i, F2i, F1u, and F2u.

[0086] In this embodiment, the feature measurement value X in the speech feature set is selected based on the theoretical value screening range table. Fi Filter and obtain the remaining first measurement value X Fit , and calculate the first measurement value X based on the kernel density estimate Fit The kernel density function is used to remove the measured values ​​in the low-density range according to the preset density threshold K to obtain the final target feature measurement value X Fit-k , which is the first layer of speech features.

[0087] In this embodiment, the density threshold K is set as a proportional coefficient, for example, K=0.3, then only the most core 30% of the data (the top 30% of the kernel density value, i.e., the top 30% of the data with the highest frequency of occurrence) are retained, and the remaining 70% of the data will be deleted.

[0088] In this embodiment, the first layer of speech features is iteratively screened based on preset acoustic indicators and indicator standards to obtain the second layer of speech features, wherein the acoustic indicators include VSA, FCR and F2R.

[0089] Table 2

[0090]

[0091] Please refer to Table 2, which is an acoustic index calculation table provided in an embodiment of the present invention.

[0092] In this embodiment, the target characteristic value X is calculated based on the calculation formula of each acoustic index. Fit-kIterative screening is performed to obtain data sets of three second-layer speech features, namely FCR, VSA, and F2R, where the number of iterations is preset to 10,000 times.

[0093] In this embodiment, the first two layers of features are concatenated to form the third layer of speech features.

[0094] In this embodiment, the acquiring of multi-view sparse distribution based on a preset feature range, a preset activation strength, the first-layer speech features, and the second-layer speech features includes:

[0095] Acquire a plurality of continuous sub-feature spaces based on the feature range, and screen the number of features of target features in each sub-feature space; the target features include first-layer speech features or second-layer speech features;

[0096] Normalizing the sub-feature space based on the number of features to obtain activation strength of each sub-feature space;

[0097] Acquire a first SDR form of the first layer of speech features and a second SDR form of the second layer of speech features based on the sub-feature space and the activation strength;

[0098] The first SDR form and the second SDR form are spliced ​​to generate the multi-view sparse distribution.

[0099] In this embodiment, for the first two layers of features, a fixed feature range is set for each feature, and the fixed feature range simulates that vowels and consonants have corresponding stable activation areas in the primary auditory cortex during speech perception. Then, this feature range is discretized into consecutive sub-feature spaces, each of which simulates a neuron, and then the acoustic feature set (X) after threshold screening is calculated. Fit-k ) is the number of features in each sub-feature space, and is further normalized to a range of 0-1 through the minimum and maximum values ​​to simulate the activation intensity of each neuron, thus giving these feature sets an SDR form (X Fit-k-d ). By fixing the feature range and normalizing, horizontal comparability between individuals is achieved.

[0100] Please refer to Figure 3 , Figure 3 A schematic diagram of an SDR form provided in an embodiment of the present invention.

[0101] In this embodiment, since the existing feature data is not evenly distributed within a fixed feature range but concentrated in an interval close to the lower limit, it is set as a logarithmically evenly divided sub-feature space, and the number of features is recorded as D, thereby improving the spatial resolution of the central area of ​​the data set.

[0102] In this embodiment, the density threshold K refers to the bandwidth parameter in the kernel density estimation. The bandwidth parameter determines the smoothness of the kernel density estimation. A smaller K value will cause the estimation result to be more volatile, and a larger K value will cause the estimation result to be smoother. The number of features D refers to the sparsity parameter in the sparse distributed representation (SDR). The sparsity parameter determines the number of neurons activated in each sub-feature space. A larger D value means that the number of activated neurons is smaller and the features are more sparse.

[0103] In this embodiment, the acoustic feature set (X Fit-k ) is the first layer of speech features, the first SDR form of the first layer of speech features is generated, and the acoustic feature set (X Fit-k ) is the second-layer speech feature, a second SDR form of the second-layer speech feature is generated.

[0104] Please refer to Figure 4 , Figure 4 A schematic diagram of multi-view sparse distribution provided by an embodiment of the present invention.

[0105] Finally, under each K and D setting, the first two layers of features after SDR conversion are concatenated in one dimension to construct the third layer of speech features, i.e., multi-view sparse distribution, representing the features at the individual level, which is labeled as ALL.

[0106] In this embodiment, through sub-feature space division and feature screening, the feature set is optimized, noise and redundant information are reduced, the purity and representativeness of the feature are improved, and abnormal data that does not meet the intensity threshold is removed to improve the quality of the feature. And according to the normalization process, the activation intensity of different sub-feature spaces is made comparable to generate a sparse distributed representation of the first layer of speech features and the second layer of speech features, optimize the feature structure, improve the expressiveness of the features and the performance of the model, and generate a multi-view sparse distribution by splicing features at different levels to provide optimized feature input for subsequent machine learning tasks.

[0107] Step 102: constructing a group-level mapping network based on the multi-view sparse distribution and cross-sample learning, and acquiring a phonemic feature set based on the group-level mapping network;

[0108] In this embodiment, constructing a group-level mapping network based on the multi-view sparse distribution and cross-sample learning, and acquiring a speechomics feature set based on the group-level mapping network, includes:

[0109] Construct a group-level mapping network based on cross-sample learning using principal component analysis;

[0110] The principal component matrix of the multi-view sparse distribution is obtained based on the group-level mapping network, and the phonemic feature set is obtained based on the principal component matrix.

[0111] In this embodiment, cross-sample learning is performed on the multi-view sparse distribution to construct a population-level mapping network, namely the "digital paralinguistic brain" (DPB), which specifically includes unsupervised learning and orthogonal decomposition.

[0112] In this embodiment, the PCA algorithm is used to perform dimensionality reduction processing on the speech features of the SDR form, and the principal components (eigenvectors) and corresponding eigenvalues ​​(variance) of the data set are calculated, so that the principal components are selected based on the variance to form a speech genomics feature set.

[0113] In this embodiment, the SDR speech features (X Fit-k-d ) to conduct cross-sample learning to achieve the purpose of dimensionality reduction. This dimensionality reduction process simulates the orthogonal decomposition strategy of the cerebral cortex for speech signals. The results of learning include two aspects. On the one hand, the SDR form of speech features is quantified (PCA_f_n_K_D), which is used as input data for subsequent clinical diagnosis models, forming a "meta-learning" strategy from the perspective of machine learning. On the other hand, these pattern information is stored in the form of a PCA model, thus giving the "digital paralanguage brain" the memristive property.

[0114] As a specific example of an embodiment of the present invention, the SDR morphological data of the FCR feature in the speech feature is decomposed into a principal component pattern, i.e., a principal component matrix, in high dimension through the PCA model. In order to further filter the random noise, only the first 20 principal components whose cumulative contribution to the variance just reaches 95% of the total variance are included. These principal components are the speech omics features, and the load value of the principal component is the measurement value of the feature. The speech omics features of each speaker are quantified.

[0115] In this embodiment, the DPB method is used to identify dynamic patterns in the population level mapping network, such as periodic changes, trend changes, etc., so as to construct a dynamically changing DPB model.

[0116] In this embodiment, through the combination of different K values ​​and D values, multi-perspective learning of paralinguistic features is provided, different thresholds simulate the intensity of information sources, and different spatial resolutions simulate the number of neurons in specific functionalized brain areas. The decomposed speech omics feature representations under multiple threshold and spatial resolution combinations are necessarily redundant. The three-layer speech feature system simulates the organizational hierarchy of extensive connectivity, structure and functionalization of the brain, thereby giving DPB modularity. Therefore, the feature representation in DPB has omics properties.

[0117] In this embodiment, the voice data of a new speaker is obtained, and the "digital paralanguage brain" performs a new round of unsupervised learning (PCA modeling) just like the human brain learns new knowledge. The learning result (PCA model) is updated and iterated, and new knowledge (principal components, i.e., the phonetic features PCA_f_n_K_D) may be generated. The model is analogous to neurons, which are trained after receiving new information. The new phonetic features are analogous to neurons sending out new dendrites to establish richer connections with other neurons.

[0118] In this example, after obtaining the DPB, the DPB model was evaluated based on spatial visualization, redundancy, modularity, and single phenotypic value.

[0119] Please refer to Figure 5 and Figure 6 , Figure 5 A schematic diagram of a visual analysis of speech omics features provided by an embodiment of the present invention, Figure 6 A schematic diagram of redundancy analysis of speech omics features provided in an embodiment of the present invention.

[0120] In this embodiment, based on the graph theory analysis method, the correlation between the two phonetic omics features (PCA_f_n_K_D) is calculated as a quantitative indicator, and various morphological weighted graphs are tried to visualize the spatial arrangement relationship between all features and intuitively evaluate the redundancy and information load within the features. The dense area in the weighted graph indicates feature redundancy, and the more linear the spatial arrangement of the weighted graph, the richer the information load.

[0121] In this embodiment, based on the feature screening idea of ​​minimum redundancy–maximum relevance criterion (mRMR), clinical information of local lesions and systemic status of head and neck cancer patients is selected to construct multiple biological profiles (biological profiles) information matrices, and canonical correlation analysis (CCA) based on Gaussian kernel is performed between them and the phonetic omics feature matrix. Then, a single feature is deleted one by one and iterative screening is performed until the CCA coefficient between the two matrices no longer increases with the deletion of a certain phonetic omics feature number, thereby identifying the optimal phonetic omics feature set that best represents a certain biological profile and quantifying its redundancy.

[0122] In this embodiment, the canonical correlation analysis (CCA) based on Gaussian kernel is used to quantify the pairwise information consistency between the speech omics features from 22 feature dimensions, and a total of 231 (C_22^2) new quantitative indicators are obtained to evaluate the modularity of the internal features of DPB. These new indicators are named AB, where A and B represent two dimensions. For example, F1a-F2a represents the CCA coefficient between the speech omics data from F1a and F2a. These new indicators can be used as input data for subsequent diagnostic models.

[0123] In this embodiment, in a single phenotypic value, based on different phenotypic states (for example, local lesions such as tumor size and systemic states such as age and immune inflammation level for head and neck cancer patients), correlation analysis and maximum information coefficient (MIC) analysis are used to quantify the biological phenotypic value represented by all speech omics features. For categorical variables such as gender and tumor T stage, differential expression analysis similar to genomics is used to analyze the differences in speech omics features between groups with different states, and Cohen's D value (effect size) is calculated to quantify the differential effect.

[0124] In this embodiment, Figure 5 This is the FCR feature weighting graph. After multi-view learning with different K (0.2-0.9, step size 0.1) and D values ​​(10-100, step size 10), a total of 1073 phonomic features were decomposed. Each point in the figure represents a phonomic feature. The darker the color of the point, the smaller the K value, the smaller the diameter, and the smaller the D value. The thickness of the line between the two points indicates the correlation between the two corresponding phonomic features. The thicker the line, the stronger the correlation, and vice versa. Figure 6 After analyzing the speaker's specific biological spectrum (including age, gender, education level, etc.), 218 features were found to be effective in representing specific paralinguistic states, with an information concentration of 20.3%. In other words, the remaining 855 features are redundant for the representation of the paralinguistic state selected here, with a redundancy of 79.7%.

[0125] In this embodiment, the features of the phonetic genomics feature matrix are screened based on OFSD (One feature stepwise deletion).

[0126] Step 103: Obtain a classification task, match a machine learning model based on the classification task, and perform speech acoustic feature classification based on the machine learning model and the speechomics feature set.

[0127] In this embodiment, the obtaining of the classification task and matching the machine learning model based on the classification task include:

[0128] Obtaining a classification task, and obtaining target speech omics features based on the classification task and preset evaluation indicators;

[0129] Determining a task type based on the classification task, and matching a machine learning model based on the task goal;

[0130] Speech acoustic feature classification is performed based on the machine learning model and the target speech omics features.

[0131] In this embodiment, the step of acquiring target speech omics features based on the classification task and the preset evaluation index includes:

[0132] Acquire feature redundancy of each phonomic feature in the phonomic feature set based on a graph theory analysis method and the classification task;

[0133] Calculate the feature contribution of each speech-omics feature in the speech-omics feature set based on the Shapley value analysis method and the classification task;

[0134] Calculate the feature importance of each phonomics feature in the phonomics feature set based on the feature importance analysis method and the classification task;

[0135] The target phonetic features are obtained based on the feature redundancy, feature contribution and feature importance.

[0136] In this embodiment, the representation of the phonomic features in the DPB is high-dimensional and redundant. On the one hand, in the process of biological phenotype identification of the "digital paralinguistic brain", the phonomic features closely related to specific clinical variables are obtained. Considering the impact of feature redundancy on the model effect, graph analysis is used to observe the spatial relationship between specific clinical variables and all phonomic features from a global perspective, which is more conducive to discovering the features that are most relevant to the target and least affected by other factors. On the other hand, a variety of feature importance analysis methods are used to quantify the contribution of each feature in the diagnostic model, and a new subset of phonomic features is obtained after screening the features, thereby further optimizing the model.

[0137] In this embodiment, by comprehensively evaluating the feature redundancy, contribution and importance of the speech omics feature set based on graph theory analysis, Shapley value analysis and feature importance analysis, the target speech omics features are screened out, which helps to build a more efficient and accurate machine learning model and improve the accuracy of classification tasks.

[0138] In this embodiment, determining the task type based on the classification task and matching the machine learning model based on the task goal includes:

[0139] Determine a task type based on the classification task, wherein the task type includes a screening task and a prediction task;

[0140] When the task type is a screening task, the machine learning model includes a vector machine model and a random forest model;

[0141] When the task type is a prediction task, the machine learning model includes a neural network model and a linear regression model.

[0142] In this embodiment, the classification task is refined, the task type is clarified, and the most appropriate machine learning model is matched according to the task type, thereby ensuring the consistency between the model's performance and the task objectives, ensuring that the model can effectively solve the classification task, and improving the accuracy and efficiency of classification.

[0143] In this embodiment, the classification tasks include tasks for different application scenarios, such as head and neck cancer feature recognition, pathological feature typing, lesion feature staging, risk identification, and prognosis stratification. The task type is determined based on tasks in different scenarios. For example, head and neck cancer feature recognition is an identification and screening task, which can be based on a vector machine model and a random forest model for feature recognition and classification, while pathological typing features, lesion staging features, risk identification, and prognosis stratification can be based on a neural network model and a linear regression model for feature recognition and classification.

[0144] In this embodiment, for the prediction task, for the trained model, various feature evaluation methods such as Shapley analysis and feature importance analysis are used to achieve two-way iterative optimization of model training and speech omics feature screening. At the same time, starting from the biological phenotype, the graph theory analysis method is used to comprehensively evaluate the spatial relationship between all speech omics features and the target, and screen the features with the closest spatial position. In addition, considering the biological process of the application scenario of the classification task, starting from the redundancy analysis, a phased feature screening process is set to explore the application value of the speech omics feature sets obtained at each stage in different diagnostic tasks. For classification tasks with extremely unbalanced grouping, the SMOTE data balancing method is adopted. The feature set after feature screening is successively subjected to manifold learning dimensionality reduction and unsupervised clustering to obtain clinical subgroups with significantly different prognoses, and the clustering results are further combined with other variables to establish a multimodal survival prognosis model.

[0145] In this embodiment, for the screening task, the area under the curve (AUC) is the main evaluation index, and multiple model indicators such as accuracy, sensitivity, specificity, recall rate, precision, F1 value, etc. are evaluated at the same time. Harrel's consistency coefficient (C-index) is used to evaluate the prognostic model. Through model training and optimization, the best model for external validation and its corresponding optimal speech omics feature set and algorithm are determined. Then a group of speech data is prospectively included, converted into quantitative values ​​represented by speech omics features after DPB learning, and input into the corresponding model to obtain the predicted results, which are compared with the actual results to obtain indicators such as AUC and accuracy.

[0146] In this embodiment, by refining the classification task, clarifying the task type, and matching the most appropriate machine learning model according to the task type, the consistency between the model performance and the task objectives is ensured, ensuring that the model can effectively solve the classification task and improve the accuracy and efficiency of classification.

[0147] In this embodiment, applied to the scene of head and neck cancer feature recognition, the original features of each module are converted into SDR form through key acoustic features and the characteristics of human cerebral cortex speech signal encoding are simulated, namely SDR and hierarchical representation, and a three-layer feature learning architecture is designed. The first stage of cross-sample learning is realized with the algorithm idea of ​​orthogonal decomposition, and the features of SDR form are mapped to the whole population network and converted into a form of omics data that can be continuously learned and dynamically evolved. These omics data can then be further used for the second stage of machine learning, including supervised algorithms to realize disease feature classification and unsupervised algorithms to realize prognosis clustering.

[0148] In this embodiment, by using kernel density estimation and sparse coding technology, multi-perspective sparse distribution features are extracted from the original speech signal, and the intrinsic structure and characteristics of the speech signal are comprehensively described according to the multi-perspective sparse distribution features, so as to combine the multi-perspective sparse distribution and cross-sample learning methods to build a group-level mapping network, further extract the speech omics feature set, improve the group representativeness and prediction performance of the features, enhance the generalization ability and prediction accuracy of the features, and at the same time, according to the specific classification task (such as disease classification, prognosis prediction, etc.), select a suitable machine learning model to ensure the matching of the selected model with the classification task, thereby improving the accuracy and efficiency of classification, so as to use the extracted speech omics feature set and the matching machine learning model to complete the classification task of speech acoustic features and improve the classification accuracy of speech acoustic features. Therefore, the speech classification method based on the group-level mapping network can realize the risk of voice pathology of the speaker based on sound recognition.

[0149] The embodiment of the present invention also provides a speech classification device of a group level mapping network, comprising: a sparse distribution construction module, a feature dimension reduction module and a classification module;

[0150] The sparse distribution construction module is used to obtain a speech signal and construct a multi-view sparse distribution of the speech signal based on kernel density estimation and sparse coding;

[0151] The feature dimension reduction module is used to construct a group-level mapping network based on the multi-view sparse distribution and cross-sample learning, and obtain a phonemic feature set based on the group-level mapping network;

[0152] The classification module is used to obtain a classification task, match a machine learning model based on the classification task, and perform speech acoustic feature classification based on the machine learning model and the speech omics feature set.

[0153] Furthermore, the sparse distribution building module is used to:

[0154] Extracting features from the speech signal based on preset parameters to obtain a speech feature set;

[0155] The speech feature set is screened based on the kernel density estimation and the vowel formant to obtain the first layer of speech features;

[0156] Iteratively operate the first layer of speech features based on preset acoustic indicators to obtain second layer of speech features;

[0157] A multi-view sparse distribution is acquired based on a preset feature range, a preset activation strength, the first-layer speech features, and the second-layer speech features.

[0158] Furthermore, the sparse distribution building module is used to:

[0159] Acquire a plurality of continuous sub-feature spaces based on the feature range, and screen the number of features of target features in each sub-feature space; the target features include first-layer speech features or second-layer speech features;

[0160] Normalizing the sub-feature space based on the number of features to obtain activation strength of each sub-feature space;

[0161] Acquire a first SDR form of the first layer of speech features and a second SDR form of the second layer of speech features based on the sub-feature space and the activation strength;

[0162] The first SDR form and the second SDR form are spliced ​​to generate the multi-view sparse distribution.

[0163] In this embodiment, the feature dimension reduction module is used:

[0164] Acquire a plurality of continuous sub-feature spaces based on the feature range, and screen the number of features of target features in each sub-feature space; the target features include first-layer speech features or second-layer speech features;

[0165] Normalizing the sub-feature space based on the number of features to obtain activation strength of each sub-feature space;

[0166] Acquire a first SDR form of the first layer of speech features and a second SDR form of the second layer of speech features based on the sub-feature space and the activation strength;

[0167] The first SDR form and the second SDR form are spliced ​​to generate the multi-view sparse distribution.

[0168] In this embodiment, the feature dimension reduction module is used to:

[0169] Construct a group-level mapping network based on cross-sample learning using principal component analysis;

[0170] The principal component matrix of the multi-view sparse distribution is obtained based on the group-level mapping network, and the phonemic feature set is obtained based on the principal component matrix.

[0171] In this embodiment, the classification module is used to:

[0172] Obtaining a classification task, and obtaining target speech omics features based on the classification task and preset evaluation indicators;

[0173] Determining a task type based on the classification task, and matching a machine learning model based on the task goal;

[0174] Speech acoustic feature classification is performed based on the machine learning model and the target speech omics features.

[0175] In this embodiment, the classification module is used to:

[0176] Acquire feature redundancy of each phonomic feature in the phonomic feature set based on a graph theory analysis method and the classification task;

[0177] Calculate the feature contribution of each speech-omics feature in the speech-omics feature set based on the Shapley value analysis method and the classification task;

[0178] Calculate the feature importance of each phonomics feature in the phonomics feature set based on the feature importance analysis method and the classification task;

[0179] The target phonetic features are obtained based on the feature redundancy, feature contribution and feature importance.

[0180] In this embodiment, the classification module is used to:

[0181] Determine a task type based on the classification task, wherein the task type includes a screening task and a prediction task;

[0182] When the task type is a screening task, the machine learning model includes a vector machine model and a random forest model;

[0183] When the task type is a prediction task, the machine learning model includes a neural network model and a linear regression model.

[0184] In an embodiment of the present invention, a terminal device is also provided, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, the speech classification method of the above-mentioned group level mapping network is implemented.

[0185] In an embodiment of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the above-mentioned speech classification method of the group level mapping network.

[0186] Exemplarily, the computer program may be divided into one or more modules, one or more modules are stored in a memory and executed by a processor to implement the present invention. One or more modules may be a series of computer program instruction segments capable of implementing specific functions, and the instruction segments are used to describe the execution process of the computer program in a terminal device.

[0187] The terminal device may be a computing device such as a desktop computer, a notebook, a PDA, and a cloud server. The terminal device may include, but is not limited to, a processor, a memory, and a display. Those skilled in the art will appreciate that the above components are merely examples of the terminal device and do not constitute a limitation on the terminal device. The terminal device may include more or fewer components than the components, or may combine certain components, or different components. For example, the terminal device may also include input and output devices, network access devices, buses, etc.

[0188] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the terminal device, and uses various interfaces and lines to connect various parts of the entire terminal device.

[0189] The memory can be used to store computer programs and / or modules. The processor implements various functions of the terminal device by running or executing the computer programs and / or modules stored in the memory and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function (such as a sound playback function, a text conversion function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, text message data, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0190] Among them, if the module based on the processing of the multi-device access platform is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0191] The specific embodiments described above further illustrate the purpose, technical solutions and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. It is particularly pointed out that for those skilled in the art, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of protection of the present invention.

Claims

1. A speech classification method using a group level mapping network, characterized in that: include: Acquire a speech signal, and construct a multi-view sparse distribution of the speech signal based on kernel density estimation and sparse coding; Constructing a group-level mapping network based on the multi-view sparse distribution and cross-sample learning, and acquiring a phonemic feature set based on the group-level mapping network; A classification task is obtained, a machine learning model is matched based on the classification task, and speech acoustic feature classification is performed based on the machine learning model and the speechomics feature set.

2. A speech classification method of a group level mapping network as claimed in claim 1, characterized in that: The method of constructing a multi-view sparse distribution of the speech signal based on kernel density estimation and sparse coding includes: Extracting features from the speech signal based on preset parameters to obtain a speech feature set; The speech feature set is screened based on the kernel density estimation and the vowel formant to obtain the first layer of speech features; Iteratively operate the first layer of speech features based on preset acoustic indicators to obtain second layer of speech features; A multi-view sparse distribution is acquired based on a preset feature range, a preset activation strength, the first-layer speech features, and the second-layer speech features.

3. A speech classification method of a group level mapping network as claimed in claim 2, characterized in that: The method of acquiring multi-view sparse distribution based on a preset feature range, a preset activation strength, the first-layer speech features, and the second-layer speech features includes: Acquire a plurality of continuous sub-feature spaces based on the feature range, and screen the number of features of target features in each sub-feature space; the target features include first-layer speech features or second-layer speech features; Normalizing the sub-feature space based on the number of features to obtain activation strength of each sub-feature space; Acquire a first SDR form of the first layer of speech features and a second SDR form of the second layer of speech features based on the sub-feature space and the activation strength; The first SDR form and the second SDR form are spliced ​​to generate the multi-view sparse distribution.

4. A speech classification method of a group level mapping network as claimed in claim 3, characterized in that: The constructing of a group-level mapping network based on the multi-view sparse distribution and cross-sample learning, and obtaining a speechomics feature set based on the group-level mapping network, includes: Construct a group-level mapping network based on cross-sample learning using principal component analysis; The principal component matrix of the multi-view sparse distribution is obtained based on the group-level mapping network, and the phonetic omics feature set is obtained based on the principal component matrix.

5. A speech classification method using a group level mapping network as claimed in claim 4, characterized in that: The obtaining of the classification task and matching the machine learning model based on the classification task include: Obtaining a classification task, and obtaining target speech omics features based on the classification task and preset evaluation indicators; Determining a task type based on the classification task, and matching a machine learning model based on the task goal; Speech acoustic feature classification is performed based on the machine learning model and the target speech omics features.

6. A speech classification method using a group level mapping network as claimed in claim 5, characterized in that: The step of obtaining target speech omics features based on the classification task and preset evaluation indicators includes: Acquire feature redundancy of each phonomic feature in the phonomic feature set based on a graph theory analysis method and the classification task; Calculate the feature contribution of each speech-omics feature in the speech-omics feature set based on the Shapley value analysis method and the classification task; Calculate the feature importance of each phonomics feature in the phonomics feature set based on the feature importance analysis method and the classification task; The target phonetic features are obtained based on the feature redundancy, feature contribution and feature importance.

7. A speech classification method using a group level mapping network as claimed in claim 6, characterized in that: The determining the task type based on the classification task and matching the machine learning model based on the task goal includes: Determine a task type based on the classification task, wherein the task type includes a screening task and a prediction task; When the task type is a screening task, the machine learning model includes a vector machine model and a random forest model; When the task type is a prediction task, the machine learning model includes a neural network model and a linear regression model.

8. A speech classification device of a group level mapping network, characterized in that: include: Sparse distribution building module, feature dimension reduction module and classification module; The sparse distribution construction module is used to obtain a speech signal and construct a multi-view sparse distribution of the speech signal based on kernel density estimation and sparse coding; The feature dimension reduction module is used to construct a group-level mapping network based on the multi-view sparse distribution and cross-sample learning, and obtain a phonemic feature set based on the group-level mapping network; The classification module is used to obtain a classification task, match a machine learning model based on the classification task, and perform speech acoustic feature classification based on the machine learning model and the speech omics feature set.

9. A speech classification device of a group level mapping network as claimed in claim 8, characterized in that: The sparse distribution building module is used to: Extracting features from the speech signal based on preset parameters to obtain a speech feature set; The speech feature set is screened based on the kernel density estimation and the vowel formant to obtain the first layer of speech features; Iteratively operate the first layer of speech features based on preset acoustic indicators to obtain second layer of speech features; A multi-view sparse distribution is acquired based on a preset feature range, a preset activation strength, the first-layer speech features, and the second-layer speech features.

10. The speech classification device of the group level mapping network as claimed in claim 9, characterized in that: The sparse distribution building module is used to: Acquire a plurality of continuous sub-feature spaces based on the feature range, and screen the number of features of target features in each sub-feature space; the target features include first-layer speech features or second-layer speech features; Normalizing the sub-feature space based on the number of features to obtain activation strength of each sub-feature space; Acquire a first SDR form of the first layer of speech features and a second SDR form of the second layer of speech features based on the sub-feature space and the activation strength; The first SDR form and the second SDR form are spliced ​​to generate the multi-view sparse distribution.

Images