Data augmentation and hybrid expert mechanism feature selection classification method and system

Abstract

The application discloses a data enhancement and mixed expert mechanism feature selection classification method and system, comprising an automatic speech recognition (ASR) module, a text-to-speech synthesis (TTS) module, a multi-modal feature extraction module, a mixed expert mechanism (MoE) module, a co-attention mechanism module, a feature fusion module and a classification module. The application uses a speech data enhancement module based on a speech-to-text (TTS) technology to improve data diversity and model generalization capability; uses multi-level acoustic and text feature extraction to more comprehensively represent language changes; uses a mixed expert mechanism (MoE) mechanism to realize dynamic selection of multi-modal features and improve feature utilization efficiency; and uses a co-attention mechanism to optimize the fusion mode between different modal features, enhance the interactive expression capability between features, and significantly improve the recognition accuracy in a multi-modal environment and the robustness of the system, so that the above problems are overcome.

Description

Technical Field

[0001] This invention relates to a technology in the field of information processing based on speech analysis, specifically a classification method and system based on text-to-speech data augmentation and multi-level feature selection using a hybrid expert mechanism. Background Technology

[0002] With the development of artificial intelligence and speech processing technologies, speech analysis-based methods for detecting cognitive function changes have gradually become an important direction in feature recognition research. Existing studies have found that early stages of cognitive function changes may manifest as disorganized language, decreased speech rate, frequent pauses, unclear pronunciation, and word repetition, and these characteristics are highly quantifiable. Compared with traditional detection methods, speech analysis offers advantages such as convenient data collection, low cost, and non-invasiveness, making it particularly suitable for large-scale screening tasks in community or remote settings. Summary of the Invention

[0003] This invention addresses the shortcomings of existing technologies, such as data scarcity, low feature selection efficiency, and limited feature fusion effects. It proposes a classification method and system that combines data augmentation and a hybrid expert mechanism for feature selection. The method utilizes a speech data augmentation module based on speech-to-text (TTS) technology to enhance data diversity and model generalization ability; it employs multi-level acoustic and text feature extraction to more comprehensively represent language variations; it utilizes a hybrid expert mechanism (MoE) to achieve dynamic selection of multimodal features, improving feature utilization efficiency; and it uses a co-attention mechanism to optimize the fusion method between different modal features, enhancing the interactive expressive ability between features. These improvements significantly enhance recognition accuracy and system robustness in multimodal environments, overcoming the aforementioned shortcomings.

[0004] This invention is achieved through the following technical solution:

[0005] This invention relates to a classification method for multi-level feature selection using data augmentation and hybrid expert mechanisms, comprising:

[0006] Step 1: Collect the subject's natural speech input as the raw audio signal to provide a basic data source for subsequent speech recognition and feature extraction, which helps to capture the naturalness of the patient's language;

[0007] Step 2: Use a pre-trained ASR model (such as Whisper) to transcribe the original audio into text, and obtain the text transcription content corresponding to the speech sample.

[0008] Step 3: Re-input the transcribed text into a TTS model (such as FishSpeech). Through cross-sample pairing (i.e., the speaker's voice comes from sample A, and the text content comes from sample B, where A ≠ B, and A and B belong to the same category), enhanced speech is synthesized, preserving the speaker's timbre but with different semantic content. The synthesized speech and the original speech together constitute an extended speech dataset.

[0009] Step 4: For the original speech and TTS-enhanced speech samples, extract the corresponding modal features using different encoders, specifically including:

[0010] a. Spectrum Features: First, a spectrum of the audio signal is generated. Then, the spectrum is processed using a ResNet18 deep neural network model to extract spectrogram features and capture the time-frequency information of the audio. Spectrum features provide high-order frequency domain information of the audio signal and are one of the common features in speech signal analysis.

[0011] b. Mel-frequency cepstral coefficients (MFCC) features: MFCC features are processed using a bidirectional long short-term memory network (BiLSTM). These features reflect the perceptual characteristics of the human auditory system and have strong biological relevance. By using BiLSTM, the time dependence of MFCC features can be effectively captured.

[0012] c. Text Features: ASR transcription is performed using the Whisper model to convert speech content into text. Then, a BERT pre-trained model is used to perform semantic understanding and feature extraction on the transcribed text, providing semantic-level feature support for subsequent classification.

[0013] d. Wav2Vec2 Features: Deep temporal features of the original audio signal are extracted using the Wav2Vec2 model based on the Transformer architecture. The Wav2Vec2 model can capture long-term dependencies in speech and can handle more complex speech patterns.

[0014] Step 5: Input the three modal features extracted by ResNet18, BiLSTM and BERT into the corresponding MoE sub-modules containing several expert networks and gated networks, and dynamically select the most discriminative feature representation to output.

[0015] The MoE submodule includes: k expert networks Gating network corresponding to the features of each input group And a normalized weights, a Softmax function to ensure that all weights are positive and their sum is 1, wherein: each expert network is based on features Get output Gated networks generate weight vectors The MoE output for each feature class is represented as a weighted sum of the outputs from each expert, specifically: Spectral plot features: MFCC characteristics: Text features: , ; This is the weight matrix. This is a bias term.

[0016] Step 6: Input the features selected by MoE and the Wav2Vec2 features not processed by MoE into the co-attention mechanism module. The Wav2Vec2 features are then weighted and adjusted through feature correlation calculation to improve the interaction between different modalities. Specifically: First, the spectrogram features output by the MoE module are... MFCC characteristics and text features The features are concatenated to obtain the intermediate fused feature representation. And through attention transformation function Generate fusion weight vector Then the fusion weights Compared with deep acoustic features extracted by the Wav2Vec2 model Element-wise multiplication and aggregation along the temporal dimension are performed to obtain the final weighted Wav2Vec2 feature vector. Its calculation formula is ,in: Time step features of Wav2Vec2 After weight adjustment, the final output This is the low-dimensional representation after fusion.

[0017] This process enables explicit co-modeling between the original temporal features and other modal features, thereby enhancing Wav2Vec2's comprehensive perception of spectrum, prosody, and semantics; then... and , , These features are then combined to form the final fused feature representation, which is then input into the downstream classification module.

[0018] Step 7: Concatenate and fuse all optimized multimodal features to generate the final fused representation. This includes early fusion and late fusion. The early fusion strategy fuses all modal features before entering the co-attention mechanism, unifying the modeling interaction weights; while the late fusion strategy processes each type of feature separately, integrating text features only in the final classification stage. This invention verifies the performance of the two strategies through experimental comparison. The results show that late fusion performs better in maintaining feature independence, improving model stability, and classification accuracy. Therefore, this system ultimately adopts the late fusion strategy.

[0019] Step 8: Input the fused features into a deep neural network classifier (such as a multilayer perceptron MLP) and output the prediction result for the current sample, that is, determine whether the speaker is a cognitively abnormal (or cognitively normal) individual.

[0020] Technical effect

[0021] This invention constructs a cross-sample speech-text pairing TTS enhancement strategy and combines multimodal feature extraction with a MoE module to achieve dynamic selection of spectrogram, MFCC, and text features. It also introduces the co-attention mechanism Wav2Vec2 for information fusion with other features, constructing a fully optimized speech feature classification system architecture. Compared with existing technologies, this invention significantly enhances the speech classification system's adaptability to small sample sizes, variable semantics, and pronunciation differences; improves the discriminative power of feature selection and the robustness of the system; achieves significantly better classification performance (accuracy exceeding 85%) on public datasets than existing methods; and enables accurate identification of implicit key feature changes in speech, making it suitable for low-cost, non-invasive large-scale speech screening scenarios. Attached Figure Description

[0022] Figure 1 This is a flowchart of the present invention;

[0023] Figure 2 This is a schematic diagram of the system structure of the present invention;

[0024] Figure 3 This is a schematic diagram of the pretreatment stage of the present invention;

[0025] Figure 4 This is a schematic diagram illustrating the generation paths of different types of features in this invention;

[0026] Figure 5 This is a schematic diagram of the feature selection structure of the hybrid expert mechanism of the present invention;

[0027] Figure 6 This is a schematic diagram of the common attention mechanism and classification structure of the present invention;

[0028] Figure 7 This is a schematic diagram illustrating the deployment of the present invention in a practical application scenario;

[0029] Figure 8 This is a performance comparison chart between our method and existing technologies in terms of accuracy.

[0030] The horizontal axis in the figure represents the names of the comparison methods, and the vertical axis represents the classification accuracy (%). Detailed Implementation

[0031] like Figure 2 As shown in the figure, this embodiment relates to a method for implementing the above-mentioned... Figure 1 The data augmentation and hybrid expert mechanism multi-level feature selection classification system shown includes: an automatic speech recognition (ASR) module, a text-to-speech (TTS) module, a multimodal feature extraction module, a hybrid expert mechanism (MoE) module, a co-attention mechanism module, a feature fusion module, and a classification module. Among them, the automatic speech recognition module transcribes the original audio into speech using a pre-trained ASR model to obtain the text transcription content corresponding to the speech sample; the text-to-speech synthesis unit re-inputs the transcribed text into the TTS model and synthesizes enhanced speech that retains the speaker's timbre but has different semantic content through cross-sample pairing. Synthetic speech and original speech together constitute an extended speech dataset. Multimodal feature extraction is performed on the original audio and TTS-enhanced speech samples, extracting spectrogram features, Mel-frequency cepstral coefficient (MFCC) features, deep temporal audio embedding features, and semantic text features through different encoders. The Hybrid Expert Mechanism (MoE) module inputs the spectrogram features, MFCC features, and deep temporal audio embedding features into their respective MoE sub-modules to obtain feature representations. The co-attention mechanism module performs weighted adjustment of semantic text features by calculating the correlation between features, thereby improving the interaction effect between different modalities. The feature fusion module concatenates and fuses all optimized multimodal features to generate the final fused representation. The classification module inputs the fused features into a deep neural network classifier and outputs the prediction result for the current sample, that is, judging whether the speaker is a cognitively abnormal or cognitively normal individual.

[0032] like Figure 3 As shown, the Automatic Speech Recognition (ASR) module includes an audio preprocessing unit, an acoustic modeling unit, and a language decoding unit, wherein: the audio preprocessing unit performs endpoint detection, normalization, and noise reduction on the input speech signal; the acoustic modeling unit inputs the extracted features into an ASR model (such as Whisper) for inference; and the language decoding unit outputs the final text transcription result based on the language model.

[0033] like Figure 3 As shown, the TTS speech data enhancement module includes: a raw speech acquisition and recognition unit, a speech-text cross-sample pairer, a text-to-speech synthesis unit, and an enhanced speech set construction unit, wherein: the raw speech acquisition and recognition unit acquires the raw speech set. The text is then input into an automatic speech recognition module, which uses models such as Whisper to transcribe it into a corresponding text set. Voice-text cross-sample pairer from Select one voice message and text from different speech samples of the same category (where i≠j) are paired; the text-to-speech synthesis unit will process the pairing results (where i≠j). Input TTS model (such as FishSpeech) to generate synthesized speech samples ; Enhance speech set building blocks for all and different The combination of these elements is used to perform a traversal synthesis operation to construct a synthesized speech set. The generated speech and the original speech will be combined to form an expanded speech training dataset.

[0034] Each voice record in the original voice set was generated by a text-image description task.

[0035] The synthesized speech maintains While conveying the speaker's timbre, rhythm, and emotional characteristics, it also carries... The semantic content.

[0036] This TTS enhancement module significantly improves the robustness and recognition ability of the model when dealing with complex speech scenarios (such as unclear pronunciation, short sentences, or semantic confusion), effectively enhances the system's ability to generalize to different speaker features and semantic combinations, and provides richer and more balanced data support for intelligent feature classification tasks.

[0037] like Figure 4 As shown, the multimodal feature extraction module includes: a ResNet18 spectrogram feature extraction unit, a BiLSTM MFCC modeling unit, a Whisper transcription combined with BERT text encoding unit, and a Wav2Vec2 deep embedding extraction unit. These modules are responsible for extracting structural features of different modalities from enhanced speech and transcribed text, laying the foundation for subsequent feature fusion.

[0038] like Figure 5 As shown, the hybrid expert mechanism module dynamically selects multi-level audio and text features to optimize feature discriminativeness and robustness. This module includes: a feature acquisition unit, a MoE expert network, a gating network and weighting unit, and a weighted output unit. Specifically: the feature acquisition unit acquires 1) spectrogram features (extracted via ResNet18); 2) MFCC features (extracted via BiLSTM); and 3) text features (extracted via BERT). The MoE expert network processes the three input features independently through three sub-networks, obtaining multiple expert outputs. The gating network and weighting unit uses the softmax function to calculate weights and dynamically adjusts the weight vector to optimize feature selection. For each feature, the weighted output unit multiplies the expert network output by the corresponding weight to obtain the weighted feature.

[0039] For example, spectrogram features, MFCC features, and text features are processed by expert networks and gating networks, respectively, to generate weighted final feature representations. These weighted feature representations will then enter the downstream co-attention mechanism to further optimize the fusion representation of cross-modal information.

[0040] This embodiment effectively reduces interference from redundant features and improves feature discriminativeness by employing a hybrid expert mechanism module. Compared to traditional feature selection methods, the MoE module is more flexible and precise in selecting multimodal features, thereby significantly improving the accuracy and robustness of the final classification. Particularly in cognitive classification tasks, the application of this module improves classification accuracy and optimizes the model's generalization ability.

[0041] like Figure 6 As shown, the co-attention mechanism module fuses weighted features from multiple feature sources and classifies cognitive states based on these fused features. This module includes a feature acquisition unit, a co-attention mechanism unit, a classification unit, and an output unit. Specifically: the feature acquisition unit acquires spectrogram features, MFCC features, text features, and Wav2Vec2 features after being filtered and weighted by the MoE module; the co-attention mechanism unit weights and fuses the input features (spectral graph features, MFCC features, text features, and Wav2Vec2 features) to obtain a comprehensive feature matrix, then multiplies it element-wise with the Wav2Vec2 features to obtain weighted Wav2Vec2 features, and finally concatenates all weighted features to obtain a comprehensive feature representation; the classification unit uses a deep neural network (such as MLP) to predict and classify the comprehensive feature representation; and the output unit outputs the results based on the predicted classification.

[0042] like Figure 7 The diagram illustrates the application architecture based on the aforementioned classification system in this embodiment. It includes a user device, a classification system located in a server / cloud processing center, and an output terminal. The user device collects the subject's voice input and uploads the data via a mobile terminal or local application. The classification system sequentially performs audio signal preprocessing, feature extraction, expert mechanism selection, and fusion classification operations, returning the classification results to the output terminal for display and report generation by the doctor's system. This architecture is suitable for various deployment scenarios such as hospitals and mobile health devices, demonstrating practical feasibility and scalability.

[0043] Experiments were conducted on the ADReSSo dataset, using Whisper as the ASR model, FishSpeech as the TTS module, and BERT-base as the text feature extractor. Training parameters were batch size=32, learning rate=1e-4, 200 training epochs, and each experiment was run five times with the average value taken. In cognitive classification tasks, this method demonstrated higher classification accuracy and robustness compared to existing methods, particularly excelling in applications with diverse speech data and small sample sets. Table 1 shows a comparison of the performance of this method and existing methods in predicting cognitive abnormality (CA) and cognitive normality (CN).

[0044] Table 1

[0045] As shown in Table 1, by comparing the performance of different methods in terms of accuracy, precision, recall and F1 score, we can comprehensively evaluate the effectiveness and advantages of the methods in cognitive impairment screening.

[0046] Accuracy: As shown in Table 1, our method achieved high classification accuracy of 85.71% in both CA and CN. Compared with other methods, this result indicates that our method can classify more accurately, especially with the application of multimodal feature input and data augmentation, it can better adapt to the diversity of different samples and reduce classification errors.

[0047] Accuracy: In this method, CA achieves an accuracy of 93.10%, significantly higher than other comparative methods, especially some single-modal methods such as Wav2Vec2+TB (accuracy 72.50%) and Whisper-TLmedium (accuracy 77.78%). This indicates that this method can more effectively distinguish CA individuals and reduce false alarms.

[0048] Recall: The recall rate of this method reached 94.29%, which is higher than most methods, such as Whisper-TLmedium with a recall rate of 77.78% and Wav2Vec2+TB with a recall rate of 72.73%. The high recall rate means that this method can effectively identify most CA individuals and reduce false negatives, which is especially important when the early symptoms of CA may be mild.

[0049] F1 score: This method also significantly outperforms other methods in terms of F1 score, reaching 84.38%, higher than methods such as Whisper-TLmedium (F1 score of 77.78%) and Wav2Vec2+TB (F1 score of 76.32%). This indicates that this method achieves a good balance between high precision and high recall.

[0050] like Figure 8 The bar chart shown compares the performance of our method with existing technologies in terms of classification accuracy. This experiment was conducted on the publicly available ADRESSo cognitive impairment recognition dataset, evaluating the performance of different methods on the same test set to ensure fairness and consistency in the comparison. The experimental results in this bar chart further support the practicality and robustness of our system in cognitive impairment screening tasks.

[0051] In summary, compared with existing technologies, our proposed method demonstrates superior performance across multiple key performance indicators. These results indicate that by combining text-to-speech (TTS) data augmentation and hybrid expert mechanism (MoE) dynamic feature selection, our method not only enhances the recognition capability of cognitive functions but also addresses the challenges of complex speech features and diverse data in practical applications, demonstrating high application value and practicality.

[0052] The above-described specific implementations can be partially adjusted by those skilled in the art in different ways without departing from the principles and purpose of the present invention. The scope of protection of the present invention is defined by the claims and is not limited to the above-described specific implementations. All implementation schemes within the scope of the claims are bound by the present invention.

Claims

1. A classification method for multi-level feature selection using data augmentation and hybrid expert mechanisms, characterized in that, include: Step 1: Collect the subject's natural speech input as the raw audio signal to provide the basic data source for subsequent speech recognition and feature extraction; Step 2: Use a pre-trained ASR model to transcribe the original audio into text, obtaining the text transcription content corresponding to the speech sample. Step 3: Re-input the transcribed text into the TTS model and synthesize enhanced speech that retains the speaker's timbre but has different semantic content by cross-sample pairing. Combine the synthesized speech with the original speech to form an extended speech dataset. The cross-sample pairing refers to the following: the speaker's voice comes from sample A, the text content comes from sample B, A ≠ B, and A and B belong to the same category of samples; Step 4: Extract the corresponding modal features from the original speech and TTS-enhanced speech samples using ResNet18, BiLSTM, BERT, and Wav2Vec2. Step 5: Input the three modal features extracted by ResNet18, BiLSTM and BERT into the corresponding MoE sub-modules containing several expert networks and gated networks, and dynamically select the most discriminative feature representation to output. Step 6: Input the features selected by MoE and the modal features extracted by Wav2Vec2 without MoE into the co-attention mechanism module. The modal features extracted by Wav2Vec2 are weighted and adjusted by calculating the correlation between features, thereby improving the interaction effect between different modalities. Step 7: Concatenate and fuse the features output by the weighted and adjusted Wav2Vec2 and the three modal features output by the MoE submodule to generate the final fused representation; Step 8: Input the fused features into the deep neural network classifier and output the prediction result for the current sample, that is, determine whether the speaker is a cognitively abnormal or cognitively normal individual.

2. The classification method for multi-level feature selection using data augmentation and hybrid expert mechanisms according to claim 1, characterized in that, The corresponding modal features specifically include: a. Modal features extracted by ResNet18: First, the audio is used to generate a corresponding spectrogram, and then the spectrogram is processed by the ResNet18 deep neural network model to obtain the modal features. b. Modal features extracted by BiLSTM: obtained by processing MFCC features through a bidirectional long short-term memory network (BiLSTM); c. Text features: Text features are obtained by transcribing ASR using the Whisper model, converting speech content into text, and then using a BERT pre-trained model to perform semantic understanding and feature extraction on the transcribed text. d.Wav2Vec2 features: Deep acoustic features extracted from the original audio signal using the Wav2Vec2 model based on the Transformer architecture.

3. The classification method for multi-level feature selection using data augmentation and hybrid expert mechanisms according to claim 1, characterized in that, The MoE submodule includes: k expert networks Gating network corresponding to the features of each input group And a normalized weights, a Softmax function to ensure that all weights are positive and their sum is 1, wherein: each expert network is based on features Get output Gated networks generate weight vectors The MoE output for each feature class is represented as a weighted sum of the outputs from each expert, specifically: Spectral plot features: MFCC characteristics: Text features: , ; This is the weight matrix. This is a bias term.

4. The classification method for multi-level feature selection using data augmentation and hybrid expert mechanisms according to claim 1, characterized in that, Step 6 specifically involves: first, analyzing the spectrogram characteristics output by the MoE module. MFCC characteristics and text features The features are concatenated to obtain the intermediate fused feature representation. And through attention transformation function Generate fusion weight vector Then the fusion weights Compared with deep acoustic features extracted by the Wav2Vec2 model Element-wise multiplication and aggregation along the temporal dimension are performed to obtain the final weighted Wav2Vec2 feature vector. Its calculation formula is ,in: Time step features of Wav2Vec2 After weight adjustment, the final output This is the low-dimensional representation after fusion.

5. A classification system that implements the data augmentation and multi-level feature selection using a hybrid expert mechanism according to any one of claims 1-4, characterized in that, include: The system comprises an Automatic Speech Recognition (ASR) module, a Text-to-Speech (TTS) module, a multimodal feature extraction module, a Hybrid Expert Equation (MoE) module, a co-attention mechanism module, a feature fusion module, and a classification module. Specifically: the ASR module uses a pre-trained ASR model to transcribe the original audio, obtaining the corresponding text transcription. The TTS module re-inputs the transcribed text into the TTS model, synthesizing enhanced speech that retains the speaker's timbre but differs in semantic content through cross-sample pairing. The synthesized speech and the original speech together constitute an extended speech dataset. The multimodal feature extraction module is used for both the original speech and the TTS enhanced speech samples, employing ResNet18, BiLSTM, BERT, and Wav2Vec2 to extract features. The corresponding modal features are extracted. The hybrid expert mechanism (MoE) module is used to input the three modal features extracted by ResNet18, BiLSTM, and BERT into the corresponding MoE sub-modules containing several expert networks and gating networks, and dynamically selects the most discriminative feature representation to output. The co-attention mechanism module is used to input the features selected by MoE and the modal features extracted by Wav2Vec2 without MoE into the co-attention mechanism module. The modal features extracted by Wav2Vec2 are weighted and adjusted by calculating the correlation between features, thereby improving the interaction effect between different modalities. The feature fusion module concatenates and fuses the features output by Wav2Vec2 and the three modal features output by the MoE sub-modules to generate the final fused representation. The classification module will integrate features into a deep neural network classifier and output a prediction result for the current sample, that is, determine whether the speaker is a cognitively abnormal or cognitively normal individual.

6. The classification system for data augmentation and hybrid expert mechanism multi-level feature selection according to claim 5, characterized in that, The text-to-speech (TTS) module includes: a raw speech acquisition and recognition unit, a speech-text cross-sample pairer, a text-to-speech synthesis unit, and an enhanced speech set construction unit, wherein: the raw speech acquisition and recognition unit acquires the raw speech set. The text is then input into an automatic speech recognition module, which uses Whisper to transcribe it into a corresponding text set. Voice-text cross-sample pairer from Select one voice message and text from different speech samples of the same category Pairing is performed, where i ≠ j; the text-to-speech synthesis unit will process the pairing results ( Input TTS model to generate synthesized speech samples ; Enhance speech set building blocks for all and different The combination of these elements is used to perform a traversal synthesis operation to construct a synthesized speech set. The generated speech and the original speech will be combined to form an expanded speech training dataset.

7. The classification system for data augmentation and hybrid expert mechanism multi-level feature selection according to claim 5, characterized in that, The hybrid expert mechanism module includes: a feature acquisition unit, a MoE expert network, a gating network and weighting unit, and a weighted output unit. Specifically: the feature acquisition unit acquires three modal features extracted by ResNet18, BiLSTM, and BERT; the MoE expert network processes the three input features independently through three sub-networks, obtaining multiple expert outputs; the gating network and weighting unit uses the softmax function to calculate weights and dynamically adjusts the weight vector to optimize feature selection; and the weighted output unit multiplies the expert network output by the corresponding weight for each feature to obtain the weighted feature.

8. The classification system for data augmentation and hybrid expert mechanism multi-level feature selection according to claim 5, characterized in that, The aforementioned co-attention mechanism module includes: a feature acquisition unit, a co-attention mechanism unit, a classification unit, and an output unit. Specifically: the feature acquisition unit acquires three modal features (after filtering and weighting by the MoE module) and Wav2Vec2 features; the co-attention mechanism unit weights and fuses the input three modal features and Wav2Vec2 features to obtain a comprehensive feature matrix, then multiplies it element-wise with the Wav2Vec2 features to obtain weighted Wav2Vec2 features, and finally concatenates all weighted features to obtain a comprehensive feature representation; the classification unit uses a deep neural network to predict and classify the comprehensive feature representation; and the output unit outputs the results based on the predicted classification.

9. A classification system for multi-level feature selection using data augmentation and hybrid expert mechanisms, characterized in that, include: The user device, the classification system and output terminal located in the server / cloud processing center as described in any of claims 5-8, wherein: the user device collects the subject's voice input and uploads the data through a mobile terminal or local application; The classification system executes any one of the classification methods described in claims 1-5 in sequence and returns the classification result to the output.

Images