Speech data cleaning and labeling method and device based on multi-modal feedback

Abstract

The present application relates to the technical field of speech data processing, in particular to a speech data cleaning and labeling method and device based on multi-modal feedback, the method comprising the following steps: S1: collecting auxiliary environment data, processing the auxiliary environment data to obtain a feedback signal; S2: collecting user original speech data, cleaning and labeling the user original speech data according to the feedback signal to obtain final speech data; the device comprises a collection module, an analysis module, a data cleaning and labeling module, and a reminding module; the collection module, the analysis module, and the data cleaning and labeling module are connected in pairs, and the reminding module is connected to the analysis module. The present application solves the technical problem that the traditional speech processing mode ignores the state of the collection process, cannot dynamically adjust the processing strategy according to the device working condition and user posture, and causes the processing mode to be mismatched with the actual situation, resulting in audio secondary distortion.

Description

Technical Field

[0001] This invention relates to the field of speech data processing technology, and more specifically, to a method and apparatus for cleaning and labeling speech data based on multimodal feedback. Background Technology

[0002] With the rapid development of intelligent voice technology, voice data has become a core information carrier in scenarios such as human-computer interaction, intelligent conferencing, medical records, and in-vehicle voice systems. In these applications, voice data cleaning and annotation are crucial steps to ensure the accuracy of subsequent tasks such as speech recognition, speaker tagging, and sentiment analysis. Cleaning operations such as noise reduction and gain adjustment improve signal quality, while annotation operations such as transcription and tagging imbue the data with semantic information, thereby supporting the reliable operation of upper-layer intelligent applications.

[0003] However, the scenarios described above are often highly dynamic and uncertain: users may speak while moving, causing changes in distance; fingers may unintentionally cover the microphone while holding the device; multiple people may speak simultaneously in a meeting environment; and aging equipment may lead to decreased sensitivity. Traditional speech processing methods typically only perform generalized processing on the audio signal itself, failing to consider the real-time status of the acquisition device, such as whether it is obstructed or whether the frequency response is abnormal, and also failing to perceive the user's posture information, such as face orientation and lip movements. This results in a severe disconnect between the audio processing method and the actual situation. This blind processing mode, when faced with complex acquisition environments, often uses general algorithms with fixed parameters, which can easily lead to excessive speech suppression, distortion, or labeling errors.

[0004] Furthermore, audio data is a one-dimensional temporal signal, lacking spatial intuitiveness and unable to directly represent changes in physical state during the acquisition process. Therefore, spatial data-oriented technical solutions are difficult to directly transfer to the field of speech processing. A speech data cleaning and annotation method that can sense the state of the acquisition process and dynamically adjust the processing strategy according to the real-time situation is needed to solve the problem of data quality degradation caused by ignoring the acquisition context in existing technologies. Summary of the Invention

[0005] The purpose of this application is to provide a method and apparatus for cleaning and labeling speech data based on multimodal feedback. It solves the technical problem that traditional speech processing methods ignore the state of the acquisition process and cannot dynamically adjust the processing strategy according to the equipment condition and user posture, resulting in a mismatch between the processing method and the actual situation and causing secondary audio distortion.

[0006] To solve the above-mentioned technical problems, the solution adopted in this application is as follows:

[0007] This invention provides a method for cleaning and labeling speech data based on multimodal feedback, characterized by the following steps:

[0008] Step S1: Collect auxiliary environment data and process the auxiliary environment data to obtain a feedback signal; the auxiliary environment data includes, but is not limited to, voice acquisition device status data and user posture data; the voice acquisition device status data includes, but is not limited to, the occlusion status, sensitivity, battery level, and frequency response characteristics of the voice acquisition device; the user posture data includes, but is not limited to, the user's face orientation, the user's lip movements, and the user's distance from the voice acquisition device.

[0009] Step S2: Collect the user's original voice data, and clean and label the user's original voice data according to the feedback signal to obtain the final voice data;

[0010] The implementation of step S2 above includes the following steps:

[0011] Step S201: Collect the user's original voice data;

[0012] Step S202: Select an audio data cleaning and processing method based on the feedback signal; perform cleaning and processing on the original audio data according to the cleaning and processing method to obtain intermediate audio data;

[0013] Step S203: Evaluate the intermediate speech data and select an annotation processing method based on the evaluation results and the feedback signal;

[0014] Step S204: Perform annotation processing on the intermediate speech data according to the annotation processing method to obtain the final speech data.

[0015] In some embodiments, step S1 includes the following steps:

[0016] Step S101: Collect and process auxiliary environment data to obtain the health status of the audio data acquisition process; the health status represents a quantitative indicator of the quality of the current audio acquisition process.

[0017] Step S102: Generate a corresponding feedback signal based on the health status.

[0018] In some embodiments, in step S101, the formula is used:

[0019]

[0020] The health score is obtained; where H is the health score, w1, w2, w3, ..., w n Let x1, x2, x3, ..., x be the weighting coefficients. n To provide auxiliary environmental data.

[0021] In some embodiments, in step S102, a corresponding feedback signal is generated based on the health status, using the formula:

[0022]

[0023] Therefore, a corresponding feedback signal is generated based on the stated health status; where Y is the feedback signal. b and c are the values ​​of the feedback signal, H is the health level, and y1, y2, and y3 are the health level thresholds.

[0024] In some embodiments, step S2 further includes step S205: evaluating the final voice data, adjusting the cleaning method according to the evaluation result, and using the adjusted cleaning method for subsequent audio data cleaning.

[0025] In some embodiments, the cleaning process includes at least one of the following:

[0026] Perform frequency response compensation filtering on the audio;

[0027] Perform dynamic gain smoothing on the audio;

[0028] The beamforming algorithm guides the sound pickup direction;

[0029] A noise reduction algorithm is used for the low-frequency range.

[0030] In some embodiments, the annotation processing method includes at least one of the following:

[0031] Adjust the confidence threshold of the recognition results;

[0032] Correct the speaker tagging results;

[0033] The sentiment annotation results were corrected.

[0034] Distinguish and label the speaker identities of overlapping speech segments;

[0035] Verify and correct the voice event detection results.

[0036] The present invention also provides a speech data cleaning and annotation device based on multimodal feedback, characterized in that, in order to implement the above-mentioned speech data cleaning and annotation method based on multimodal feedback, it includes an acquisition module, a parsing module, and a data cleaning and annotation module; the acquisition module, parsing module, and data cleaning and annotation module are connected in pairs;

[0037] The acquisition module is used to acquire auxiliary environment data and user's original voice data; the acquisition module processes the auxiliary environment data and transmits it to the parsing module; the acquisition module processes the user's original voice data and transmits it to the data cleaning and annotation module.

[0038] The parsing module is used to process the auxiliary environmental data, and the generated feedback signal is transmitted to the data cleaning and labeling module.

[0039] The data cleaning and labeling module cleans and labels the user's original voice data based on the feedback signal.

[0040] In some embodiments, the data cleaning and labeling module includes a cleaning module, a labeling module, an evaluation module, and a storage module; the cleaning module is connected to the labeling module, the evaluation module, and the storage module, respectively, and the labeling module is connected to the evaluation module and the storage module, respectively.

[0041] The storage module is used to store cleaning processing methods and labeling processing methods. The cleaning module and the labeling module can select the data processing method from the storage module.

[0042] The cleaning module reads the audio data cleaning processing method from the storage module according to the feedback signal, and cleans the received original user voice data according to the audio data cleaning processing method to obtain intermediate voice data; the cleaning module transmits the intermediate voice data to the evaluation module and the annotation module respectively.

[0043] The annotation module reads the audio data annotation processing method from the storage module according to the feedback signal and the evaluation result transmitted by the evaluation module, and performs annotation processing on the received intermediate speech data according to the audio data annotation processing method to obtain the final speech data.

[0044] The annotation module transmits the final voice data to the evaluation module for evaluation.

[0045] The evaluation module receives the intermediate audio data transmitted by the cleaning module and evaluates it, and the evaluation result is transmitted to the annotation module; the evaluation module receives the final audio data transmitted by the annotation module and evaluates it, and the evaluation result is transmitted to the cleaning module; the cleaning module adjusts the cleaning processing method according to the evaluation result, and the adjusted cleaning processing method is used for subsequent audio data cleaning processing.

[0046] In some embodiments, a reminder module is further included, which is connected to the parsing module; the reminder module sends a reminder to the user based on the feedback signal transmitted by the parsing module; the reminder content includes, but is not limited to, correcting abnormal user posture, cleaning or adjusting the voice acquisition device; the reminder module is connected to a reminder device terminal, which includes, but is not limited to, a speaker, a display, and an indicator light.

[0047] The technical solution of this application has at least the following advantages and beneficial effects:

[0048] 1. This invention generates feedback signals reflecting the quality of the acquisition process by collecting device status data and user posture data, enabling subsequent cleaning and labeling processing to be dynamically adjusted according to actual working conditions. Compared with traditional methods that use fixed parameters to process all audio uniformly, this invention can select appropriate processing methods for different situations such as occlusion, distance changes, and posture deviations, effectively avoiding secondary audio distortion caused by the mismatch between processing strategies and actual conditions, and significantly improving the accuracy of voice data processing.

[0049] 2. This invention adds an evaluation step after cleaning, forming a dual-optimization structure of an inner closed loop of "cleaning-evaluation-labeling" and an outer closed loop of "final data evaluation-cleaning adjustment". Through dual evaluation of intermediate and final data, the device can quantify the processing effect of each step and feed the evaluation results back to the preceding steps to dynamically adjust the processing strategy, thereby continuously optimizing the selection of processing methods and continuously improving accuracy.

[0050] 3. This application uses an alert module to transform the feedback signals generated by the parsing module into intuitive user prompts, enabling users to perceive and correct abnormal postures or clean the equipment in real time. This proactive intervention mechanism focuses on the source of data collection, effectively avoiding data quality degradation caused by improper user behavior, reducing the difficulty and error of subsequent cleaning and labeling, and improving the user experience and the overall intelligence level of the device. Attached Figure Description

[0051] Figure 1 This is a flowchart illustrating the overall workflow of the present invention;

[0052] Figure 2 This is a flowchart of step S2 in this embodiment;

[0053] Figure 3 This is a schematic diagram of the overall structure of the present invention;

[0054] Figure 4 This is a schematic diagram of the data cleaning and labeling module in this embodiment. Detailed Implementation

[0055] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0056] It should be noted that similar reference numerals and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. The terms "center," "upper," "lower," "inner," and "outer," indicating orientation or positional relationships based on the orientation or positional relationships shown in the figures, or the orientation or positional relationships commonly used when the product is in use, are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed or operated in a specific orientation, and therefore should not be construed as a limitation on this application. It should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," and "connect" should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; mechanical connections or electrical connections; direct connections or indirect connections through an intermediate medium; and internal communication between two elements. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0057] Example

[0058] This invention provides a method for cleaning and labeling speech data based on multimodal feedback, such as... Figure 1 As shown, its implementation includes the following steps:

[0059] Step S1: Collect auxiliary environment data and process the auxiliary environment data to obtain feedback signals; the auxiliary environment data includes, but is not limited to, voice acquisition device status data and user posture data; the voice acquisition device status data includes, but is not limited to, the occlusion status, sensitivity, battery level, and frequency response characteristics of the voice acquisition device; the user posture data includes, but is not limited to, the user's face orientation, the user's lip movements, and the user's distance from the voice acquisition device.

[0060] Step S2: Collect the user's original voice data, clean and label the user's original voice data according to the feedback signal, and obtain the final voice data.

[0061] Traditional speech processing devices typically only process the acquired audio signal itself, ignoring the state in which the audio signal was acquired. This approach means that the device can only use a general processing algorithm to deal with all scenarios. When the actual acquisition state does not match the algorithm's preset conditions, it is easy to cause audio distortion, speech damage, or labeling errors after processing, and it cannot provide targeted optimization based on different acquisition states.

[0062] This invention collects and analyzes multimodal auxiliary data such as the status data of the voice acquisition device and the user's posture data to obtain feedback signals that reflect the quality of the current acquisition process. Based on these feedback signals, appropriate processing methods are selected for subsequent audio cleaning and annotation, enabling audio processing to fully consider the actual situation of the acquisition process and achieve a shift from passive processing to active perception and adaptive processing. The advantage of this design is that it can select targeted cleaning and annotation strategies based on the health of the acquisition process, avoiding secondary distortion or insufficient processing caused by general processing methods, and ultimately obtaining more faithful and natural voice data, significantly improving the accuracy of subsequent tasks such as speech recognition, speaker tagging, and sentiment analysis.

[0063] Furthermore, step S1 includes the following steps:

[0064] Step S101: Collect and process auxiliary environment data to obtain the health status of the audio data acquisition process; the health status represents a quantitative indicator of the quality of the current audio acquisition process.

[0065] Step S102: Generate corresponding feedback signals based on health status.

[0066] In this embodiment, in step S101, the formula is used:

[0067]

[0068] The health score is obtained; where H is the health score, and w1, w2, w3, ..., w n Let x1, x2, x3, ..., x be the weighting coefficients. n To provide auxiliary environmental data.

[0069] In this embodiment, step S102 generates a corresponding feedback signal based on the health status, using the formula:

[0070]

[0071] Therefore, a corresponding feedback signal is generated based on the health status; where Y is the feedback signal. b and c are the values ​​of the feedback signal, H is the health level, and y1, y2, and y3 are the health level thresholds.

[0072] Furthermore, such as Figure 2 As shown, the implementation of step S2 above includes the following steps:

[0073] Step S201: Collect the user's original voice data;

[0074] Step S202: Select an audio data cleaning and processing method based on the feedback signal; perform cleaning and processing on the original audio data according to the cleaning and processing method to obtain intermediate audio data;

[0075] Step S203: Evaluate the intermediate speech data and select the annotation processing method based on the evaluation results and feedback signals;

[0076] Step S204: Perform annotation processing on the intermediate speech data according to the annotation processing method to obtain the final speech data.

[0077] In this embodiment, step S2 further includes step S205: evaluating the final speech data, adjusting the cleaning process based on the evaluation results, and using the adjusted cleaning process for subsequent audio data cleaning. The advantage of this design is that by evaluating the quality of the final speech data, the device can quantify the actual effect of the current cleaning strategy and perform joint analysis with the multimodal auxiliary data collected in step S1. When a specific modal anomaly is detected, the device can automatically adjust the cleaning process, achieving precise optimization for different modal anomalies. Simultaneously, the closed-loop adjustment based on multimodal feedback avoids one-sided optimization of a single parameter, significantly improving the long-term stability of the device in complex environments and greatly reducing manual parameter tuning and maintenance costs.

[0078] It should be noted that, in this embodiment, the cleaning process includes at least one of the following:

[0079] Perform frequency response compensation filtering on the audio;

[0080] Perform dynamic gain smoothing on the audio;

[0081] The beamforming algorithm guides the sound pickup direction;

[0082] A noise reduction algorithm is used for the low-frequency range.

[0083] It should be noted that, in this embodiment, the annotation processing method includes at least one of the following:

[0084] Adjust the confidence threshold of the recognition results;

[0085] Correct the speaker tagging results;

[0086] The sentiment annotation results were corrected.

[0087] Distinguish and label the speaker identities of overlapping speech segments;

[0088] Verify and correct the voice event detection results.

[0089] It should be noted that the audio data cleaning and annotation methods used in this invention are all common techniques in this field. The core innovation of this invention lies not in improving a specific cleaning or annotation algorithm formula, but in dynamically selecting and optimizing these processing methods based on multimodal feedback signals—that is, intelligently selecting the most suitable processing method for the current acquisition situation based on real-time acquired device status data and user posture data, and adaptively adjusting the processing parameters. This optimization mechanism for selecting processing methods enables general-purpose technologies to achieve optimal results in different scenarios, thereby significantly improving the overall quality and adaptability of voice data processing.

[0090] This multimodal feedback-based selection optimization mechanism eliminates the need for pre-installed complex algorithms or redundant processing of all data. Instead, it allows the device to invoke appropriate general processing methods on demand based on real-time acquisition status, significantly reducing computational burden and storage requirements. By precisely matching processing strategies with acquisition scenarios, it avoids resource waste caused by over-processing or algorithm redundancy, achieving efficient and lightweight operation and making it easier to deploy and apply on resource-constrained edge devices.

[0091] The present invention also provides a speech data cleaning and annotation device based on multimodal feedback to realize the above-mentioned speech data cleaning and annotation method based on multimodal feedback, which includes an acquisition module, a parsing module, and a data cleaning and annotation module; the acquisition module, parsing module, and data cleaning and annotation module are connected in pairs;

[0092] The acquisition module is used to acquire auxiliary environment data and user's raw voice data; the acquisition module processes the auxiliary environment data and transmits it to the parsing module; the acquisition module processes the user's raw voice data and transmits it to the data cleaning and annotation module.

[0093] The parsing module processes the auxiliary environmental data, and the generated feedback signal is transmitted to the data cleaning and labeling module.

[0094] The data cleaning and labeling module cleans and labels the user's original voice data based on feedback signals.

[0095] Furthermore, the data cleaning and labeling module includes a cleaning module, a labeling module, an evaluation module, and a storage module; the cleaning module is connected to the labeling module, the evaluation module, and the storage module, respectively, and the labeling module is connected to the evaluation module and the storage module, respectively.

[0096] The storage module stores the cleaning and labeling processing methods, and the cleaning and labeling modules can select the data processing method from the storage module.

[0097] The cleaning module reads the audio data cleaning and processing method from the storage module based on the feedback signal, and cleans the received raw user voice data according to the audio data cleaning and processing method to obtain intermediate voice data; the cleaning module transmits the intermediate voice data to the evaluation module and the annotation module respectively.

[0098] The annotation module reads the audio data annotation processing method from the storage module based on the feedback signal and the evaluation result transmitted by the evaluation module, and performs annotation processing on the received intermediate speech data according to the audio data annotation processing method to obtain the final speech data.

[0099] The annotation module transmits the final voice data to the evaluation module for assessment.

[0100] The evaluation module receives the intermediate audio data transmitted from the cleaning module and evaluates it. The evaluation result is then transmitted to the annotation module. The evaluation module receives the final audio data transmitted from the annotation module and evaluates it. The evaluation result is then transmitted to the cleaning module. The cleaning module adjusts the cleaning process based on the evaluation result. The adjusted cleaning process is then used for cleaning subsequent audio data.

[0101] In this embodiment, a reminder module is also included, which is connected to the parsing module. The reminder module sends a reminder to the user based on the feedback signal transmitted by the parsing module. The reminder content includes, but is not limited to, correcting abnormal user posture, cleaning or adjusting the voice acquisition device. The reminder module is connected to a reminder device terminal, which includes, but is not limited to, a speaker, a display, and an indicator light.

[0102] It should be noted that the advantage of the reminder module in this embodiment lies in its ability to obtain real-time feedback signals by connecting to the parsing module. This allows it to detect abnormalities in the acquisition device and user posture during the acquisition process immediately and issue targeted reminders to the user in a direct manner, such as through voice, images, or lights. This real-time human-computer interaction mechanism shifts passive processing to proactive intervention, enabling users to correct abnormal behaviors or clean the device in a timely manner. This ensures the quality of voice data acquisition from the source, effectively reduces the difficulty and error of subsequent cleaning and labeling, and improves the overall intelligence level of the device.

[0103] The various embodiments of the present invention have now been described in detail. To avoid obscuring the concept of the invention, some details known in the art have not been described. Those skilled in the art will fully understand how to implement the technical solutions of this invention based on the above description, and the scope of the invention is defined by the appended claims.

Claims

1. A method for cleaning and labeling speech data based on multimodal feedback, characterized in that, Its implementation includes the following steps: Step S1: Collect auxiliary environment data and process the auxiliary environment data to obtain a feedback signal; the auxiliary environment data includes, but is not limited to, voice acquisition device status data and user posture data; the voice acquisition device status data includes, but is not limited to, the occlusion status, sensitivity, battery level, and frequency response characteristics of the voice acquisition device; the user posture data includes, but is not limited to, the user's face orientation, the user's lip movements, and the user's distance from the voice acquisition device. Step S2: Collect the user's original voice data, and clean and label the user's original voice data according to the feedback signal to obtain the final voice data; The implementation of step S2 above includes the following steps: Step S201: Collect the user's original voice data; Step S202: Select an audio data cleaning and processing method based on the feedback signal; perform cleaning and processing on the original audio data according to the cleaning and processing method to obtain intermediate audio data; Step S203: Evaluate the intermediate speech data and select an annotation processing method based on the evaluation results and the feedback signal; Step S204: Perform annotation processing on the intermediate speech data according to the annotation processing method to obtain the final speech data.

2. The method for cleaning and labeling speech data based on multimodal feedback according to claim 1, characterized in that, Step S1 includes the following steps: Step S101: Collect and process auxiliary environment data to obtain the health status of the audio data acquisition process; the health status represents a quantitative indicator of the quality of the current audio acquisition process. Step S102: Generate a corresponding feedback signal based on the health status.

3. The method for cleaning and labeling speech data based on multimodal feedback according to claim 2, characterized in that, In step S101, the formula is used: The health score is obtained; where H is the health score, w1, w2, w3, ..., w n Let x1, x2, x3, ..., x be the weighting coefficients. n To provide auxiliary environmental data.

4. The method for cleaning and labeling speech data based on multimodal feedback according to claim 2, characterized in that, In step S102, a corresponding feedback signal is generated based on the health status, using the formula: Therefore, a corresponding feedback signal is generated based on the stated health status; where Y is the feedback signal. b and c are the values ​​of the feedback signal, H is the health level, and y1, y2, and y3 are the health level thresholds.

5. The speech data cleaning and annotation method based on multimodal feedback according to claim 1, characterized in that, Step S2 further includes step S205: evaluating the final voice data, adjusting the cleaning process based on the evaluation results, and using the adjusted cleaning process for subsequent audio data cleaning.

6. The speech data cleaning and annotation method based on multimodal feedback according to claim 1, characterized in that, The cleaning process includes at least one of the following: Perform frequency response compensation filtering on the audio; Perform dynamic gain smoothing on the audio; The beamforming algorithm guides the sound pickup direction; A noise reduction algorithm is used for the low-frequency range.

7. The method for cleaning and labeling speech data based on multimodal feedback according to claim 1, characterized in that, The annotation processing method includes at least one of the following: Adjust the confidence threshold of the recognition results; Correct the speaker tagging results; The sentiment annotation results were corrected. Distinguish and label the speaker identities of overlapping speech segments; Verify and correct the voice event detection results.

8. A speech data cleaning and annotation device based on multimodal feedback, characterized in that, To implement the speech data cleaning and annotation method based on multimodal feedback as described in claims 1-7, the method includes an acquisition module, a parsing module, and a data cleaning and annotation module; the acquisition module, parsing module, and data cleaning and annotation module are connected in pairs. The acquisition module is used to acquire auxiliary environment data and user's original voice data; the acquisition module processes the auxiliary environment data and transmits it to the parsing module; the acquisition module processes the user's original voice data and transmits it to the data cleaning and annotation module. The parsing module is used to process the auxiliary environmental data, and the generated feedback signal is transmitted to the data cleaning and labeling module. The data cleaning and labeling module cleans and labels the user's original voice data based on the feedback signal.

9. The speech data cleaning and annotation device based on multimodal feedback according to claim 8, characterized in that, The data cleaning and labeling module includes a cleaning module, a labeling module, an evaluation module, and a storage module; the cleaning module is connected to the labeling module, the evaluation module, and the storage module, and the labeling module is connected to the evaluation module and the storage module, respectively. The storage module is used to store cleaning processing methods and labeling processing methods. The cleaning module and the labeling module can select the data processing method from the storage module. The cleaning module reads the audio data cleaning processing method from the storage module according to the feedback signal, and cleans the received original user voice data according to the audio data cleaning processing method to obtain intermediate voice data; the cleaning module transmits the intermediate voice data to the evaluation module and the annotation module respectively. The annotation module reads the audio data annotation processing method from the storage module according to the feedback signal and the evaluation result transmitted by the evaluation module, and performs annotation processing on the received intermediate speech data according to the audio data annotation processing method to obtain the final speech data. The annotation module transmits the final voice data to the evaluation module for evaluation. The evaluation module receives the intermediate voice data transmitted by the cleaning module and evaluates it, and the evaluation result is transmitted to the annotation module. The evaluation module receives the final voice data transmitted by the annotation module and evaluates it. The evaluation result is transmitted to the cleaning module. The cleaning module adjusts the cleaning method according to the evaluation result. The adjusted cleaning method is used for subsequent audio data cleaning.

10. The speech data cleaning and annotation device based on multimodal feedback according to claim 8, characterized in that, It also includes a reminder module, which is connected to the parsing module; the reminder module sends a reminder to the user based on the feedback signal transmitted by the parsing module; the reminder content includes, but is not limited to, correcting abnormal user posture, cleaning or adjusting the voice acquisition device; the reminder module is connected to a reminder device terminal, which includes, but is not limited to, a speaker, a display, and an indicator light.

Images