Data annotation method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a data annotation method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining to-be-annotated voice data in response to a data annotation request; performing voice-to-text processing and side language information recognition processing on the voice data to obtain an initial labeling result; wherein the initial recognition result comprises a plurality of sub-texts and a sub-language information tag corresponding to each sub-text; and inputting the initial labeling result into a pre-trained natural language processing model to correct and / or optimize the sub-language information label to obtain a target labeling result, and feeding back the target labeling result. Through the embodiment of the invention, the automation of voice data labeling is realized, the sublingual information label is labeled with higher efficiency and accuracy, and the cost and workload of data labeling are reduced.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a data labeling method and device, an electronic device, and a storage medium. Background Art

[0002] With the continuous development of artificial intelligence technology, various models are being used more and more widely in various fields.

[0003] Data Annotation is an important step before training a model. It refers to adding labels or annotations to the original sample data so that the model can understand and process the data.

[0004] In the related art, the scene of voice data annotation is completed by manual annotation. This method is time-consuming and labor-intensive, especially for voice data with complex information. Relying solely on manual annotation is not only inefficient, but also may affect the accuracy of annotation. Summary of the invention

[0005] In view of the above problems, a data annotation method and device, an electronic device, and a storage medium are proposed to overcome the above problems or at least partially solve the above problems, including:

[0006] A data labeling method, the method comprising:

[0007] In response to the data annotation request, obtaining the voice data to be annotated;

[0008] Performing speech-to-text processing and paralanguage information recognition processing on the speech data to obtain an initial annotation result; wherein the initial recognition result includes a plurality of subtexts and a paralanguage information label corresponding to each subtext;

[0009] The initial annotation result is input into a pre-trained natural language processing model, the paralanguage information label is corrected and / or optimized, a target annotation result is obtained, and the target annotation result is fed back.

[0010] Optionally, performing speech-to-text processing and paralanguage information recognition processing on the speech data to obtain an initial annotation result includes:

[0011] The speech data is input into a pre-trained speech recognition model for parallel speech-to-text processing and paralanguage information recognition processing to obtain the initial annotation result; wherein the paralanguage information tags in the initial recognition result include emotion information tags, tone information tags and role information tags.

[0012] Optionally, inputting the initial annotation result into a pre-trained natural language processing model to correct and / or optimize the paralanguage information label includes:

[0013] Determining an associated subtext corresponding to each subtext, and correcting and / or optimizing the paralinguistic information tag corresponding to each subtext according to the associated subtext, including:

[0014] determining whether the sub-language information tag is incorrect according to the associated sub-text, and if the sub-language information tag is incorrect, correcting the sub-language information tag;

[0015] and / or, judging whether the voice source of each sub-text has a tone change according to the associated sub-text, and if the voice source has a tone change, updating the tone information tag to a tone change information tag;

[0016] And / or, judging whether there is an emotion change in the voice source according to the associated subtext, and if there is an emotion change in the voice source, updating the emotion information tag to an emotion change information tag.

[0017] Optionally, the feeding back of the target labeling result includes:

[0018] Determining the confidence level corresponding to the target labeling result;

[0019] If the confidence level is greater than or equal to a preset confidence level, feeding back the target labeling result;

[0020] If the confidence level is less than the preset confidence level, a request for correcting the target labeling result is initiated.

[0021] Optionally, after initiating a request to correct the target annotation result, the method further includes:

[0022] A corrected annotation result is received after the target annotation result is corrected, and the corrected annotation result is fed back.

[0023] Optionally, before performing speech-to-text processing and paralanguage information recognition processing on the speech data, the method further includes:

[0024] The voice data is subjected to one or more data preprocessing including noise reduction processing, format conversion processing, cropping processing and ambient sound muting processing.

[0025] Optionally, after feeding back the revised annotation result, the method further includes:

[0026] Based on the modified annotation results, the natural language processing model and / or the speech recognition model is optimized.

[0027] A data labeling device, comprising:

[0028] A voice data acquisition module to be annotated, used to acquire the voice data to be annotated in response to a data annotation request;

[0029] An initial annotation result generating module is used to perform speech-to-text processing and paralanguage information recognition processing on the speech data to obtain an initial annotation result; wherein the initial recognition result includes a plurality of subtexts and a paralanguage information label corresponding to each subtext;

[0030] The target labeling result processing module is used to input the initial labeling result into the pre-trained natural language processing model to correct and / or optimize the paralanguage information label, obtain the target labeling result, and feed back the target labeling result.

[0031] An electronic device comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor, wherein the computer program implements the data labeling method as described above when executed by the processor.

[0032] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the data labeling method described above is implemented.

[0033] The embodiments of the present invention have the following advantages: by responding to a data annotation request, speech data to be annotated is obtained; speech data is subjected to speech-to-text processing and paralanguage information recognition processing to obtain an initial annotation result; wherein the initial recognition result includes a plurality of subtexts and a paralanguage information label corresponding to each subtext; the initial annotation result is input into a pre-trained natural language processing model to correct and / or optimize the paralanguage information label to obtain a target annotation result, and the target annotation result is fed back, thereby realizing automation of speech data annotation, annotating the paralanguage information label with higher efficiency and accuracy, and reducing the cost and workload of data annotation. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solution of the present invention, the accompanying drawings required for use in the description of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0035] Figure 1 is a flowchart of a data labeling method provided by an embodiment of the present invention;

[0036] Figure 2It is a diagram of an implementation process of combining a model to perform data annotation provided by an embodiment of the present invention;

[0037] Figure 3 is a structural block diagram of a data labeling device provided by an embodiment of the present invention;

[0038] Figure 4 It is a structural schematic diagram of a data labeling electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0039] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] In related technologies, the annotation of speech data mainly relies on manual work, especially the annotation of complex paralinguistic information (such as emotion, tone, speaker conversion, etc.), which is time-consuming and labor-intensive.

[0041] In other related technologies, although some automatic annotation has been achieved, most of them are limited to the conversion of speech to text, and cannot effectively process complex paralinguistic information such as emotions and tone, and the annotation quality is not high. In addition, even the existing multi-task annotation system is not yet perfect. Annotation often requires multiple steps, each of which requires an independent model and processing module. Finally, the processing results of different modules need to be integrated and aligned, which is not efficient and prone to errors.

[0042] Therefore, the present invention is based on the core idea of ​​performing speech recognition and paralanguage information recognition on the speech data to be annotated to obtain an initial annotation result, and then inputting the initial annotation result into a pre-trained natural language processing model for correction and / or optimization to obtain a target annotation result, thereby realizing automatic annotation and achieving the purpose of reducing the complexity and time cost of manual annotation.

[0043] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0044] Reference Figure 1 , shows a flowchart of a data labeling method provided by an embodiment of the present invention, which may specifically include the following steps:

[0045] Step 101, in response to a data annotation request, obtaining voice data to be annotated;

[0046] The speech data to be annotated refers to the original speech sample data that needs to be annotated based on the needs of model training, data analysis, etc.

[0047] Exemplarily, the voice data may be raw waveform data stored in the form of an audio file, such as an audio file in a WAV, MP3, FLAC, etc. Specifically, it may also be a recording file in certain application scenarios, such as a conversation recording file, a conference recording file, etc.

[0048] Step 102, performing speech-to-text processing and paralanguage information recognition processing on the speech data to obtain an initial annotation result; wherein the initial recognition result includes a plurality of subtexts and a paralanguage information label corresponding to each subtext;

[0049] Speech-to-text processing is the process of converting data stored in audio form into data stored in text form after analysis and recognition.

[0050] Paralanguage refers to information conveyed in language communication through voice, intonation, volume, rhythm, pauses, etc., in addition to actual vocabulary and grammatical structure. This information can affect the emotional color, attitude and intention of communication, and can convey rich information even without actual language content. Paralanguage information recognition is to identify the paralanguage information in speech data and forward it to the corresponding paralanguage information tag.

[0051] The subtext is a subtext in the complete text after the speech data is converted into text, for example, it can be a subtext such as a sentence, a phrase, a word, etc. Each subtext can correspond to multiple paralinguistic information tags to reflect the paralinguistic information of the subtext.

[0052] Exemplarily, a preset algorithm or model and data analysis may be used to perform speech-to-text processing and paralanguage information recognition processing on the speech data.

[0053] In some embodiments of the present invention, before performing speech-to-text processing and paralanguage information recognition processing on the speech data, the method further includes:

[0054] The voice data is subjected to one or more data preprocessing including noise reduction processing, format conversion processing, cropping processing and ambient sound muting processing.

[0055] In practical applications, data preprocessing before speech-to-text processing and paralinguistic information recognition processing can improve the accuracy of the initial annotation results. Among them, noise reduction processing can improve speech clarity and enhance speech features; format conversion processing can unify data formats and improve compatibility; cropping processing can remove useless data and focus on effective information in speech data; ambient sound muting processing can retain only the voice of the other party and enhance speech features.

[0056] In some embodiments of the present invention, the performing speech-to-text processing and paralanguage information recognition processing on the speech data to obtain an initial annotation result includes:

[0057] The speech data is input into a pre-trained speech recognition model for parallel speech-to-text processing and paralanguage information recognition processing to obtain the initial annotation result; wherein the paralanguage information tags in the initial recognition result include emotion information tags, tone information tags and role information tags.

[0058] Speech recognition models, that is, models that can convert speech into text.

[0059] The emotional information label can reflect the emotional information of the voice source corresponding to the annotated sub-text, such as anger, calmness, tension, satisfaction, etc.

[0060] The tone information tag can reflect the tone information of the voice source, such as question, sarcasm, affirmation, etc.

[0061] The role information tag can reflect the role information of the voice source, such as customer, customer service, etc.

[0062] Exemplarily, the Whisper speech recognition model may be used to perform multi-task, parallel processing of speech-to-text processing and paralinguistic information recognition processing on speech data.

[0063] Whisper is a model based on the Transformer architecture (a deep learning model architecture) and is specifically designed to handle multilingual speech-to-text tasks. Whisper performs well in handling multiple languages ​​and dialects and is able to maintain high recognition accuracy in noisy environments.

[0064] In practical applications, the Whisper model needs to be fine-tuned and trained so that it can have the multi-task capabilities of speech-to-text and paralinguistic information recognition, as follows:

[0065] 1) Data preparation: First, you need to prepare a speech dataset containing multi-dimensional labels to train the Whisper model. The speech dataset includes pre-labeled speech files, text transcriptions, emotion information labels, tone information labels, role information labels, etc. For example, for customer service scenarios, the speech dataset can be a recording file of the conversation between the customer and the customer service.

[0066] 2) Model fine-tuning: Use the above speech dataset to fine-tune the Whisper model to enable multi-task learning capabilities. During the training process, the model not only learns speech-to-text transcription, but also learns to generate additional emotion, tone, and role labels. For example, for a customer service scenario, after inputting the recording file into the Whisper model, the output obtained is as follows:

[0067] {

[0068]

[0069] Among them, text is the subtext, the emotion information label (emotion) is neutral, the tone information label (tone) is polite, and the role information label (speaker) is agent.

[0070] Furthermore, after the Whisper model is trained to convergence, the speech data to be labeled is input into the Whisper model.

[0071] For example, after inputting a recording of a conversation between a customer and a customer service representative into the Whisper model, the output obtained is as follows:

[0072] Customer: "I have waited for a long time, why hasn't it been processed yet?"

[0073]

[0074] In this conversation clip, the model identified the customer’s emotion as “angry”, the tone as “questioning”, and the role as “customer”.

[0075] Customer Service: "I'm sorry for the inconvenience. I'll check it for you right away."

[0076]

[0077] In this conversation clip, the model identified the customer service agent’s emotion as “calm”, the tone as “apologetic”, and the role as agent.

[0078] Step 103: input the initial annotation result into a pre-trained natural language processing model, modify and / or optimize the paralanguage information label to obtain a target annotation result, and feed back the target annotation result.

[0079] The natural language processing model is a computer model used to process and understand human language. By inputting the initial annotation results into the pre-trained natural language processing model and correcting and / or optimizing the paralinguistic information labels, the paralinguistic information labels can be made more accurate and reflect more dimensional information.

[0080] In practical applications, the natural language processing model can be a large language model (LLM), which is a deep learning model with a large number of parameters and complex structure, capable of processing and generating natural language text. The LLM model can further expand the paralinguistic information label to reflect more dimensional information.

[0081] For example, the paralinguistic information label can be further refined to identify potential paralinguistic information; for example, the tone information label expressed as "peaceful" can be analyzed as its potential "sarcastic" tone information label through a natural language processing model combined with context analysis, thereby achieving optimization of the paralinguistic information label.

[0082] In this way, by further correcting and optimizing the initial annotation results of the natural language processing model, multi-dimensional labels including speech content, emotion, tone and speaker are automatically generated, thereby significantly improving the annotation efficiency and accuracy and reducing the cost and workload of manual annotation.

[0083] In some embodiments of the present invention, inputting the initial annotation result into a pre-trained natural language processing model to correct and / or optimize the paralinguistic information label includes:

[0084] Determining an associated subtext corresponding to each subtext, and correcting and / or optimizing the paralinguistic information tag corresponding to each subtext according to the associated subtext, including:

[0085] determining whether the sub-language information tag is incorrect according to the associated sub-text, and if the sub-language information tag is incorrect, correcting the sub-language information tag;

[0086] Related subtexts are other subtexts related to the current subtext, such as the context of the current subtext, other subtexts from the same language source, etc.

[0087] For example, if the role information tag of a sub-text is "user", through the natural language processing model combined with context analysis, it is found that the voice is actually uttered by customer service, then the role information tag of the text is corrected to "customer service".

[0088] and / or, judging whether the voice source of each sub-text has a tone change according to the associated sub-text, and if the voice source has a tone change, updating the tone information tag to a tone change information tag;

[0089] For example, for a customer whose tone information label is "affirmative", through the natural language processing model combined with context analysis, the customer's tone changes from "question" to "affirmative" after the customer service's answer, the tone information label "affirmative" is updated to the tone change information label of "question to affirmative".

[0090] And / or, judging whether there is an emotion change in the voice source according to the associated subtext, and if there is an emotion change in the voice source, updating the emotion information tag to an emotion change information tag.

[0091] For example, for a customer whose tone information label is "calm", through the natural language processing model combined with context analysis, the customer changes from "angry" to "calm" after being comforted by customer service, then the tone information label "calm" is updated to the emotion change information label of "angry to calm".

[0092] In some examples, the Whisper model can also preliminarily annotate different speakers and their transition points, and LLM can further annotate the identities of different speakers and the emotional changes between conversations based on the text dialogue relationship. By combining Whisper's multi-task learning capabilities with LLM's contextual understanding capabilities, multi-dimensional labels including speech content, emotion, tone, and speaker are automatically generated, thereby significantly improving annotation efficiency and accuracy and reducing the cost and workload of manual annotation.

[0093] In some embodiments of the present invention, the feeding back of the target labeling result includes:

[0094] Determining the confidence level corresponding to the target labeling result;

[0095] If the confidence level is greater than or equal to a preset confidence level, feeding back the target labeling result;

[0096] If the confidence level is less than the preset confidence level, a request for correcting the target labeling result is initiated.

[0097] Confidence is used to measure the certainty or reliability of the model's output results. When the confidence is greater than or equal to the preset confidence, the target labeling result can be considered reliable and feedback is given accordingly. When the confidence is less than the preset confidence, the target labeling result is reliable and the target labeling result needs to be corrected.

[0098] Exemplarily, after initiating a request to correct the target annotation result, the recipient may correct the target annotation result through manual recognition.

[0099] In some embodiments of the present invention, after initiating a request to correct the target annotation result, the method further includes:

[0100] A corrected annotation result is received after the target annotation result is corrected, and the corrected annotation result is fed back.

[0101] In this embodiment, after receiving the revised annotation result, the revised annotation result is considered to be reliable, and feedback on the revised annotation result can be provided.

[0102] In some embodiments of the present invention, after feeding back the revised annotation result, the method further includes:

[0103] Based on the modified annotation results, the natural language processing model and / or the speech recognition model is optimized.

[0104] In practical applications, the corrected annotation results can be the target annotation results that have been manually corrected. The corrected annotation results are used again as sample data for natural language processing models (such as LLM models) and speech recognition models (such as Whisper models) for training to perform iteration and optimization.

[0105] Specifically, through the system's feedback mechanism, if the annotator or user finds that the target annotation result does not match the actual situation, they can provide feedback through the reinforcement learning framework, and the natural language processing model and speech recognition model will use the feedback for iterative optimization. For example, if the model misjudges a certain emotional label, the annotator can provide a correction, and the model will optimize its future annotation capabilities based on the corrected annotation results, thereby achieving continuous optimization and iteration of the model.

[0106] In some examples, such as Figure 2 As shown in the figure, the implementation process diagram of labeling language data and strengthening learning feedback is also provided, as follows:

[0107] 1) Input the speech data to be annotated and perform preprocessing;

[0108] 2) Input the whisper model to perform speech-to-text conversion and paralinguistic information recognition to obtain the initial annotation results;

[0109] 3) Input the initial annotation results into the LLM model for correction and analysis to obtain the optimized target annotation results;

[0110] 4) The target labeling results with high confidence are directly integrated and output, and the target labeling results with low confidence are output after manual labeling and feedback, and the results of manual labeling and feedback are used to reinforce the learning feedback of whisper and LLM.

[0111] 5) Generate results and provide feedback.

[0112] In some examples, the target annotation results can also be used as pre-annotation results in the manual data annotation process, which can help annotators process large amounts of conversation data more efficiently, reduce manual intervention, and improve annotation efficiency.

[0113] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0114] The embodiments of the present invention have the following advantages: by responding to a data annotation request, speech data to be annotated is obtained; speech data is subjected to speech-to-text processing and paralanguage information recognition processing to obtain an initial annotation result; wherein the initial recognition result includes a plurality of subtexts and a paralanguage information label corresponding to each subtext; the initial annotation result is input into a pre-trained natural language processing model to correct and / or optimize the paralanguage information label to obtain a target annotation result, and the target annotation result is fed back, thereby realizing automation of speech data annotation, annotating the paralanguage information label with higher efficiency and accuracy, and reducing the cost and workload of data annotation.

[0115] In addition, some embodiments of the present invention also have the following advantages:

[0116] Improve annotation efficiency: Through automated annotation, a large amount of manual annotation work is reduced, which is particularly advantageous when processing large-scale voice data.

[0117] Multi-dimensional information generation: It not only annotates speech content, but also generates complex paralinguistic information, covering emotions, tone, speaker changes, etc., greatly enriching the annotation dimensions of speech data.

[0118] High-quality annotation: LLM’s text understanding and semantic inference capabilities are used to generate more accurate and detailed labels such as sentiment and tone, improving the overall quality of annotation.

[0119] Reference Figure 3 , shows a schematic diagram of the structure of a data labeling device provided by an embodiment of the present invention, which may specifically include the following modules:

[0120] The voice data to be annotated acquisition module 301 is used to acquire the voice data to be annotated in response to a data annotation request;

[0121] The initial annotation result generating module 302 is used to perform speech-to-text processing and paralanguage information recognition processing on the speech data to obtain an initial annotation result; wherein the initial recognition result includes a plurality of subtexts and a paralanguage information label corresponding to each subtext;

[0122] The target labeling result processing module 303 is used to input the initial labeling result into the pre-trained natural language processing model, correct and / or optimize the paralanguage information label, obtain the target labeling result, and feed back the target labeling result.

[0123] In some embodiments of the present invention, the initial annotation result generating module 302 includes:

[0124] The speech recognition model processing submodule is used to input the speech data into a pre-trained speech recognition model to perform parallel speech-to-text processing and paralanguage information recognition processing to obtain the initial annotation result; wherein the paralanguage information tags in the initial recognition result include emotion information tags, tone information tags and role information tags.

[0125] In some embodiments of the present invention, the target labeling result processing module 303 includes:

[0126] The target labeling result processing submodule is used to determine the associated subtext corresponding to each subtext, and to modify and / or optimize the paralinguistic information tag corresponding to each subtext according to the associated subtext, including:

[0127] determining whether the sub-language information tag is incorrect according to the associated sub-text, and if the sub-language information tag is incorrect, correcting the sub-language information tag;

[0128] and / or, judging whether the voice source of each sub-text has a tone change according to the associated sub-text, and if the voice source has a tone change, updating the tone information tag to a tone change information tag;

[0129] And / or, judging whether there is an emotion change in the voice source according to the associated subtext, and if there is an emotion change in the voice source, updating the emotion information tag to an emotion change information tag.

[0130] In some embodiments of the present invention, the target labeling result processing module 303 includes:

[0131] A confidence determination submodule, used to determine the confidence corresponding to the target labeling result;

[0132] A target labeling result feedback submodule, used for feeding back the target labeling result if the confidence level is greater than or equal to a preset confidence level;

[0133] The target labeling result correction request submodule is used to initiate a request to correct the target labeling result if the confidence level is less than the preset confidence level.

[0134] In some embodiments of the present invention, the device further comprises:

[0135] The revised annotation result receiving module is used to receive the revised annotation result after the target annotation result is revised, and feed back the revised annotation result.

[0136] In some embodiments of the present invention, the device further comprises:

[0137] The preprocessing module is used to perform one or more data preprocessing of noise reduction processing, format conversion processing, cropping processing and ambient sound muting processing on the voice data.

[0138] In some embodiments of the present invention, the device further comprises:

[0139] A model optimization module is used to optimize the natural language processing model and / or the speech recognition model based on the modified annotation results.

[0140] Some embodiments of the present invention further provide an electronic device, which may include a processor, a memory, and a computer program stored in the memory and capable of running on the processor, and the computer program implements the above data labeling method when executed by the processor.

[0141] Specific as Figure 4 As shown, the electronic device includes a processor 401, a communication interface 402, a memory 403 and a communication bus 404, wherein the processor 401, the communication interface 402, and the memory 403 communicate with each other through the communication bus 404;

[0142] Memory 403, used for storing computer programs;

[0143] The processor 401 is used to execute the program stored in the memory 403 to implement the following steps:

[0144] In response to the data annotation request, obtaining the voice data to be annotated;

[0145] Performing speech-to-text processing and paralanguage information recognition processing on the speech data to obtain an initial annotation result; wherein the initial recognition result includes a plurality of subtexts and a paralanguage information label corresponding to each subtext;

[0146] The initial annotation result is input into a pre-trained natural language processing model, the paralanguage information label is corrected and / or optimized, a target annotation result is obtained, and the target annotation result is fed back.

[0147] In some embodiments of the present invention, the performing speech-to-text processing and paralanguage information recognition processing on the speech data to obtain an initial annotation result includes:

[0148] The speech data is input into a pre-trained speech recognition model for parallel speech-to-text processing and paralanguage information recognition processing to obtain the initial annotation result; wherein the paralanguage information tags in the initial recognition result include emotion information tags, tone information tags and role information tags.

[0149] In some embodiments of the present invention, inputting the initial annotation result into a pre-trained natural language processing model to correct and / or optimize the paralinguistic information label includes:

[0150] Determining an associated subtext corresponding to each subtext, and correcting and / or optimizing the paralinguistic information tag corresponding to each subtext according to the associated subtext, including:

[0151] determining whether the sub-language information tag is incorrect according to the associated sub-text, and if the sub-language information tag is incorrect, correcting the sub-language information tag;

[0152] and / or, judging whether the voice source of each sub-text has a tone change according to the associated sub-text, and if the voice source has a tone change, updating the tone information tag to a tone change information tag;

[0153] And / or, judging whether there is an emotion change in the voice source according to the associated subtext, and if there is an emotion change in the voice source, updating the emotion information tag to an emotion change information tag.

[0154] In some embodiments of the present invention, the feeding back of the target labeling result includes:

[0155] Determining the confidence level corresponding to the target labeling result;

[0156] If the confidence level is greater than or equal to a preset confidence level, feeding back the target labeling result;

[0157] If the confidence level is less than the preset confidence level, a request for correcting the target labeling result is initiated.

[0158] In some embodiments of the present invention, after initiating a request to correct the target annotation result, the method further includes:

[0159] A corrected annotation result is received after the target annotation result is corrected, and the corrected annotation result is fed back.

[0160] In some embodiments of the present invention, before performing speech-to-text processing and paralanguage information recognition processing on the speech data, the method further includes:

[0161] The voice data is subjected to one or more data preprocessing including noise reduction processing, format conversion processing, cropping processing and ambient sound muting processing.

[0162] In some embodiments of the present invention, after feeding back the revised annotation result, the method further includes:

[0163] Based on the modified annotation results, the natural language processing model and / or the speech recognition model is optimized.

[0164] Some embodiments of the present invention further provide a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the above data labeling method is implemented.

[0165] Some embodiments of the present invention further provide a computer program product, including a computer program, which implements the above data labeling method when executed by a processor.

[0166] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0167] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0168] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0169] The communication bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0170] The communication interface is used for communication between the above terminal and other devices.

[0171] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0172] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0173] Those skilled in the art will appreciate that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0174] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0175] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0176] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0177] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0178] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the above elements.

[0179] The data annotation method and device, electronic device, and storage medium provided are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A data labeling method, characterized in that: The method comprises: In response to the data annotation request, obtaining the voice data to be annotated; Performing speech-to-text processing and paralanguage information recognition processing on the speech data to obtain an initial annotation result; wherein the initial recognition result includes a plurality of subtexts and a paralanguage information label corresponding to each subtext; The initial annotation result is input into a pre-trained natural language processing model, the paralanguage information label is corrected and / or optimized, a target annotation result is obtained, and the target annotation result is fed back.

2. The method according to claim 1, characterized in that The performing speech-to-text processing and paralanguage information recognition processing on the speech data to obtain an initial annotation result includes: The speech data is input into a pre-trained speech recognition model for parallel speech-to-text processing and paralanguage information recognition processing to obtain the initial annotation result; wherein the paralanguage information tags in the initial recognition result include emotion information tags, tone information tags and role information tags.

3. The method according to claim 2, characterized in that The step of inputting the initial annotation result into a pre-trained natural language processing model to correct and / or optimize the paralanguage information label includes: Determining an associated subtext corresponding to each subtext, and correcting and / or optimizing the paralinguistic information tag corresponding to each subtext according to the associated subtext, including: determining whether the sub-language information tag is incorrect according to the associated sub-text, and if the sub-language information tag is incorrect, correcting the sub-language information tag; and / or, judging whether the voice source of each sub-text has a tone change according to the associated sub-text, and if the voice source has a tone change, updating the tone information tag to a tone change information tag; And / or, judging whether there is an emotion change in the voice source according to the associated subtext, and if there is an emotion change in the voice source, updating the emotion information tag to an emotion change information tag.

4. The method according to claim 2, characterized in that: The feedback of the target labeling result includes: Determining the confidence level corresponding to the target labeling result; If the confidence level is greater than or equal to a preset confidence level, feeding back the target labeling result; If the confidence level is less than the preset confidence level, a request for correcting the target labeling result is initiated.

5. The method according to claim 4, characterized in that After initiating a request to correct the target annotation result, the method further includes: A corrected annotation result is received after the target annotation result is corrected, and the corrected annotation result is fed back.

6. The method according to claim 1, characterized in that Before performing speech-to-text processing and paralanguage information recognition processing on the speech data, the method further includes: The voice data is subjected to one or more data preprocessing including noise reduction processing, format conversion processing, cropping processing and ambient sound muting processing.

7. The method according to claim 5, characterized in that After feeding back the revised annotation result, the method further includes: Based on the modified annotation results, the natural language processing model and / or the speech recognition model is optimized.

8. A data labeling device, characterized in that: The device comprises: A voice data acquisition module to be annotated, used to acquire the voice data to be annotated in response to a data annotation request; An initial annotation result generating module is used to perform speech-to-text processing and paralanguage information recognition processing on the speech data to obtain an initial annotation result; wherein the initial recognition result includes a plurality of subtexts and a paralanguage information label corresponding to each subtext; The target labeling result processing module is used to input the initial labeling result into the pre-trained natural language processing model, correct and / or optimize the paralanguage information label, obtain the target labeling result, and feed back the target labeling result.

9. An electronic device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the data labeling method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the data labeling method according to any one of claims 1 to 7 is implemented.

Images