Labeling support device, labeling support method, and program
The labeling support device enhances the efficiency of label application in real-time voice recognition systems by using an existing model to estimate preliminary labels and providing interactive tools for updating, thus reducing the time and effort needed for manual labeling.
Patent Information
- Application Number
- JP2023503534
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2021-03-01
- Publication Date
- 2024-11-27
- Estimated Expiration
- 2041-03-01
AI Technical Summary
Existing techniques for real-time voice recognition in contact centers require a large amount of training data and manual labeling, which is time-consuming and burdensome for operators.
A labeling support device and method that uses an existing model to estimate preliminary labels and provides a user-friendly interface for updating and correcting labels, including features like highlighted words, audio playback, and waveform images to enhance labeling efficiency.
Facilitates easier and more efficient application of labels by operators, reducing the time and effort required for manual labeling and improving the accuracy of label assignment.
Smart Images

Figure 0007590672000001 
Figure 0007590672000002 
Figure 0007590672000003
Abstract
Description
[Technical field]
[0001] The present disclosure relates to a labeling assistance device, a labeling assistance method, and a program. [Background technology]
[0002] In recent years, with the aim of improving the quality of service in contact centers, systems have been proposed that perform real-time speech recognition of call contents and automatically present appropriate information to an operator by making full use of natural language processing technology.
[0003] For example, Non-Patent Document 1 discloses a technology that presents anticipated questions and answers (FAQs) to an operator in a dialogue between an operator and a customer. In this technology, the dialogue between an operator and a customer is speech-recognized, and converted into a semantically coherent spoken text by a "speech end determination" that determines whether the speaker has finished speaking. Next, a "message scene estimation" is performed to estimate which interaction scene in the dialogue the utterance corresponding to the speech text is from, such as the operator's greeting, confirmation of the customer's business, response to the business, or closing the dialogue. The dialogue is structured by the "message scene estimation". From the result of the "message scene estimation", a "FAQ search utterance determination" is performed to extract an utterance including the customer's business or an utterance by the operator confirming the customer's business. A search is performed on a database of FAQs prepared in advance using a search query based on the utterance extracted by the "FAQ search utterance determination", and the search results are presented to the operator.
[0004] The above-mentioned "end of speech determination", "interaction scene estimation" and "FAQ search utterance determination" use a model constructed by learning, using a deep neural network or the like, training data in which labels that classify utterances are assigned to speech texts. Therefore, "end of speech determination", "interaction scene estimation" and "FAQ search utterance determination" can be regarded as a sequence labeling problem in which sequential elements (utterances in a dialogue) are labeled. Non-Patent Document 2 describes a technology for estimating an interaction scene by learning, using a deep neural network including long and short-term memory, a large amount of training data in which sequential utterances are assigned labels corresponding to the interaction scenes in which the utterances are included. [Prior art documents] [Non-patent literature]
[0005] [Non-Patent Document 1] T. Hasegawa, Y. Sekiguchi, S. Yamada, and M. Tamoto, "Automatic Knowledge Support System to Support Operators' Responses," NTT Technical Review, vol. 31, no. 7, pp. 16-19, Jul. 2019. [Non-Patent Document 2] R. Masumura, S. Yamada, T. Tanaka, A. Ando, H. Kamiyama, and Y. Aono, “Online Call Scene Segmentation of Contact Center Dialogues based on Role Aware Hierarchical LSTM-RNNs,” Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), Nov. 2018. Summary of the Invention [Problem to be solved by the invention]
[0006] The techniques described in the above-mentioned Non-Patent Documents 1 and 2 require a large amount of training data to achieve a practical level of estimation accuracy. For example, according to Non-Patent Document 1, high estimation accuracy can be obtained by creating training data from conversation logs of a call center of about 1,000 calls and learning a model. The training data is created by an operator assigning a label to each spoken text while referring to the spoken text obtained by speech recognition of the spoken voice.
[0007] Fig. 13 is a diagram showing an example of adding a label to a spoken text. Fig. 13 shows an example of adding a label to a spoken text corresponding to an utterance in a dialogue between an operator and a customer (hereinafter, the spoken text corresponding to an utterance may be simply referred to as "spoken text"). In Fig. 13, the spoken text of the operator is shown in a speech bubble with a solid line, and the spoken text of the customer is shown in a speech bubble with a dotted line.
[0008] In the example shown in FIG. 13, training data for "end of speech determination" is created by assigning to each utterance text an end-of-speech label indicating whether the utterance is the end of speech or not. In addition, training data for "interaction scene estimation" is created by assigning to each utterance text a scene label indicating the interaction scene in which the utterance is included. In addition, training data for "FAQ search utterance determination" is created by assigning an interaction label indicating that the utterance indicates the customer's interaction among utterances included in the interaction scene of "subject understanding" in which the customer's interaction is understood, to the utterance indicating the customer's interaction, and by assigning an interaction confirmation label indicating that the utterance confirms the customer's interaction to the utterance in which the operator confirms the customer's interaction.
[0009] There is a problem that it takes a huge amount of work time to create a large amount of training data as shown in Fig. 13. In addition, in the example shown in Fig. 13, the labels of multiple items have a hierarchical structure. Specifically, a subject label or a subject confirmation label is assigned to a speech text to which a scene label of "subject understanding" has been assigned. That is, the structure is such that the scene label is a higher-level label, and the subject label / subject confirmation label is a lower-level label. If an operator assigns labels including multiple items having such a hierarchical structure without being given any guidelines, there is a problem that the burden on the operator increases.
[0010] Therefore, there is a demand for a technique that allows workers to apply labels more easily and efficiently.
[0011] An object of the present disclosure, made in consideration of the above problems, is to provide a label application assistance device, a label application assistance method, and a program that allow a worker to apply labels more easily and efficiently. [Means for solving the problem]
[0012] In order to solve the above problem, a labeling assistance device according to the present disclosure is a labeling assistance device that assists in assigning a label to each of a plurality of elements, and includes: an a priori label estimation unit that uses an existing model prepared in advance to estimate a priori labels, which are labels for each of the plurality of elements, and assigns the priori labels to each of the plurality of elements; an output unit that generates a labeling work screen that corresponds to each of the plurality of elements and shows the labels assigned to each of the plurality of elements in correspondence with each other, and outputs the labeling work screen to an external input / output interface; and a label update unit that, when the labels assigned to the elements are updated by the update operation via the labeling work screen, assigns the updated label to the element.
[0013] Furthermore, in order to solve the above-mentioned problems, a labeling assistance device according to the present disclosure is a labeling assistance device that assists in the assignment of labels to each of a plurality of elements, and includes a label assignment work screen for a user to update the labels assigned to the plurality of elements, and an output unit that generates the label assignment work screen showing each of the plurality of elements in correspondence with the labels assigned to each of the plurality of elements and outputs the generated label assignment work screen to an external input / output interface, the labels include labels of a plurality of items, and the output unit arranges the plurality of elements in a row on the label assignment work screen, and distributes and arranges the labels of the plurality of items on one side and the other side of the element corresponding to the labels based on a label structure of the plurality of items.
[0014] Furthermore, in order to solve the above-mentioned problems, a labeling assistance device according to the present disclosure is a labeling assistance device that assists in the assignment of labels to each of a plurality of elements, and includes a label assignment work screen for a user to update the labels assigned to the plurality of elements, and an output unit that generates the label assignment work screen showing each of the plurality of elements in correspondence with the labels assigned to each of the plurality of elements and outputs the generated label to an external input / output interface, the labels include labels of a plurality of items, and when a label to be updated is selected on the label assignment work screen or a label is updated, the output unit changes a display mode of the label to be updated or a label related to the updated label based on a hierarchical structure of the labels of the plurality of items.
[0015] Furthermore, in order to solve the above-mentioned problems, a labeling assistance method according to the present disclosure is a labeling assistance method for assisting in the assignment of labels to each of a plurality of elements, the method including the steps of: estimating a pre-label, which is a label for each of the plurality of elements, using an existing model prepared in advance, and assigning the pre-label to each of the plurality of elements; generating a labeling work screen for a user to perform an operation of updating the labels assigned to the plurality of elements, the labeling work screen showing each of the plurality of elements in correspondence with the labels assigned to each of the plurality of elements, and outputting the labeling work screen to an external input / output interface; and, when the labels assigned to the elements are updated by the update operation via the labeling work screen, assigning the updated labels to the elements.
[0016] In order to solve the above problem, a program according to the present disclosure causes a computer to function as the label attachment assistance device described above. Effect of the Invention
[0017] According to the labeling assistance device, the labeling assistance method, and the program according to the present disclosure, a worker can more easily and efficiently attach labels. [Brief description of the drawings]
[0018] [Figure 1] 1 is a block diagram showing a schematic configuration of a computer functioning as a labeling assistance device according to a first embodiment of the present disclosure. [Diagram 2] 1 is a diagram illustrating an example of a functional configuration of a labeling assistance device according to a first embodiment of the present disclosure. [Diagram 3] 3 is a flowchart showing an example of the operation of the label attachment assistance device shown in FIG. [Figure 4] 3 is a diagram showing an example of a labeling work screen generated by a labeling work screen output unit shown in FIG. 2. FIG. [Diagram 5] FIG. 11 is a diagram illustrating an example of the configuration of a labeling assistance device according to a second embodiment of the present disclosure. [Figure 6]6 is a flowchart showing an example of the operation of the label attachment assistance device shown in FIG. 5. [Figure 7] FIG. 13 is a diagram illustrating an example of the configuration of a labeling assistance device according to a third embodiment of the present disclosure. [Figure 8] 8 is a diagram showing an example of a labeling work screen generated by a labeling work screen output unit shown in FIG. 7. FIG. [Figure 9] 8 is a diagram showing an example of a waveform image generated by a waveform image generating section shown in FIG. 7. [Figure 10] 8 is a flowchart showing an example of the operation of the label attachment assistance device shown in FIG. [Figure 11A] FIG. 13 is a diagram showing an example of a first labeling work screen. [Figure 11B] FIG. 13 is a diagram showing an example of a second labeling work screen. [Figure 12] FIG. 13 is a diagram showing a comparison result of the work efficiency of labeling between a conventional method and the method according to the present disclosure. [Figure 13] FIG. 13 is a diagram showing an example of a label structure made up of a plurality of items. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0019] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.
[0020] (First embodiment)
[0021] 1 is a block diagram showing a hardware configuration of a labeling assistance device 10 according to a first embodiment of the present disclosure, which is a computer capable of executing program instructions. Here, the computer may be a general-purpose computer, a dedicated computer, a workstation, a PC (Personal Computer), an electronic notepad, etc. The program instructions may be program code, code segments, etc. for performing necessary tasks.
[0022] 1, the labeling assistance device 10 has a processor 110, a ROM (Read Only Memory) 120, a RAM (Random Access Memory) 130, a storage 140, an input unit 150, a display unit 160, and a communication interface (I / F) 170. Each component is connected to each other so as to be able to communicate with each other via a bus 190. The processor 110 is specifically a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), a SoC (System on a Chip), or the like, and may be composed of multiple processors of the same type or different types.
[0023] The processor 110 controls each of the components and executes various arithmetic processing. That is, the processor 110 reads a program from the ROM 120 or the storage 140, and executes the program using the RAM 130 as a working area. The processor 110 controls each of the components and executes various arithmetic processing according to the program stored in the ROM 120 or the storage 140. In this embodiment, the ROM 120 or the storage 140 stores a program according to the present disclosure.
[0024] The program may be provided in a form stored in a non-transitory storage medium such as a CD-ROM (Compact Disk Read Only Memory), a DVD-ROM (Digital Versatile Disk Read Only Memory), a USB (Universal Serial Bus) memory, etc. The program may also be provided in a form downloaded from an external device via a network.
[0025] The ROM 120 stores various programs and various data. The RAM 130 temporarily stores programs or data as a working area. The storage 140 is configured with a hard disk drive (HDD) or a solid state drive (SSD), and stores various programs including an operating system and various data.
[0026] The input unit 150 includes a pointing device such as a mouse and a keyboard, and is used to perform various input operations.
[0027] The display unit 160 is, for example, a liquid crystal display, and displays various information. The display unit 160 may function as the input unit 150 by adopting a touch panel system.
[0028] The communication interface 170 is an interface for communicating with other devices such as external devices (not shown), and uses standards such as Ethernet (registered trademark), FDDI, and Wi-Fi (registered trademark).
[0029] Next, the functional configuration of the label attachment assistance device 10 of this embodiment will be described.
[0030] FIG. 2 is a diagram showing an example of a functional configuration of a labeling support device 10 according to the present embodiment. The labeling support device 10 according to the present embodiment supports a worker who creates training data to label each of a plurality of sequential elements. In the following, an example of labeling a speech text obtained by speech recognition of an utterance in a dialogue between a plurality of speakers (operators and customers) at a contact center will be described. In the following, the labels include labels of a plurality of items having a hierarchical structure. Specifically, an example of attaching an end-of-speech label, a scene label, and a task label / task confirmation label will be described. As described above, the scene label and the task label / task confirmation label have a hierarchical structure in which the scene label is a higher-level label and the task label / task confirmation label is a lower-level label. However, the present disclosure is not limited to this example, and can be applied to attaching a label to each of any plurality of elements. In addition, the speech text may be not only a text of an utterance in a telephone call, but also an utterance in a text dialogue such as a chat. In addition, the speaker in the dialogue is not limited to a human being, but may be a robot or a virtual agent.
[0031] 2, the labeling assistance device 10 according to the present embodiment includes a pre-label estimation unit 11, a switching unit 12, a labeling work screen output unit 13 as an output unit, a label memory 14, and a label update unit 15. The pre-label estimation unit 11, the switching unit 12, the labeling work screen output unit 13, and the label update unit 15 may be configured with dedicated hardware such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field-Programmable Gate Array), or may be configured with one or more processors as described above. The labeling assistance device 10 includes a storage unit, and the storage unit has at least the label memory 14.
[0032] A plurality of speech texts (elements) included in a dialogue, such as those shown in Fig. 13, are sequentially input to the a priori label estimation unit 11. The a priori label estimation unit 11 performs a priori label estimation process, using an existing model prepared in advance, to estimate and assign labels (a priori labels) such as an end-of-speech label, a scene label, a subject label, and a subject confirmation label to each of the plurality of input speech texts.
[0033] The existing model is, for example, a model that has been trained on a small amount of training data created by a conventional method in which an operator manually assigns all labels, a model that has been trained on training data created using the label assignment support device 10 and that is created for a contact center other than the contact center that uses the model (desired model), or a general-purpose model that can be applied to multiple contact centers. That is, the existing model is, for example, a model that has been constructed by training on a smaller amount of training data than the desired model, or a model that has been constructed for a target that is different from the target to which the desired model is applied. Therefore, in a system to which the desired model is applied, the existing model usually has lower label estimation accuracy than the desired model.
[0034] The a priori label estimation unit 11 outputs the estimated a priori labels to the switching unit 12.
[0035] When an a priori label is output from the a priori label estimation unit 11, the switching unit 12 outputs the a priori label to the label assignment work screen output unit 13 and the label memory 14. In addition, when an updated label is output from a label update unit 15 described later, the switching unit 12 outputs the updated label to the label assignment work screen output unit 13 and the label memory 14.
[0036] The label assignment work screen output unit 13 receives the label (pre-label or updated label) output from the switching unit 12, the spoken text, and the label structure information. The label structure information is information about the label structure of the target system, or information about whether or not a long-term context should be considered when assigning each label. A label that should consider a long-term context is a label that should be assigned to an uttered text and should be determined based on the contents of multiple uttered texts including the uttered text. In the above example, the label that should consider a long-term context is, for example, a scene label. Note that in this embodiment, since the assignment of a label to an uttered text in a dialogue between an operator and a customer is taken as an example, the contents of multiple uttered texts are considered for a specific label, but the present disclosure is not limited to this. In short, the label structure information may include information about whether or not a label to be assigned to a certain element should be determined based on multiple elements including the element.
[0037] Based on the input spoken text, labels, and label structure information, the label assignment work screen output unit 13 generates a label assignment work screen for an update operation in which a worker (user) who assigns a label updates (modifies) the label assigned to the spoken text. The label assignment work screen output unit 13 outputs the generated label assignment work screen to the external input / output interface 1. In this way, the label assignment work screen output unit 13 performs label assignment work screen output processing to generate a label assignment work screen and output it to the external input / output interface 1, and causes the external input / output interface 1 to display the label assignment work screen.
[0038] The external input / output interface 1 is a device used by a worker to assign a label to an utterance text. When an update operation for updating a label assigned to an utterance text is performed via the displayed label assignment work screen, the external input / output interface 1 outputs the updated label to the label assignment assistance device 10. The external input / output interface 1 may have any configuration as long as it has a function of communicating with the label assignment assistance device 10, a function of displaying the label assignment work screen, and a function of accepting operation input by a worker. The label assignment work screen will be described in detail later.
[0039] The label memory 14 stores the label output from the switching unit 12. When the label updating unit 15 performs a label updating process described later, the label memory 14 outputs the stored label to the label updating unit 15 as a pre-update label.
[0040] When the label of the spoken text updated by the operator (updated label) is output from the external input / output interface 1, the label update unit 15 performs a label update process of replacing the pre-update label assigned to the spoken text output from the label memory 14 with the updated label and assigning it to the spoken text. In this way, when the label assigned to the spoken text (element) is updated by an update operation via the label assignment work screen, the label update unit 15 assigns the updated label to the spoken text. Instead of the pre-update label, identification information for identifying the pre-update label may be input to the label update unit 15.
[0041] The label update unit 15 outputs the label (updated label) assigned to the spoken text by the label update process to the switching unit 12 as an updated label. As described above, when the updated label is output from the label update unit 15, the switching unit 12 outputs the updated label to the label assignment work screen output unit 13. In response to the output of the updated label from the switching unit 12, the label assignment work screen output unit 13 generates a new label assignment work screen and outputs it to the external input / output interface 1. In this way, a label assignment work screen reflecting the label update by the update operation of the worker is displayed. Until an end operation is performed to end the label assignment work, the label update and the display of the label assignment work screen reflecting the update content are repeated.
[0042] In this embodiment, when an update operation is performed to update the prior labels assigned to the utterance text after assigning the prior labels to the utterance text using an existing model, the updated labels are assigned to the utterance text. Therefore, the worker only needs to update the labels that need to be updated, and can easily assign the labels. In addition, compared to the case where labels are assigned without a guideline for label assignment, the labels can be assigned more easily and the work time can be reduced. Therefore, the label assignment assistance device 10 according to this embodiment allows the worker to assign labels more easily and efficiently.
[0043] The external input / output interface 1 or the label assignment assistance device 10 may accumulate a history of label updates due to an update operation. The external input / output interface 1 may display the accumulated label update history. When an operation to end the label assignment work is performed, the label assigned to the spoken text at the time the operation to end is performed may be determined as the label of the spoken text.
[0044] Next, the operation of the labeling assistance device 10 according to this embodiment will be described.
[0045] FIG. 3 is a flowchart showing an example of the operation of the label attachment assistance device 10, and is a diagram for explaining the label attachment assistance method by the label attachment assistance device 10 according to the present embodiment.
[0046] When multiple spoken texts are input, the a priori label estimation unit 11 performs a priori label estimation process to estimate a priori labels for each of the multiple spoken texts input using an existing model prepared in advance (step S11). The a priori labels estimated by the a priori label estimation unit 11 are input to the label assignment work screen output unit 13 via the switching unit 12.
[0047] When a label is input via the switching unit 12, the label assignment work screen output unit 13 generates a label assignment work screen including the spoken text and the input label, and performs a label assignment work screen output process to output the screen to an interface for external input / output (step S12).
[0048] FIG. 4 is a diagram showing an example of a labeling work screen.
[0049] As shown in Fig. 4, the labeling work screen output unit 13 arranges the spoken texts of the operator and the customer in a row in chronological order on the labeling work screen. In addition, the labeling work screen output unit 13 arranges, in association with each spoken text, the start time when the speech started, the end time when the speech ended, and the labels assigned to the speech (scene label, subject label, subject confirmation label, and end of speech label). As shown in Fig. 4, the labeling work screen output unit 13 may display the spoken text of the operator and the spoken text of the customer in different colors. Note that in Fig. 4, the difference in color is expressed by different hatching.
[0050] As shown in FIG. 4, the label assignment work screen output unit 13 may arrange multiple elements in a row on the label assignment work screen, and may distribute and arrange the labels of multiple items on one side and the other side of the element corresponding to the labels based on the structure of the labels of the multiple items.
[0051] Generally, it is easier to assign labels to areas closer to the spoken text. Therefore, by arranging the spoken text in a line and allocating labels for multiple items to both sides of the spoken text, the area close to the spoken text can be effectively utilized, and the efficiency of the label assignment work can be improved.
[0052] In the example shown in FIG. 4, the scene label, the subject label, and the subject confirmation label are arranged on the left side of the spoken text, and the end-of-speech label is arranged on the right side of the spoken text. As described above, when assigning the scene label, the subject label, and the subject confirmation label to the spoken text, not only the spoken text but also the contents of the spoken text before and after the spoken text are taken into consideration. That is, the scene label, the subject label, and the subject confirmation label to the spoken text are labels that require consideration of the long-term context. On the other hand, when assigning the end-of-speech label to the spoken text, it is sufficient to mainly consider only the spoken text. Therefore, the label assignment work screen output unit 13 may arrange a label that requires consideration of the long-term context (a label for a certain element is determined based on a plurality of elements including the element) on the left side of the spoken text. Also, the label assignment work screen output unit 13 may arrange a label that does not require consideration of the long-term context (a label for a certain element is determined based mainly only on the element) on the right side of the spoken text.
[0053] In the example shown in FIG. 4, the labeling work screen output unit 13 places the subject label and the subject confirmation label closer to the spoken text than the scene label. As described above, the subject label or the subject confirmation label is added to the spoken text to which the scene label of "subject understanding" is added. That is, the scene label is a higher-level label, and the subject label / subject confirmation label is a lower-level label. Therefore, the labeling work screen output unit 13 may place the lower-level label closer to the spoken text among the labels of a plurality of items having a hierarchical structure. In this way, the lower the label, the easier it is to perform the labeling work by looking at the spoken text, so that the work efficiency can be improved. In addition, the end-of-speech label is mainly added by focusing on the end of the utterance. Therefore, by placing the end-of-speech label on the right side of the utterance text, the worker can easily see the end of the utterance text, so that the work efficiency of adding the end-of-speech label can be improved.
[0054] Furthermore, when a label to be updated is selected by the worker on the label assignment work screen, or when a label is updated by the worker, the label assignment work screen output unit 13 may change the display mode of the label to be updated or labels related to the updated label (higher label and lower label) based on the hierarchical structure of labels of multiple items. In the example shown in FIG. 4, it is assumed that the scene label of "subject understanding" is selected as the label to be updated or has been updated. In this case, the label assignment work screen output unit 13 changes the display mode by, for example, changing the display color of the subject label and the subject confirmation utterance label, which are lower labels of the scene label. This makes it easier for the worker to understand the labels related to the label to be updated, and the work efficiency of label assignment can be improved.
[0055] Furthermore, if a contradiction occurs between related labels when a higher-level label or a lower-level label is updated, the label assignment work screen output unit 13 may change the display mode of the label where the contradiction occurs. This can eliminate the contradiction between the labels of multiple items having a hierarchical structure and reduce the occurrence of errors in label assignment, thereby improving the efficiency of the label assignment work.
[0056] In addition, the labeling work screen output unit 13 may display utterance text that is not the target of the training data, for example, short utterance text such as fillers and "yes", in a different manner from other utterance text. This allows the worker to easily grasp utterance text that does not need to be labeled, thereby improving work efficiency.
[0057] 2 again, the label update unit 15 judges whether or not a label update operation has been performed (step S13). Specifically, the label update unit 15 judges whether or not a label update operation has been performed based on whether or not the updated label has been output from the external input / output interface 1.
[0058] If the updated label is not output from the external input / output interface 1 and, for example, an end operation is performed, the label update unit 15 determines that the label update operation has not been performed (step S13: No). If the label update unit 15 determines that the label update operation has not been performed, the label attachment assistance device 10 ends the process.
[0059] When the updated label is output from the external input / output interface 1 and it is determined that the label update operation has been performed (step S13: Yes), the label update unit 15 performs a label update process of assigning the updated label to the spoken text whose label has been updated by the update operation (step S14). After the label update process, the process returns to step S12, and the label assignment work screen output unit 13 generates a label assignment work screen including the updated label and outputs it to the external input / output interface 1.
[0060] As described above, the label assignment assistance device 10 according to the present embodiment includes a pre-label estimation unit 11, a label assignment work screen output unit 13, and a label update unit 15. The pre-label estimation unit 11 estimates a pre-label for each of a plurality of elements using an existing model prepared in advance, and assigns the pre-label to each of the plurality of elements. The label assignment work screen output unit 13 generates a label assignment work screen in which each of the plurality of elements is associated with a label assigned to each of the plurality of elements, and outputs the generated label assignment work screen to the external input / output interface 1. When a label assigned to an element is updated by an update operation via the label assignment work screen, the label update unit 15 assigns the updated label to the element.
[0061] The label assignment support method according to the present embodiment includes a step of assigning a pre-label (step S11), a step of outputting a label assignment work screen to the external input / output interface 1 (step S12), and a step of updating the label (step S14). In the step of assigning a pre-label, a pre-label for each of a plurality of elements is estimated using an existing model prepared in advance, and assigned to each of the plurality of elements. In the step of outputting the label assignment work screen to the external input / output interface 1, each of the plurality of elements is associated with a label assigned to each of the plurality of elements, a label assignment work screen for a label update operation is generated, and output to the external input / output interface 1. In the step of updating the label, when the label assigned to the element is updated by an update operation via the label assignment work screen, the updated label is assigned to the element.
[0062] When an update operation is performed after a pre-label is assigned to the spoken text using an existing model, the updated label is assigned to the spoken text, so that the worker only needs to update the labels that need to be updated, and can easily assign the labels. Furthermore, compared to the case where labels are assigned without a label assignment guideline, the labels can be assigned more easily, and the work time can be reduced. Therefore, according to the present disclosure, the worker can assign the labels more easily and efficiently. Furthermore, since a pre-label is obtained as a guideline for label assignment, a certain label assignment guideline can be presented to a worker who is not skilled in the label assignment work, and the worker can perform the label assignment work based on the guideline, and the learning period can be reduced.
[0063] The a priori label estimation unit 11 does not necessarily need to estimate a priori labels for all labels. For example, the a priori label estimation unit 11 may estimate a priori labels only for scene labels and subject / subject confirmation labels, and may not estimate a priori labels for end-of-speech labels. In this case, on the label assignment work screen shown in FIG. 4, labels for which a priori labels have not been estimated are left blank, and the worker may assign a label to the blank portion. Labels for which a priori labels are estimated or labels for which a priori labels are not estimated may be designated in advance by the worker, etc.
[0064] In the case where some of the labels are not estimated as described above, the estimation accuracy of certain a priori labels may be very poor, and it may be more efficient for the worker to assign the labels rather than correcting the a priori labels.
[0065] Second embodiment Fig. 5 is a diagram showing a configuration example of a labeling assistance device 10A according to a second embodiment of the present disclosure. In Fig. 5, the same components as those in Fig. 2 are denoted by the same reference numerals, and description thereof will be omitted.
[0066] The labeling assistance device 10A of this embodiment differs from the labeling assistance device 10 of the first embodiment in that a pre-label correction unit 21 is added and the switching unit 12 is changed to a switching unit 12A.
[0067] The a priori label correction unit 21 receives the a priori labels estimated by the a priori label estimation unit 11. The a priori label correction unit 21 performs a priori label correction process to correct labels determined to be erroneous based on a predetermined rule among the input a priori labels. The a priori label correction unit 21 outputs the labels after the a priori label correction process (corrected labels for labels determined to be erroneous among the a priori labels of each of the multiple utterance texts) to the switching unit 12A as corrected a priori labels.
[0068] As a pre-label correction process, the pre-label correction unit 21 corrects the scene label of an extremely short conversation scene consisting of only a single utterance, for example, so that the scene label is incorporated into an adjacent conversation scene. As a pre-label correction process, the pre-label correction unit 21 appropriately divides a speech text with a long utterance length and assigns an end-of-speech label. As a pre-label correction process, if the estimation probability of a label estimated by an existing model is an ambiguous value that does not necessarily mean that the estimation is either incorrect or correct (for example, if the estimation probability is a value in the range from 0 to 1, a value in a predetermined range including 0.5), the label is considered indefinite.
[0069] When the corrected pre-advance label is output from the pre-advance label correction unit 21, the switching unit 12A outputs the corrected pre-advance label to the labeling work screen output unit 13 and the label memory 14. In addition, when the updated label is output from the label update unit 15, the switching unit 12A outputs the updated label to the labeling work screen output unit 13 and the label memory 14.
[0070] Next, the operation of the labeling assistance device 10A according to this embodiment will be described.
[0071] Fig. 6 is a flowchart showing an example of the operation of the labeling assistance device 10A according to the present embodiment. In Fig. 6, the same processes as those in Fig. 3 are denoted by the same reference numerals, and the description thereof will be omitted.
[0072] When the a-priori label estimation process is performed (step S11), the a-priori label correction unit 21 performs a a-priori label correction process for correcting, among the estimated a-priori labels, labels that are determined to be incorrect based on a predetermined rule (step S21). The a-priori label correction unit 21 outputs the corrected a-priori labels to the switching unit 12A. The switching unit 12A outputs the corrected a-priori labels output from the a-priori label correction unit 21 to the label assignment work screen output unit 13 and the label memory 14. Thereafter, as in the first embodiment, the label assignment work screen output unit 13 performs a label assignment work screen output process (step S12).
[0073] As described above, in this embodiment, the labeling assistance device 10A includes the pre-label correction unit 21. The pre-label correction unit 21 corrects, among the estimated pre-labels, any label that is determined to be erroneous based on a predetermined rule.
[0074] Therefore, labels that are determined to be incorrect based on predetermined rules are corrected and the labeling work screen is output, reducing the need for workers to update labels and improving work efficiency.
[0075] (Third embodiment) Fig. 7 is a diagram showing an example of the configuration of a labeling assistance device 10B according to the third embodiment of the present disclosure. In Fig. 7, the same components as those in Fig. 5 are denoted by the same reference numerals, and the description thereof will be omitted.
[0076] The labeling support device 10B of this embodiment differs from the labeling support device 10A of the second embodiment in that it adds an emphasis word search unit 31, a voice extraction unit 32, and a waveform image generation unit 33, and in that the labeling work screen output unit 13 is changed to a labeling work screen output unit 13B.
[0077] The emphasis word search unit 31 receives emphasis target words, which are words to be highlighted in the utterance text, and the utterance text. The emphasis target words are specific words that are useful for the labeling process. An example of an emphasis target word is a word such as "contract" that is expected to appear at the beginning of the "identification" interaction scene. The emphasis target words may be specified in advance by the worker. In addition, the emphasis target words may be automatically determined during the labeling process based on the distribution of the occurrence frequency of words that tend to appear in each interaction scene in the entire dialogue.
[0078] The emphasis word search unit 31 performs emphasis word search processing to search for emphasis target words from the utterance text. The emphasis word search unit 31 determines the words searched by the emphasis processing as emphasis points, generates emphasised utterance text in which the emphasis points in the utterance text are emphasized, and outputs the emphasised utterance text to the label assignment work screen output unit 13B.
[0079] The voice extraction unit 32 receives the spoken voice on which the spoken text is based and the voice playback time output from the external input / output interface 1. The voice playback time is a time designated by the worker as a starting point for playing the spoken voice in the dialogue. The voice extraction unit 32 performs a voice extraction process to extract a voice utterance from the time designated by the voice playback time from the inputted spoken voice. The voice extraction unit 32 performs a voice output process to output the extracted spoken voice to the label assignment work screen output unit 13B.
[0080] The waveform image generating unit 33 receives the spoken voice that is the basis of the spoken text and the waveform display time output from the external input / output interface 1. The waveform display time is the time designated by the worker as the starting point for displaying a waveform image showing the waveform of the spoken voice in the dialogue. The waveform image generating unit 33 performs a waveform image generating process to generate a waveform image of the spoken voice from the time designated by the waveform display time from the inputted spoken voice. The waveform image generating unit 33 performs a waveform image output process to output the generated waveform image to the labeling work screen output unit 13B.
[0081] The label assignment work screen output unit 13B receives the label (corrected pre-label or updated label) output from the switching unit 12A, the highlighted utterance text output from the highlighted word search unit 31, and label structure information. The label assignment work screen output unit 13B performs a label assignment work screen output process of generating a label assignment work screen based on the input highlighted utterance text, label, and label structure information and outputting the screen to the external input / output interface 1.
[0082] Fig. 8 is a diagram showing an example of a labeling work screen output by the labeling work screen output unit 13 B. In Fig. 8, the description of the same parts as in Fig. 4 will be omitted.
[0083] As shown in Fig. 8, the labeling work screen output unit 13B highlights the highlighted parts of the utterance text that have been highlighted by the highlighted word search unit 31 by changing the color, underlining the highlighted parts, or the like. Fig. 8 shows an example in which the highlighted parts are underlined. Furthermore, the labeling work screen output unit 13B arranges an audio playback button for playing the spoken voice of the utterance in association with the utterance text. Furthermore, the labeling work screen output unit 13 arranges a waveform display button for displaying a waveform image of the spoken voice of the utterance in association with the utterance text.
[0084] For example, when a voice playback operation is performed to select a voice playback button, the start time of the spoken text associated with the selected voice playback button is output as a voice playback time from the external input / output interface 1 to the labeling support device 10B. When the voice playback time is input, the voice extraction unit 32 outputs the spoken voice from the voice playback time to the labeling work screen output unit 13B. The labeling work screen output unit 13B outputs the voice utterance output from the voice extraction unit 32 to the external input / output interface 1 and plays the voice. As described above, the spoken text may be an utterance in a dialogue via text such as a chat. In that case, the labeling work screen output unit 13B may output a voice corresponding to the spoken text using a technology such as voice synthesis.
[0085] The section for playing the speech can be specified, for example, by dragging from the speech playback button corresponding to the spoken text from which the speech playback starts to the speech playback button corresponding to the spoken text from which the speech playback ends. When the start point for starting the speech playback is specified, the speech playback starts, and the speech playback may continue, for example, until the end of the speech is reached or a stop operation is performed.
[0086] By playing back the spoken voice, for example, when the readability of the speech recognition result (spoken text) is low and the spoken text alone does not provide enough information for labeling, the worker can check the content of the utterance, which results in more accurate labeling.
[0087] Furthermore, for example, when a waveform display operation is performed to select a waveform display button, the start time of the spoken text associated with the selected waveform display button is output as the waveform display time from the external input / output interface 1 to the labeling assistance device 10B. When the waveform display time is input, the waveform image generating unit 33 generates a waveform image of the spoken voice from the waveform display time, and outputs it to the labeling work screen output unit 13B. The labeling work screen output unit 13B outputs the waveform image output from the waveform image generating unit 33 to the external input / output interface 1 to be displayed.
[0088] FIG. 9 is a diagram showing an example of a waveform image generated by the waveform image generating section 33. As shown in FIG.
[0089] When the section for generating the waveform image includes both the operator's speech and the customer's speech, the waveform image generating unit 33 generates a waveform image that separates the waveform of the operator's speech from the waveform of the customer's speech, as shown in Figure 9.
[0090] The section for displaying the waveform image can be specified, for example, by dragging from the waveform image button corresponding to the spoken text for which the display of the waveform image is to begin to be performed to the waveform image button corresponding to the spoken text for which the display of the waveform image is to end to be performed. When a predetermined display operation, for example, pressing the waveform display button, is performed, the waveform image generating unit 33 displays the waveform image for the section corresponding to the waveform display button. In addition, when a start point for starting the display of the waveform image is specified, the waveform image generating unit 33 may start displaying the waveform image, and continue displaying the waveform image until a stop operation is performed.
[0091] By displaying the waveform image, a skilled worker can refer to the waveform image and assign appropriate labels, for example, to understand that a call is on hold when only the operator is speaking softly, or that the topic is changing when the tone of voice becomes stronger. Also, by displaying the waveform image, when only the spoken text of the operator or the customer continues for a long period of time, it is possible to easily check whether the other's speech is missing. As a result, the efficiency of the label assignment work can be improved.
[0092] In this way, the spoken voice is played in response to the voice playback operation, and a waveform image is displayed in response to the waveform display operation, allowing the worker to selectively refer to the information he or she needs depending on the readability of the voice recognition results (spoken text) and the worker's skills and experience, etc.
[0093] When the readability of the spoken text is determined to be low, the labeling work screen output unit 13B may highlight the voice playback button and the waveform display button corresponding to the spoken text, for example, by blinking them. The readability of the spoken text is evaluated, for example, by a mechanism separate from the labeling support device 10B.
[0094] Next, the operation of the labeling assistance device 10B according to this embodiment will be described.
[0095] Fig. 10 is a flowchart showing an example of the operation of the label attachment assistance device 10B according to the present embodiment. In Fig. 10, the same processes as those in Fig. 6 are denoted by the same reference numerals, and the description thereof will be omitted.
[0096] When the advance label estimation process (step S11) and the advance label correction process (step S21) are completed, the emphasis word search unit 31 performs an emphasis word search process for searching for emphasis target words from the utterance text (step S31). Note that the emphasis word search process may be performed in parallel with the advance label estimation process and the advance label correction process.
[0097] Next, the labeling work screen output unit 13B performs a labeling work screen output process to generate a labeling work screen based on the emphasized utterance text in which the words to be emphasized are emphasized by the emphasis word search unit 31, and the labels and label structure information output from the switching unit 12A, and outputs the generated screen to the external input / output interface 1 (step S32). As shown in Fig. 8, the labeling work screen output unit 13B highlights the parts of the utterance text emphasized by the emphasis word search unit 31 by displaying them in a different color or by underlining them. As shown in Fig. 8, the labeling work screen output unit 13B also arranges, in association with each utterance text, an audio playback button for playing the speech of the utterance and a waveform image button for displaying a waveform image of the speech of the utterance.
[0098] When the label attachment work screen output process is performed, the label update unit 15 determines whether or not a label update operation has been performed (step S13).
[0099] If the updated label is not output from the external input / output interface 1 and, for example, an end operation is performed, the label update unit 15 determines that the label update operation has not been performed (step S13: No). If the label update unit 15 determines that the label update operation has not been performed, the label attachment assistance device 10 ends the process.
[0100] When the updated label is output from the external input / output interface 1 and it is determined that a label update operation has been performed (step S13: Yes), the label update unit 15 performs a label update process to assign the updated label to the spoken text whose label has been updated by the update operation (step S14).
[0101] When the labeling work screen output process is performed, the audio extraction unit 32 determines whether or not an audio playback operation has been performed (step S33). Specifically, the audio extraction unit 32 determines whether or not an audio playback time has been output from the external input / output interface 1.
[0102] If it is determined that an audio playback operation has been performed (step S33: Yes), the audio extraction unit 32 performs an audio extraction process to extract the speech audio from the audio playback time from the speech audio of the entire dialogue, and an audio output process to output the extracted speech audio to the label assignment work screen output unit 13B as extracted speech audio (step S34).
[0103] When the voice extraction unit 32 determines that a voice playback operation is not being performed (step S33: No), the labeling assistance device 10 ends the process.
[0104] When the labeling work screen output process is performed, the waveform image generating unit 33 determines whether or not a waveform display operation has been performed (step S35). Specifically, the waveform image generating unit 33 determines whether or not a waveform display time has been output from the external input / output interface 1.
[0105] When the waveform image generating unit 33 determines that the waveform display operation is not being performed (step S35: No), the labeling assistance device 10B ends the process.
[0106] If it is determined that a waveform display operation has been performed (step S35: Yes), the waveform image generation unit 33 performs a waveform image generation process to generate a waveform image of the spoken voice from the waveform display time and a waveform image output process to output the generated waveform image to the label assignment work screen output unit 13B (step S36).
[0107] After the process of step S14, step S34, or step S36, the labeling work screen output unit 13B performs a labeling work screen output process (step S32). When an extracted uttered voice is output from the voice extraction unit 32, the labeling work screen output unit 13B outputs the extracted uttered voice to the external input / output interface 1 for playback. When a waveform image is output from the waveform image generation unit 33, the labeling work screen output unit 13B outputs the waveform image to the external input / output interface 1 for display.
[0108] As described above, the labeling support device 10B according to the present embodiment includes an emphasis word search unit 31 and a labeling work screen output unit 13B. The emphasis word search unit 31 searches for words to be emphasized from the spoken text, and determines the searched words as the emphasized parts. The labeling work screen output unit 13B highlights the words determined as the emphasized parts by the emphasis word search unit 31 in the spoken text on the labeling work screen.
[0109] Therefore, words that are useful for labeling can be highlighted, and the efficiency of the labeling process can be improved.
[0110] The labeling support device 10B according to the present embodiment includes a voice extraction unit 32 and a labeling work screen output unit 13B. The voice extraction unit 32 extracts the spoken voice of the utterance selected by the voice playback operation. The labeling work screen output unit 13B outputs the spoken voice extracted by the voice extraction unit 32 to the external input / output interface 1 for playback.
[0111] Therefore, when the readability of the spoken text is low and the spoken content cannot be confirmed, the spoken audio can be played back to allow the worker to confirm the spoken content, thereby improving the work efficiency of labeling.
[0112] Moreover, the labeling support device 10B according to this embodiment includes a waveform image generating unit 33 and a labeling work screen output unit 13B. The waveform image generating unit 33 generates a waveform image of a speech sound selected by a waveform display operation. The labeling work screen output unit 13B outputs the waveform image generated by the waveform image generating unit 33 to the external input / output interface 1 for display.
[0113] This makes it possible to check for topic changes and missing speech from the waveform image, thereby improving the efficiency of labeling work.
[0114] In the present embodiment, the labeling assistance device 10B has been described using an example having a function of searching for and highlighting a search target word, a function of playing back an utterance, and a function of displaying a waveform image, but the present disclosure is not limited thereto. The labeling assistance device 10B may have at least one of the above-mentioned three functions. In addition, the labeling assistance device 10 according to the first embodiment and the labeling assistance device 10A according to the second embodiment may have at least one of the above-mentioned three functions. Therefore, when an utterance text is selected via the labeling work screen, the labeling work screen output unit 13B may output at least one of the utterance voice of the utterance corresponding to the selected utterance text and the audio waveform of the utterance voice of the utterance corresponding to the selected utterance text to the external input / output interface 1.
[0115] In addition, in the first to third embodiments described above, an example has been described in which all of the labels of multiple items to be assigned to an element (spoken text) are arranged on a single label assignment work screen, but the present disclosure is not limited to this.
[0116] As described above, an element may be assigned multiple labels. The multiple labels assigned to one element mainly include a label (first label) of an item that can be independently assigned to one element. The multiple labels assigned to one element include a label having a hierarchical structure, or a label (second label) of an item that determines the label of an element based on multiple elements including the label. The first label is, for example, an end-of-speech label. As described above, the scene label and the subject label / subject confirmation label have a hierarchical structure. Also, for example, the scene label is assigned taking into consideration the contents of multiple spoken texts. Therefore, the second label is, for example, a scene label, a subject label, and a subject confirmation label.
[0117] The label assignment work screen output units 13, 13B may output and display a label assignment work screen (first label assignment work screen) for the first label update operation and a label assignment work screen (second label assignment work screen) for the second label update operation independently to the external input / output interface 1.
[0118] 11A is a diagram showing an example of a first label assignment work screen for updating a first label. As shown in FIG. 11A, the label assignment work screen output unit 13, 13B may arrange the spoken text and the end-of-speech label in association with each other on the first label assignment work screen.
[0119] FIG. 11B is a diagram showing an example of a second label assignment work screen for updating the second label. As shown in FIG. 11B, the label assignment work screen output unit 13, 13B may arrange the waveform of the operator's speech and the waveform of the customer's speech on the second label assignment work screen. In addition, the label assignment work screen output unit 13, 13B may arrange the operator's speech text and the customer's speech text in chronological order along a predetermined direction (for example, from left to right) on the second label assignment work screen. In addition, the label assignment work screen output unit 13, 13B may arrange the scene label, the subject label, and the subject confirmation label assigned to the speech text in accordance with the position of the speech text. The second label has a hierarchical structure, cannot be assigned independently by one element, and should be assigned while taking other elements into consideration. Therefore, by arranging elements (utterance text) in a sequential manner along a predetermined direction as in the second labeling work screen shown in Fig. 11B, it becomes easier to assign labels that take into account the elements before and after. As a result, the efficiency of the labeling work can be improved.
[0120] FIG. 12 is a diagram showing a comparison result of the work efficiency of labeling by a conventional method (first method) in which a worker assigns labels from the beginning and a method (second method) according to the present disclosure in which a prior label estimated by an existing model is updated. FIG. 12 shows the time required for subject A and subject B to assign labels (scene label, subject label, subject confirmation label, and end of speech label) to the speech text included in 14 calls by the first method and the second method. Subject A is a worker who has been engaged in the task of creating teacher data for several years and is skilled in labeling. Subject B is a worker who has been engaged in the task of creating teacher data for only a few months and is less skilled in labeling than subject A. The existing model used in the second method was created by learning teacher data for 100 calls.
[0121] As shown in Fig. 12, in the case of assigning scene labels, subject labels, and subject confirmation labels, the time required for label assignment was shorter in the second method than in the first method for both subjects A and B. Similarly, in the case of assigning end-of-speech labels, the time required for label assignment was shorter in the second method than in the first method for both subjects A and B. From these results, it was found that the present disclosure allows workers to assign labels more easily and efficiently.
[0122] The following supplementary notes are further disclosed regarding the above embodiment.
[0123] (Additional note 1) Memory, at least one processor coupled to the memory; Including, The processor, Using an existing model prepared in advance, a prior label, which is a label for each of a plurality of elements, is estimated and assigned to each of the plurality of elements; generating a label assignment work screen for a user to perform an operation of updating the labels assigned to the plurality of elements, the label assignment work screen showing each of the plurality of elements in association with the label assigned to each of the plurality of elements, and outputting the generated label assignment work screen to an external input / output interface; When the label assigned to the element is updated by the update operation via the label assignment work screen, the label assignment assistance device assigns the updated label to the element.
[0124] (Additional note 2) Memory, at least one processor coupled to the memory; Including, The processor, generating a label assignment work screen for a user to perform an operation of updating the labels assigned to the plurality of elements, the label assignment work screen showing each of the plurality of elements in association with the label assigned to each of the plurality of elements, and outputting the generated label assignment work screen to an external input / output interface; The label includes labels for a plurality of items; A labeling support device that arranges the multiple elements in a row on the labeling work screen, and distributes and arranges the labels of the multiple items on one side and the other side of the element corresponding to the labels based on the structure of the labels of the multiple items.
[0125] (Additional note 3) Memory, at least one processor coupled to the memory; Including, The processor, generating a label assignment work screen for a user to perform an operation of updating the labels assigned to the plurality of elements, the label assignment work screen showing each of the plurality of elements in association with the label assigned to each of the plurality of elements, and outputting the generated label assignment work screen to an external input / output interface; The label includes labels for a plurality of items; When a label to be updated is selected on the labeling work screen or a label is updated, the output unit changes a display mode of the label to be updated or a label related to the updated label based on a hierarchical structure of the labels of the multiple items.
[0126] (Additional note 4) A non-transitory storage medium storing a program executable by a computer, the program causing the computer to function as the labeling assistance device according to any one of claims 1 to 3.
[0127] All publications, patent applications, and standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, and standard was specifically and individually indicated to be incorporated by reference. [Explanation of symbols]
[0128] 1 External I / O Interface 10,10A,10B 11 Pre-label estimation 12,12A Switching section 13, 13B Label assignment work screen output section (output section) 14 Label Memory 15 Label Update Department 21 Pre-Label Correction Section 31 Emphasis word search section 32 Audio Extraction Section 33 Waveform image generator 110 Processor 120 ROM 130 RAM 140 Storage 150 Input section 160 Display section 170 Communication Interface 190 Bus
Claims
1. A labeling assistance device that assists in the assignment of labels to each of a plurality of elements, comprising: an a priori label estimation unit that estimates a priori labels, which are labels for each of the plurality of elements, by using an existing model that has been prepared in advance, and assigns the a priori labels to each of the plurality of elements; an output unit that generates a label assignment work screen for a user to perform an operation of updating the labels assigned to the plurality of elements, the label assignment work screen showing each of the plurality of elements in association with the label assigned to each of the plurality of elements, and outputs the generated label assignment work screen to an external input / output interface; a label updating unit that, when the label assigned to the element is updated by the update operation via the label assignment work screen, assigns the updated label to the element, The labeling assistance device further includes an a priori label correction unit that corrects a label that is determined to be incorrect based on a predetermined rule, among the a priori labels estimated by the a priori label estimation unit.
2. A labeling assistance device that assists in the assignment of labels to each of a plurality of elements, comprising: an a priori label estimation unit that estimates a priori labels, which are labels for each of the plurality of elements, by using an existing model that has been prepared in advance, and assigns the a priori labels to each of the plurality of elements; an output unit that generates a label assignment work screen for a user to perform an operation of updating the labels assigned to the plurality of elements, the label assignment work screen showing each of the plurality of elements in association with the label assigned to each of the plurality of elements, and outputs the generated label assignment work screen to an external input / output interface; a label updating unit that, when the label assigned to the element is updated by the update operation via the label assignment work screen, assigns the updated label to the element, When the element is a speech text corresponding to an utterance by each of a plurality of speakers in a dialogue by the plurality of speakers, The labeling support device, when the spoken text is selected on the labeling work screen, outputs at least one of a spoken voice corresponding to the selected spoken text and a waveform image of the spoken voice corresponding to the selected spoken text to the external input / output interface.
3. A labeling assistance device that assists in labeling of a plurality of elements, comprising: a label assignment work screen for a user to perform an operation of updating the labels assigned to the plurality of elements, the label assignment work screen including an output unit that generates the label assignment work screen showing each of the plurality of elements in association with the label assigned to each of the plurality of elements, and outputs the generated label assignment work screen to an external input / output interface; The label includes labels for a plurality of items; The output unit arranges the multiple elements in a row on the label assignment work screen, and distributes and arranges the labels of the multiple items on one side and the other side of the element corresponding to the labels based on the label structure of the multiple items.
4. A labeling assistance device that assists in labeling of a plurality of elements, comprising: a label assignment work screen for a user to perform an operation of updating the labels assigned to the plurality of elements, the label assignment work screen including an output unit that generates the label assignment work screen showing each of the plurality of elements in association with the label assigned to each of the plurality of elements, and outputs the generated label assignment work screen to an external input / output interface; The label includes labels for a plurality of items; When a label to be updated is selected on the labeling work screen or a label is updated, the output unit changes a display mode of the label to be updated or a label related to the updated label based on a hierarchical structure of the labels of the multiple items.
5. In the labeling assistance device according to claim 3 or 4, an a priori label estimation unit that estimates a priori labels, which are labels for each of the plurality of elements, by using an existing model that has been prepared in advance, and assigns the a priori labels to each of the plurality of elements; The labeling assistance device further includes an a priori label correction unit that corrects a label that is determined to be incorrect based on a predetermined rule, among the a priori labels estimated by the a priori label estimation unit.
6. In the labeling assistance device according to claim 3 or 4, When the element is a speech text corresponding to an utterance by each of a plurality of speakers in a dialogue by the plurality of speakers, The labeling support device, when the spoken text is selected on the labeling work screen, outputs at least one of a spoken voice corresponding to the selected spoken text and a waveform image of the spoken voice corresponding to the selected spoken text to the external input / output interface.
7. A labeling assistance device that assists in the assignment of labels to each of a plurality of elements, comprising: an a priori label estimation unit that estimates a priori labels, which are labels for each of the plurality of elements, by using an existing model that has been prepared in advance, and assigns the a priori labels to each of the plurality of elements; an output unit that generates a label assignment work screen for a user to perform an operation of updating the labels assigned to the plurality of elements, the label assignment work screen showing each of the plurality of elements in association with the label assigned to each of the plurality of elements, and outputs the generated label assignment work screen to an external input / output interface; a label updating unit that, when the label assigned to the element is updated by the update operation via the label assignment work screen, assigns the updated label to the element, wherein the labels include a first label and a second label, the first label is a label of an independently labelable item; the second label is a label having a hierarchical structure or a label of an item in which a label of a certain element is determined based on a plurality of elements including the certain element, the output unit outputs a first labeling work screen for an update operation of the first label and a second labeling work screen for an update operation of the second label to the external input / output interface independently.
8. A labeling support method for supporting labeling of a plurality of elements, comprising: A step of estimating a prior label, which is a label for each of the plurality of elements, using an existing model prepared in advance and assigning the prior label to each of the plurality of elements; generating a label assignment work screen for a user to perform an operation of updating the labels assigned to the plurality of elements, the label assignment work screen showing each of the plurality of elements in association with the label assigned to each of the plurality of elements, and outputting the generated label assignment work screen to an external input / output interface; when the label assigned to the element is updated by the update operation via the label assignment work screen, assigning the updated label to the element, A label assignment support method, comprising: correcting a label determined to be incorrect based on a predetermined rule among the pre-assigned labels.
9. A labeling support method for supporting labeling of a plurality of elements, comprising: A step of estimating a prior label, which is a label for each of the plurality of elements, using an existing model prepared in advance and assigning the prior label to each of the plurality of elements; generating a label assignment work screen for a user to perform an operation of updating the labels assigned to the plurality of elements, the label assignment work screen showing each of the plurality of elements in association with the label assigned to each of the plurality of elements, and outputting the generated label assignment work screen to an external input / output interface; when the label assigned to the element is updated by the update operation via the label assignment work screen, assigning the updated label to the element, When the element is a speech text corresponding to an utterance by each of a plurality of speakers in a dialogue by the plurality of speakers, When the spoken text is selected on the label assignment work screen, at least one of the spoken voice corresponding to the selected spoken text and a waveform image of the spoken voice corresponding to the selected spoken text is output to the external input / output interface.
10. A labeling support method for supporting labeling of a plurality of elements, comprising the steps of: A labeling assistance device that assists in labeling of a plurality of elements, comprising: a step of generating a label assignment work screen for a user to perform an operation of updating the labels assigned to the plurality of elements, the label assignment work screen showing each of the plurality of elements in association with the labels assigned to each of the plurality of elements, and outputting the generated label assignment work screen to an external input / output interface; The label includes labels for a plurality of items; a labeling support method for arranging the plurality of elements in a row on the labeling work screen, and distributing and arranging the labels of the plurality of items on one side and the other side of the element corresponding to the labels based on a structure of the labels of the plurality of items.
11. A labeling support method for supporting labeling of a plurality of elements, comprising the steps of: a step of generating a label assignment work screen for a user to perform an operation of updating the labels assigned to the plurality of elements, the label assignment work screen showing each of the plurality of elements in association with the labels assigned to each of the plurality of elements, and outputting the generated label assignment work screen to an external input / output interface; The label includes labels for a plurality of items; When a label to be updated is selected on the label assignment work screen or a label is updated, a display manner of the label to be updated or a label related to the updated label is changed based on a hierarchical structure of the labels of the multiple items.
12. A program for causing a computer to function as the label attachment assisting device according to claim 1.
13. A program for causing a computer to function as the labeling assistance device described in claim 2.
14. A program for causing a computer to function as the labeling assistance device described in claim 3.
15. A program for causing a computer to function as the labeling assistance device described in claim 4.
Citation Information
Patent Citations
Learning data creation device, learning data creation method, and program
WO2020036191A1
