Methods, apparatus, electronic devices and media for obtaining slot labeling data

Abstract

This application discloses a method, apparatus, electronic device, and medium for acquiring slot annotation data. By applying the technical solution of this application, a unified boundary determination can be assigned to the words to be annotated, thereby ensuring that the probability distribution value corresponding to each slot is determined based on its boundary. Simultaneously, useless symbols and words in the document are discarded, thus ensuring a smaller computational load and reducing the burden on computing devices. This avoids the problem of low annotation efficiency caused by inconsistent boundary and category determinations of the data to be annotated in the document, which occurs in related technologies.

Description

Technical Field

[0001] This application relates to data processing technology, and in particular to a method, apparatus, electronic device and medium for acquiring slot labeling data. Background Technology

[0002] In related technologies, slot recognition (including entity recognition, keyword recognition, etc.) requires manually labeled data for training, evaluation, and testing. Reliable and robust recognition models often require a large amount of data, especially for tasks with many slot categories. Manual labeling often suffers from inconsistencies.

[0003] Specifically, in slot identification, the main problems are inconsistencies in boundary determination and category determination. These issues can impair data quality and reduce the effectiveness of annotation. Therefore, designing a method that can efficiently identify labeled data in documents has become a problem that needs to be solved. Summary of the Invention

[0004] This application provides a method, apparatus, electronic device, and medium for acquiring slot annotation data. This solves the problem of low annotation efficiency caused by inconsistent boundary and category determinations of the data to be annotated in documents, which exists in related technologies.

[0005] According to one aspect of the embodiments of this application, a method for obtaining slot labeling data is provided, comprising:

[0006] Obtain all complete slot information contained in the text to be annotated. The complete slot information includes the slot, the slot content carried in the slot, and the slot category label. The slot category label is used to represent the category to which the slot content belongs.

[0007] Based on the slot category tags, determine the boundary words of the complete slot information;

[0008] Determine the quantity distribution and probability distribution of the boundary words to the slot in each complete slot information;

[0009] The slot content corresponding to the complete slot information with a value greater than a preset threshold in the probability distribution is selected as the annotation data of the text to be annotated.

[0010] Optionally, in another embodiment based on the method described above in this application, determining the boundary words of the complete slot information based on the slot category label includes:

[0011] The category label located to the left of the slot category label is determined as the left category label of the complete slot information; and the category label located to the right of the slot category label is determined as the right category label of the complete slot information.

[0012] Based on the left category label and the right category label of the complete slot information, the boundary words of the complete slot information are determined, and the boundary words include left boundary words and right boundary words.

[0013] Optionally, in another embodiment based on the method described above in this application, determining the boundary words of the complete slot information based on the left category label and the right category label of the complete slot information includes:

[0014] The first field to the left of the left category label in the complete slot information is taken as the left boundary word of the complete slot information; and...

[0015] The first field to the right of the right category label in the complete slot information is taken as the right boundary word of the complete slot information.

[0016] Optionally, in another embodiment based on the method described above in this application, determining the distribution value of the number of boundary words to the slot in each complete slot information includes:

[0017] Determine a first number of slot fields contained between the left boundary word and the right category label of each complete slot information; and determine a second number of slot fields contained between the right boundary word and the left category label of each complete slot information;

[0018] The first quantity and the second quantity are used as the corresponding quantity distribution values ​​in the complete slot information.

[0019] Optionally, in another embodiment based on the method described above in this application, the step of using the first quantity and the second quantity as the corresponding quantity distribution values ​​in the complete slot information includes:

[0020] Detect whether the adjacent side of each slot content in the first number of slot fields is a slot, and obtain a first detection result; and detect whether the adjacent side of each slot content in the second number of slot fields is a slot, and obtain a second detection result.

[0021] The first quantity and the first detection result, and the second quantity and the second detection result are used as the corresponding quantity distribution values ​​in the complete slot information.

[0022] Optionally, in another embodiment based on the method described above in this application, determining the probability distribution value from the boundary word to the slot in each complete slot information includes:

[0023] The quantity distribution value is normalized to obtain the probability distribution value corresponding to the complete slot information.

[0024] Optionally, in another embodiment based on the method described above in this application, retrieving all complete slot information contained in the text to be annotated includes:

[0025] Obtain the metrics to be labeled, which include the number of slots, the number of words in the content of the slots, and the category to which the content of the slots belongs;

[0026] Based on the indicators to be labeled, obtain all complete slot information contained in the text to be labeled.

[0027] According to another aspect of the embodiments of this application, an apparatus for obtaining slot labeling data is provided, comprising:

[0028] The acquisition module is configured to acquire all complete slot information contained in the text to be labeled. The complete slot information includes the slot, the slot content carried in the slot, and the slot category label. The slot category label is used to represent the category to which the slot content belongs.

[0029] The determination module is configured to determine the boundary words of the complete slot information based on the slot category label;

[0030] The determining module is configured to determine the quantity distribution value and probability distribution value of the boundary words to the slot in each complete slot information;

[0031] The selection module is configured to select the slot content corresponding to the complete slot information whose value is greater than a preset threshold from the probability distribution values ​​as the annotation data of the text to be annotated.

[0032] According to another aspect of the embodiments of this application, an electronic device is provided, comprising:

[0033] Memory, used to store executable instructions; and

[0034] A display for executing the executable instructions with the memory to perform any of the methods described above for obtaining slot label data.

[0035] According to another aspect of the embodiments of this application, a computer-readable storage medium is provided for storing computer-readable instructions, which, when executed, perform the operation of any of the methods described above for obtaining slot labeling data.

[0036] This application can achieve at least the following beneficial effects:

[0037] This application applies to the field of natural language processing using deep learning technology. It can acquire all complete slot information contained in the text to be annotated. Complete slot information includes slots, slot content, and slot category labels, with the category labels representing the category to which the slot content belongs. Based on the slot category labels, it determines the boundary words of the complete slot information; determines the quantity distribution and probability distribution values ​​of boundary words to slots in each complete slot information; and selects the slot content corresponding to the complete slot information whose probability distribution value is greater than a preset threshold as the annotation data for the text to be annotated. By applying the technical solution of this application, a unified boundary determination can be given to the words to be annotated, thereby ensuring that the probability distribution value corresponding to each slot is determined based on its boundary. Simultaneously, useless symbols and words in the document are discarded, thus ensuring a smaller computational load and reducing the load on computing devices. This avoids the problem of low annotation efficiency caused by inconsistent boundary and category determination of the data to be annotated in the document, which occurs in related technologies.

[0038] The technical solution of this application will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0039] The accompanying drawings, which form part of this specification, illustrate embodiments of this application and, together with the description, serve to explain the principles of this application.

[0040] This application can be more clearly understood with reference to the accompanying drawings and the following detailed description, wherein:

[0041] Figure 1 This is a schematic diagram of a method for obtaining slot labeling data according to an embodiment of this application;

[0042] Figure 2 This is a schematic diagram of the electronic device for acquiring slot labeling data according to an embodiment of this application;

[0043] Figure 3 This is a schematic diagram of the structure of the electronic device for obtaining slot labeling data proposed in an embodiment of this application. Detailed Implementation

[0044] Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present application.

[0045] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0046] The following description of at least one exemplary embodiment is merely illustrative and is not intended to limit the scope of this application or its application or use.

[0047] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0048] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0049] Furthermore, the technical solutions of the various embodiments of this application can be combined with each other, but only if they are based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by this application.

[0050] It should be noted that all directional indicators (such as up, down, left, right, front, back, etc.) in the embodiments of this application are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indicator will also change accordingly.

[0051] The following is combined Figure 1 This application describes a method for acquiring slot labeling data according to exemplary embodiments thereof. It should be noted that the following application scenarios are shown only to facilitate understanding of the spirit and principles of this application, and the embodiments of this application are not limited in any way. Rather, the embodiments of this application can be applied to any applicable scenario.

[0052] This application also proposes a method, apparatus, electronic device, and medium for obtaining slot labeling data.

[0053] Figure 1 A schematic flowchart illustrating a method for obtaining slot labeling data according to an embodiment of this application is shown. Figure 1 As shown, the method includes:

[0054] S101, obtain all complete slot information contained in the text to be annotated. The complete slot information includes the slot, the slot content carried by the slot, and the slot category label. The slot category label is used to represent the category to which the slot content belongs.

[0055] In related technologies, slot recognition (including entity recognition, keyword recognition, etc.) requires manually labeled data for training, evaluation, and testing. Reliable and robust recognition models often require a large amount of data, especially for tasks with many slot categories. Manual labeling often suffers from inconsistencies.

[0056] Specifically, in slot identification, the main problems are inconsistencies in boundary determination and category determination. These issues can impair data quality and reduce the effectiveness of annotation. Therefore, designing a method that can efficiently identify labeled data in documents has become a problem that needs to be solved.

[0057] In the field of automatically evaluating slot labeling quality, two common methods exist: language model scoring and task model (labeling model) scoring. The former uses labeled data to train a language model and then scores each sample in the corpus; the latter trains a model according to a predetermined task (such as an entity recognition model), then predicts a sequence for each sample, while recording the path score. That is, it uses the entire dataset for statistical analysis (machine learning) to obtain a score for each sample, and then sorts the samples by the highest / lowest scores.

[0058] However, such methods often involve high computational costs, rely heavily on the developer's computing resources, and have low real-time performance; moreover, the semantic representation is rather vague and unintuitive. For example, incorrectly labeled samples and samples with rare content in the text will score low in language models; and it is difficult to compare the path probability values ​​of different samples in sequence labeling tasks.

[0059] To address the aforementioned issues, this application proposes a method for obtaining slot labeling data. This method first requires decomposing each field in the text to be labeled in order to obtain all the complete slot information contained therein.

[0060] In one approach, the task of slot labeling can be broken down into: determining the interval of a slot substring and determining the category of the slot. Therefore, labeling quality assessment can also focus on the corresponding features, namely the slot boundaries and the slot category.

[0061] Specifically, taking the content of the document to be annotated as "XX Company is located in Chaoyang District, Beijing" as an example, the category of the data to be annotated corresponds to the address category, and the number of characters in the slot content must be at least three.

[0062] In this embodiment of the application, the aforementioned document to be annotated can be parsed into the following format: <org> XX Company< / org> lie in <loc> Beijing< / loc> <loc> Chaoyang District< / loc> .

[0063] The parsed content includes three complete slot information, specifically the complete slot information corresponding to the first slot. <org> XX Company< / org>The complete slot information corresponding to the second slot is: <loc> Beijing< / loc> And the complete slot information corresponding to the third slot is <loc> Chaoyang District< / loc> .

[0064] As an example, "XX Company", "Beijing", and "Chaoyang District" are slot contents, and "org" and "loc" are slot category labels used to indicate that the slot contents are address types.

[0065] It should be noted here that "org" and "loc" are merely examples and can be other fields, as long as they are assigned a meaning that can be used to characterize the slot content as an address type. Additionally, in this embodiment, " / " can be used to distinguish the left and right sides of the slot category label. That is, <org> This can be a left-side category label for Company XX.< / org> This can be the right category label for Company XX. <loc> It can be a left-side category label for Beijing.< / loc> This can be a right-side category label for Beijing. Similarly, <loc> You can assign a left-side category label to Chaoyang District.< / loc> It can be a right-side category label for Chaoyang District.

[0066] In summary, the first complete slot information in the aforementioned document to be annotated is: <org> XX Company< / org> The second complete slot information is: <loc> Beijing< / loc> The third complete slot information is: <loc> Chaoyang District< / loc> "

[0067] It should be noted that the above examples in this application are used to represent the data to be labeled for obtaining address categories. In specific implementations, the data to be labeled can be obtained based on the labeling indicators corresponding to this task [the labeling indicators include the number of slots (e.g., 3 slots need to be labeled), the number of characters in the slot content (e.g., 3 characters of slot content need to be labeled), and the category to which the slot content belongs (e.g., labeling data of address type, labeling data of person name type, labeling data of company name type, etc.)]. All complete slot information contained in different texts to be labeled can be obtained.

[0068] S102, Based on the slot category label, determine the boundary words of complete slot information.

[0069] Using the example of the document to be annotated, which states "Company XX is located in Chaoyang District, Beijing", we will illustrate this further.

[0070] In this embodiment of the application, the aforementioned document to be annotated can be parsed into the following format: <org> XX Company< / org> lie in <loc> Beijing< / loc> <loc> Chaoyang District< / loc> .

[0071] The parsed content includes three complete slot information, specifically the complete slot information corresponding to the first slot. <org> XX Company< / org>The complete slot information corresponding to the second slot is: <loc> Beijing< / loc> And the complete slot information corresponding to the third slot is <loc> Chaoyang District< / loc> .

[0072] As an example, , and "" are the right labels for the corresponding slot content. <loc>"and" <org> "This is the left label for the content in the corresponding slot." "Located in" indicates the first one.< / org> The right boundary of "" (because "located in" is the first field to the right of the right category label of the first complete slot information), "Chaoyang District" is the first "< / loc> The right boundary of "" (because "Chaoyang District" is the first field to the right of the right category label of the second complete slot information), "Beijing" is the second " <loc>The left boundary of "Beijing" (because "Beijing" is the first field to the left of the left category label of the third complete slot information).

[0073] This field may or may not be slot content. It can be a single character, a phrase, a short sentence, or a symbol, etc. This application does not impose any limitations on this.

[0074] S103, determine the quantity distribution value and probability distribution value of boundary words to slots in each complete slot information.

[0075] Furthermore, in the embodiments of this application, the structure can be based on... <string,<string,int> The nested mapping counter records the distribution of the number of boundary words to slots in each complete slot information.

[0076] Additionally, in this application embodiment, the structure can also be based on... <string,<string,float> The probability distribution value nested mapper records the probability distribution value from boundary words to slots in each complete slot information.

[0077] Specifically, determining the quantity distribution and probability distribution of boundary words to slots in each complete slot information can include the following steps:

[0078] Step 1: Use the parser to output the tag sequence for all fields in the text to be annotated, obtaining the corresponding complete slot information. Then, find the left boundary word and the right boundary word in each complete slot information.

[0079] Furthermore, it is necessary to calculate the first quantity C1 between the left boundary word and the right category label in each complete slot information (that is, the first quantity C1 is the number of slot fields contained between them). Correspondingly, it is necessary to calculate the second quantity C2 between the right boundary word and the left category label in each complete slot information (that is, the second quantity C2 is the number of slot fields contained between them).

[0080] The slot field can be the slot content and the slot category label.

[0081] For example, still using <org> XX Company< / org> lie in <loc> Beijing< / loc> <loc> Chaoyang District< / loc> Please provide an explanation.

[0082] The first complete slot information is: " <org> XX Company< / org> The second complete slot information is: <loc> Beijing< / loc> The third complete slot information is: <loc> Chaoyang District< / loc> "To explain."

[0083] For example, for the second complete slot information, i.e. <loc> Beijing< / loc> For example, its left boundary word is "located in", and its right boundary word is "Chaoyang District". Its left category label is... <loc> The right category label is< / loc> .

[0084] Therefore, it is understandable that for this complete slot information, the first quantity C1 between its left boundary word and its right category label is 3 (i.e., it includes 3 fields, namely " <loc> "Beijing"< / loc> ").as well as,

[0085] Additionally, for this complete slot information, the second quantity C2 between its right boundary word and its left category label is 4 (i.e., it includes 4 fields, namely "Chaoyang District", "...", "..."). <loc> "Beijing"< / loc> ”).

[0086] Step 2: Based on obtaining all words in C1 and C2, this embodiment of the application, following a method similar to Step 1, iterates through each slot field to obtain cases where the left or right side is not a slot (i.e., the first detection result and the second detection result), and these cases are also included in C1 and C2. Then, normalization is performed to obtain their respective logarithmic probability distribution values ​​P1 and P2.

[0087] Step 3: Use P1 and P2 to score all complete slot information contained in the text to be labeled. That is, obtain the combination of left label and left boundary or right label and right boundary as in Step 1, and find the corresponding probability value from P1 or P2, adding it to the total score of the sample. In one approach, the total score can be normalized using the number of sample words or the total number of boundaries.

[0088] Step 4: Sort the scores obtained in Step 3 in ascending order. The first part is the most likely to have inconsistent annotations, which means it is the part that needs to be manually checked or discarded (i.e., the annotation data of the text to be annotated).

[0089] S104. Select the slot content corresponding to the complete slot information whose value is greater than the preset threshold from the probability distribution value as the annotation data of the text to be annotated.

[0090] Understandably, after determining the probability distribution value from boundary words to slots in each complete slot information, the complete slot information with the highest probability distribution value can be selected based on the probability distribution value corresponding to each complete slot information. The slot content corresponding to this complete slot information with the highest probability distribution value is then considered the most likely to have inconsistent labeling, i.e., the labeling data that most needs manual inspection or should be discarded.

[0091] In this application, all complete slot information contained in the text to be annotated can be obtained. Complete slot information includes the slot, the slot content it contains, and the slot category label, where the slot category label represents the category to which the slot content belongs. Based on the slot category label, boundary words of the complete slot information are determined. The quantity distribution and probability distribution values ​​of boundary words to slots in each complete slot information are determined. The slot content corresponding to the complete slot information whose probability distribution value is greater than a preset threshold is selected as the annotation data for the text to be annotated. By applying the technical solution of this application, a unified boundary determination can be given to the words to be annotated, thereby ensuring that the probability distribution value corresponding to each slot is determined according to the boundary of each slot. At the same time, useless symbols and words in the document are discarded, thus ensuring a small computational load and reducing the load on computing devices. This avoids the problem of low annotation efficiency caused by inconsistent boundary and category determination of the data to be annotated in the document, which occurs in related technologies.

[0092] Optionally, in another embodiment based on the method described above in this application, determining the boundary words of the complete slot information based on the slot category label includes:

[0093] The category label located to the left of the slot category label is determined as the left category label of the complete slot information; and the category label located to the right of the slot category label is determined as the right category label of the complete slot information.

[0094] Based on the left category label and the right category label of the complete slot information, the boundary words of the complete slot information are determined, and the boundary words include left boundary words and right boundary words.

[0095] Optionally, in another embodiment based on the method described above in this application, determining the boundary words of the complete slot information based on the left category label and the right category label of the complete slot information includes:

[0096] The first field to the left of the left category label in the complete slot information is taken as the left boundary word of the complete slot information; and...

[0097] The first field to the right of the right category label in the complete slot information is taken as the right boundary word of the complete slot information.

[0098] Optionally, in another embodiment based on the method described above in this application, determining the distribution value of the number of boundary words to the slot in each complete slot information includes:

[0099] Determine a first number of slot fields contained between the left boundary word and the right category label of each complete slot information; and determine a second number of slot fields contained between the right boundary word and the left category label of each complete slot information;

[0100] The first quantity and the second quantity are used as the corresponding quantity distribution values ​​in the complete slot information.

[0101] Optionally, in another embodiment based on the method described above in this application, the step of using the first quantity and the second quantity as the corresponding quantity distribution values ​​in the complete slot information includes:

[0102] Detect whether the adjacent side of each slot content in the first number of slot fields is a slot, and obtain a first detection result; and detect whether the adjacent side of each slot content in the second number of slot fields is a slot, and obtain a second detection result.

[0103] The first quantity and the first detection result, and the second quantity and the second detection result are used as the corresponding quantity distribution values ​​in the complete slot information.

[0104] As one possible implementation method, the following will still refer to... <org> XX Company< / org> lie in <loc> Beijing< / loc> <loc> Chaoyang District< / loc> Please provide an explanation.

[0105] The first complete slot information is: " <org> XX Company< / org> The second complete slot information is: <loc> Beijing< / loc> The third complete slot information is: <loc> Chaoyang District< / loc> "To explain."

[0106] For example, for the second complete slot information, i.e. <loc> Beijing< / loc> For example, its left boundary word is "located in", and its right boundary word is "Chaoyang District". Its left category label is... <loc> The right category label is< / loc> .

[0107] Therefore, it is understandable that for this complete slot information, the first quantity C1 between its left boundary word and its right category label is 3 (i.e., it includes 3 fields, namely " <loc> "Beijing"< / loc> ").as well as,

[0108] Additionally, for this complete slot information, the second quantity C2 between its right boundary word and its left category label is 4 (i.e., it includes 4 fields, namely "Chaoyang District", "...", "..."). <loc> "Beijing"< / loc> ”).

[0109] Furthermore, in the embodiments of this application, based on obtaining all words in C1 and C2, following a method similar to step 1, the cases where the left or right side of each slot field is not a slot (i.e., the first detection result and the second detection result) are also included in C1 and C2. Then, the corresponding quantity distribution value can be obtained by summing.

[0110] Optionally, in another embodiment based on the method described above in this application, determining the probability distribution value from the boundary word to the slot in each complete slot information includes:

[0111] The quantity distribution value is normalized to obtain the probability distribution value corresponding to the complete slot information.

[0112] Optionally, in another embodiment based on the method described above in this application, retrieving all complete slot information contained in the text to be annotated includes:

[0113] Obtain the metrics to be labeled, which include the number of slots, the number of words in the content of the slots, and the category to which the content of the slots belongs;

[0114] Based on the indicators to be labeled, obtain all complete slot information contained in the text to be labeled.

[0115] It should be noted that, in the process of determining the boundary words for each complete slot information in this embodiment, the left category label can be sequentially searched in the sequence output by the parser after processing, and the nearest non-label unit to its left is a left boundary word; the search for right boundary words is similar. In one approach, since the parser in this embodiment adds start and end markers, the problem of document out-of-bounds annotation will not occur.

[0116] In one embodiment, the parser proposed in this application can add functions such as word segmentation, adding beginning and end tags, and word replacement. For example, it can output "XX Company is located in Chaoyang District, Beijing" in the format shown in the table below:

[0117]

[0118] Optionally, the parser can replace "XX Company" with "Company", "Chaoyang District" with "District", etc., thereby improving generalization performance.

[0119] Furthermore, in the annotation of address lookup slots, inconsistencies often arise in the determination of address slot boundaries. This is due to the inherent attributes of address slots themselves (for example, for "No. 1, Lizheyuan, Chaoyang District, Beijing", at least "... <loc> Beijing< / loc> <loc> Chaoyang District< / loc> <loc> Lizheyuan No. 1< / loc> "and" <loc> No. 1, Lizheyuan, Chaoyang District, Beijing< / loc> (Two annotation methods).

[0120] Similarly, the identification of names, positions, and titles (" <per> open< / per> <title> teacher< / title> "still" <per> Teacher Zhang< / per> ") is also a high-incidence area for inconsistencies. In such areas, manual annotation has a high inconsistency rate, requiring frequent checks by annotators. Automated tools can improve efficiency, but the tools themselves cannot require excessive computing resources, otherwise the annotators will find them difficult to use. This method only stores and calculates for words that act as boundaries, and using boundary information as features is reasonable from an annotation perspective. Therefore, the lightweight and fast evaluation tool described in this solution meets practical needs.

[0121] By applying the technical solution of this application, a unified boundary determination can be assigned to the words to be labeled, thereby ensuring that the probability distribution value corresponding to each slot is determined based on its boundary. Simultaneously, useless symbols and words in the document are discarded, thus ensuring a smaller computational load and reducing the burden on computing devices. This avoids the problem of low labeling efficiency caused by inconsistent boundary and category determinations of the data to be labeled in the document, which occurs in related technologies.

[0122] Optionally, in another embodiment of this application, such as Figure 2 As shown, this application also provides an apparatus for acquiring slot labeling data. It includes:

[0123] The acquisition module 201 is configured to acquire all complete slot information contained in the text to be labeled. The complete slot information includes the slot, the slot content carried in the slot, and the slot category label. The slot category label is used to represent the category to which the slot content belongs.

[0124] The determining module 202 is configured to determine the boundary words of the complete slot information based on the slot category label;

[0125] The determining module 202 is configured to determine the quantity distribution value and probability distribution value of the boundary words to the slot in each complete slot information;

[0126] The selection module 203 is configured to select the slot content corresponding to the complete slot information whose value is greater than a preset threshold from the probability distribution values ​​as the annotation data of the text to be annotated.

[0127] In this application, all complete slot information contained in the text to be annotated can be obtained. Complete slot information includes the slot, the slot content it contains, and the slot category label, where the slot category label represents the category to which the slot content belongs. Based on the slot category label, boundary words of the complete slot information are determined. The quantity distribution and probability distribution values ​​of boundary words to slots in each complete slot information are determined. The slot content corresponding to the complete slot information whose probability distribution value is greater than a preset threshold is selected as the annotation data for the text to be annotated. By applying the technical solution of this application, a unified boundary determination can be given to the words to be annotated, thereby ensuring that the probability distribution value corresponding to each slot is determined according to the boundary of each slot. At the same time, useless symbols and words in the document are discarded, thus ensuring a small computational load and reducing the load on computing devices. This avoids the problem of low annotation efficiency caused by inconsistent boundary and category determination of the data to be annotated in the document, which occurs in related technologies.

[0128] In another embodiment of this application, the acquisition module 201 is configured to:

[0129] The category label located to the left of the slot category label is determined as the left category label of the complete slot information; and the category label located to the right of the slot category label is determined as the right category label of the complete slot information.

[0130] Based on the left category label and the right category label of the complete slot information, the boundary words of the complete slot information are determined, and the boundary words include left boundary words and right boundary words.

[0131] In another embodiment of this application, the determining module 202 is configured to:

[0132] The first field to the left of the left category label in the complete slot information is taken as the left boundary word of the complete slot information; and...

[0133] The first field to the right of the right category label in the complete slot information is taken as the right boundary word of the complete slot information.

[0134] In another embodiment of this application, the determining module 202 is configured to:

[0135] Determine a first number of slot fields contained between the left boundary word and the right category label of each complete slot information; and determine a second number of slot fields contained between the right boundary word and the left category label of each complete slot information;

[0136] The first quantity and the second quantity are used as the corresponding quantity distribution values ​​in the complete slot information.

[0137] In another embodiment of this application, the determining module 202 is configured to:

[0138] Detect whether the adjacent side of each slot content in the first number of slot fields is a slot, and obtain a first detection result; and detect whether the adjacent side of each slot content in the second number of slot fields is a slot, and obtain a second detection result.

[0139] The first quantity and the first detection result, and the second quantity and the second detection result are used as the corresponding quantity distribution values ​​in the complete slot information.

[0140] In another embodiment of this application, the determining module 202 is configured to:

[0141] The quantity distribution value is normalized to obtain the probability distribution value corresponding to the complete slot information.

[0142] In another embodiment of this application, the acquisition module 201 is configured to:

[0143] Obtain the metrics to be labeled, which include the number of slots, the number of words in the content of the slots, and the category to which the content of the slots belongs;

[0144] Based on the indicators to be labeled, obtain all complete slot information contained in the text to be labeled.

[0145] Figure 3 This is a logical structure block diagram of an electronic device according to an exemplary embodiment. For example, electronic device 300 may be an electronic device that includes an interference transmitter and an interference canceller.

[0146] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory including instructions. These instructions can be executed by a processor of an electronic device to complete the method for obtaining slot annotation data. The method includes: obtaining all complete slot information contained in the text to be annotated, the complete slot information including slots, slot content carried on the slots, and slot category tags, the slot category tags being used to characterize the category to which the slot content belongs; determining boundary words of the complete slot information based on the slot category tags; determining the quantity distribution value and probability distribution value of the boundary words to the slots in each complete slot information; and selecting the slot content corresponding to the complete slot information whose probability distribution value is greater than a preset threshold as the annotation data of the text to be annotated. Optionally, the instructions can also be executed by a processor of an electronic device to complete other steps involved in the exemplary embodiment. For example, the non-transitory computer-readable storage medium can be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0147] In an exemplary embodiment, an application / computer program product is also provided, including one or more instructions that can be executed by a processor of an electronic device to complete the above-described method for obtaining slot annotation data. The method includes: obtaining all complete slot information contained in the text to be annotated, the complete slot information including slots, slot content carried in the slots, and slot category tags, the slot category tags being used to characterize the category to which the slot content belongs; determining boundary words of the complete slot information based on the slot category tags; determining the quantity distribution value and probability distribution value of the boundary words to the slots in each complete slot information; and selecting the slot content corresponding to the complete slot information whose probability distribution value is greater than a preset threshold as the annotation data of the text to be annotated. Optionally, the above instructions can also be executed by a processor of an electronic device to complete other steps involved in the above exemplary embodiment.

[0148] Figure 3 This is an example diagram of an electronic device 300. Those skilled in the art will understand that it is illustrative. Figure 3 This is merely an example of electronic device 300 and does not constitute a limitation on electronic device 300. It may include more or fewer components than shown, or combine certain components, or different components. For example, electronic device 300 may also include input / output devices, network access devices, buses, etc.

[0149] The processor 302 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor, or processor 302 can be any conventional processor. Processor 302 is the control center of electronic device 300, connecting all parts of electronic device 300 via various interfaces and lines.

[0150] The memory 301 can be used to store computer-readable instructions 303. The processor 302 implements various functions of the electronic device 300 by running or executing the computer-readable instructions or modules stored in the memory 301 and calling the data stored in the memory 301. The memory 301 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device 300, etc. In addition, the memory 301 may include a hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, read-only memory (ROM), random access memory (RAM), or other non-volatile / volatile storage devices.

[0151] If the modules integrated in the electronic device 300 are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by instructing related hardware through computer-readable instructions. The computer-readable instructions can be stored in a computer-readable storage medium, and when executed by a processor, the computer-readable instructions can implement the steps of the various method embodiments described above.

[0152] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0153] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.< / loc>

Claims

1. A method for obtaining slot annotation data, characterized in that, The method comprises the following steps: acquiring all complete slot information contained in the text to be labeled, the complete slot information comprising a slot, slot content carried on the slot, and a slot category label used to represent a category to which the slot content belongs; determining boundary words of the complete slot information based on the slot category label; determining a quantity distribution value and a probability distribution value of the boundary words to the slot in each complete slot information; selecting slot content corresponding to complete slot information with a value greater than a preset threshold in the probability distribution value as labeling data of the text to be labeled; the determination of the quantity distribution value and the probability distribution value of the boundary words to the slot in each complete slot information comprises: determining a first quantity of slot fields contained between a left boundary word and a right category label of each complete slot information, and determining a second quantity of slot fields contained between a right boundary word and a left category label of each complete slot information; taking the first quantity and the second quantity as the corresponding quantity distribution value in the complete slot information; and normalizing the quantity distribution value to obtain the probability distribution value; the right category label is the slot category label located on the right side of the slot, and the left category label is the slot category label located on the left side of the slot.

2. The method of claim 1, wherein, the determination of the boundary words of the complete slot information based on the slot category label comprises: determining a category label located on the left side of the slot in the slot category label as a left category label of the complete slot information, and determining a category label located on the right side of the slot in the slot category label as a right category label of the complete slot information; determining the boundary words of the complete slot information based on the left category label and the right category label of the complete slot information, the boundary words comprising a left boundary word and a right boundary word.

3. The method of claim 2, wherein, the determination of the boundary words of the complete slot information based on the left category label and the right category label of the complete slot information comprises: taking a first field adjacent to the left side of the left category label in the complete slot information as the left boundary word of the complete slot information; and taking a first field adjacent to the right side of the right category label in the complete slot information as the right boundary word of the complete slot information.

4. The method of claim 1, wherein, the taking of the first quantity and the second quantity as the corresponding quantity distribution value in the complete slot information comprises: detecting whether the adjacent side of each slot content in the first quantity of slot fields is a slot to obtain a first detection result, and detecting whether the adjacent side of each slot content in the second quantity of slot fields is a slot to obtain a second detection result; taking the first quantity and the first detection result, and taking the second quantity and the second detection result as the corresponding quantity distribution value in the complete slot information.

5. The method of claim 1, wherein, the acquisition of all complete slot information contained in the text to be labeled comprises: acquiring an index to be labeled this time, the index to be labeled comprising a quantity of slots, a quantity of characters of slot content, and a category to which the slot content belongs; Based on the to-be-labeled index, all complete slot information contained in the to-be-labeled text is acquired.

6. An apparatus for obtaining slot annotation data, the apparatus comprising: The method comprises the steps of: An acquisition module is configured to acquire all complete slot information contained in the to-be-labeled text, wherein the complete slot information comprises a slot, slot content carried on the slot, and a slot category label used to represent a category to which the slot content belongs. A determination module is configured to determine a boundary word of the complete slot information based on the slot category label. The determination module is configured to determine a quantity distribution value and a probability distribution value of the boundary word to the slot in each complete slot information. A selection module is configured to select slot content corresponding to complete slot information with a value greater than a preset threshold in the probability distribution value as labeled data of the to-be-labeled text. The determination module is configured to determine a first number of slot fields contained between a left boundary word and a right category label of each complete slot information, and determine a second number of slot fields contained between a right boundary word and a left category label of each complete slot information; take the first number and the second number as a corresponding quantity distribution value in the complete slot information; and normalize the quantity distribution value to obtain the probability distribution value; the right category label is the slot category label located on the right side of the slot, and the left category label is the slot category label located on the left side of the slot.

7. An electronic device, comprising: The method comprises the steps of: A memory is configured to store executable instructions; and A processor is configured to execute the executable instructions with the memory to complete the operations of the method for acquiring slot labeled data according to any one of claims 1-5.

8. A computer readable storage medium storing computer readable instructions, characterized in that, The instructions are executed to perform the operations of the method for acquiring slot labeled data according to any one of claims 1-5.

Images