Label generation method and device, and electronic device

Abstract

The application relates to the technical field of data processing, and discloses a label generation method, a label generation device and electronic equipment. The label generation method comprises the following steps: labeling new call corpus by using an original label dictionary to obtain a labeling result; in the case that the labeling result represents that the new call corpus is not covered by the original label dictionary, generating a label update suggestion based on unlabeled information and unlabeled corpus in a backup corpus library; updating the original label dictionary according to the label update suggestion to obtain a new version of the label dictionary; and supplementally labeling the new call corpus and the unlabeled corpus which are not covered by the original label dictionary based on the new version of the label dictionary to generate label annotation data. The application can improve the integrity of full-amount corpus annotation and enable a label system to be dynamically updated along with changes in a business scenario.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and more specifically to a tag generation method, apparatus, and electronic device. Background Technology

[0002] The corpus labeling of call tags in related technologies usually adopts a static fixed tag system, which cannot adapt to the changes in tag requirements brought about by changes in business scenarios. Summary of the Invention

[0003] This invention provides a tag generation method, apparatus, and electronic device to solve the problem that the corpus tagging of call tags in related technologies usually adopts a static fixed tag system, which cannot adapt to the changes in tag requirements brought about by changes in business scenarios.

[0004] In a first aspect, the present invention provides a tag generation method, comprising: tagging newly added common speech corpus using an original tag dictionary to obtain tagging results; when the tagging results indicate that the newly added common speech corpus is not covered by the original tag dictionary, generating tag update suggestions based on untagged information and untagged corpus in a backup corpus, wherein the untagged information is used to describe the reasons and corresponding semantic features of the newly added common speech corpus that cannot be fully tagged by the original tag dictionary; updating the original tag dictionary according to the tag update suggestions to obtain a new version tag dictionary; and supplementing the tagging of newly added common speech corpus and untagged corpus not covered by the original tag dictionary based on the new version tag dictionary to generate tag annotation data.

[0005] The tag generation method provided by this invention performs preliminary screening and tagging of newly added communication corpora using an original tag dictionary. This allows for the rapid identification of new communication corpora that cannot be covered by existing tags due to changes in business scenarios. By generating tag update suggestions using untagged information and existing untagged corpora in a backup corpus, the method enables dynamic iterative updates of the tag dictionary. This allows for adaptation to new tag requirements without the need for manual rebuilding of the entire tag system. After generating a new version of the tag dictionary, the method uniformly supplements the tags of the relevant untagged corpora, ensuring the completeness of the full corpus annotation and allowing the tag system to be dynamically updated in response to changes in business scenarios. This improves the flexibility and efficiency of communication corpus tagging and reduces the manpower cost of maintaining the tag system.

[0006] In one optional implementation, the newly added communicative corpus is tagged using the original tag dictionary to obtain a tagging result. This includes: performing tag matching verification processing on the newly added communicative corpus based on the original tag dictionary to obtain a matching verification result. The tag matching verification includes at least one of semantic matching, parameter extraction, value range verification, and conflict detection. If the matching verification result indicates that some or all of the content in the newly added communicative corpus does not match the corresponding tags in the original tag dictionary, then the tagging result is determined to be that the newly added communicative corpus is not covered by the original tag dictionary. If the matching verification result indicates that all of the content in the newly added communicative corpus is matched, then the tagging result is determined to be that the newly added communicative corpus has been covered by the original tag dictionary.

[0007] The tag generation method provided by this invention effectively avoids misjudgment and omission caused by a single matching logic through multi-dimensional tag matching verification such as semantic matching and parameter extraction. This significantly improves the accuracy of identifying uncovered corpora and provides an accurate processing foundation for subsequent iterative updates of the tag dictionary. At the same time, this hierarchical judgment method only filters out uncovered corpora that need to be updated, without requiring full-process processing of all newly added corpora. While ensuring the accuracy of annotation, it effectively simplifies the processing flow, reduces the consumption of computing resources and the overall processing time, and enables the tag system to adapt to business changes more quickly, further improving the efficiency of the entire tag generation and update process.

[0008] In one optional implementation, when the tagging result indicates that the newly added communicative corpus is not covered by the original tag dictionary, a tag update suggestion is generated based on the untagged information and the untagged corpus in the backup corpus. This includes: obtaining untagged information when the tagging result indicates that the newly added communicative corpus is not covered by the original tag dictionary; the untagged information includes at least one of the following: untagged reason, similarity of matched tags, uncovered semantic content, and tag version; the untagged reason includes at least one of the following: full semantic mismatch, partial semantic uncovering, and tag value exceeding the preset tag value range; clustering the corpus fragments corresponding to the untagged information with the untagged corpus based on target triggering conditions to obtain at least one corpus cluster; the target triggering conditions include one of the following: timed triggering, quantity threshold triggering, untagged corpus growth triggering, and user operation command triggering; and generating a tag update suggestion based on the semantic features of each corpus cluster and the tag features of the original tag dictionary. The tag update suggestion includes at least one of the following: adding tags, splitting tags, merging tags, and updating the tag value range.

[0009] The tag generation method provided by this invention classifies and records untagged information, and generates update suggestions by clustering untagged corpora through trigger conditions. It can automatically aggregate newly added semantics that have not been covered, saving manual collection and reducing iteration workload. It also avoids tag redundancy and chaos caused by scattered additions. Multiple trigger conditions can adapt to the needs of different business and update frequencies, balancing timeliness and resource allocation. Multiple types of suggestions match different reasons for untagged information, and can provide adaptation solutions for semantic changes, maintaining the clarity and coverage of the tag dictionary, improving the accuracy of subsequent automatic tagging, and enabling the tag system to flexibly adapt to business changes, thereby improving the system iteration efficiency and adaptability.

[0010] In one optional implementation, based on the semantic features of each corpus cluster and the tag features of the original tag dictionary, a tag update suggestion is generated, including: if the number of valid corpora within the corpus cluster is greater than or equal to a first quantity threshold, the semantic features do not match any tags in the original tag dictionary, and the proportion of the corpus cluster in unlabeled corpora within the statistical period is greater than or equal to a first proportion, then a tag update suggestion for adding a new tag is generated; if there is any original tag in the original tag dictionary that matches the semantic features, but the semantic features have a feature similarity greater than or equal to a first... If a similarity threshold is set, a tag update suggestion for splitting the tags is generated. If there are at least two original tags in the original tag dictionary that match the semantic features, and the overlap of the two original tags in the corpus within the statistical period is greater than or equal to the second similarity threshold, a tag update suggestion for merging the tags is generated. If there is any original tag in the original tag dictionary that matches the semantic features of the corpus cluster, but the tag value range of the semantic features exceeds the original value range of the original tags, and the proportion of the corpus in the super-value range in the unlabeled corpus within the statistical period is greater than or equal to the second proportion, a tag update suggestion for updating the tag value range is generated.

[0011] The label generation method provided by this invention generates multiple label update suggestions based on rules, which can accurately identify the optimization needs of the label system, eliminate manual sorting and analysis, reduce maintenance costs, and avoid the decline in the applicability of the label system caused by manual operation. By adding, splitting, merging labels and adjusting value ranges, the label system can continuously adapt to business changes, maintain high coverage and accuracy, provide a reliable foundation for subsequent tasks, and improve the accuracy and efficiency of labeling.

[0012] In one optional implementation, updating the original tag dictionary according to the tag update suggestion to obtain a new version of the tag dictionary includes: detecting interactive operations in response to the tag update suggestion; and updating the original tag dictionary based on the interactive operations to obtain a new version of the tag dictionary.

[0013] The tag generation method provided by this invention combines tag optimization recognition with user demand judgment by detecting interactive operations in response to tag update suggestions. This not only improves the efficiency of tag sorting and reduces the labor cost of manually reviewing the entire tag system, but also allows for flexible control of the tag update process based on the specific needs of actual business operations. This ensures that the updated tag system meets the actual needs of the current business, further improving the applicability and accuracy of the updated tag system, and thus better adapting to the tag system maintenance needs of different business scenarios.

[0014] In one optional implementation, updating the original tag dictionary based on an interactive operation to obtain a new version of the tag dictionary includes: if the interactive operation indicates that the tag update suggestion of adding new tags is accepted, then the original tag dictionary is updated based on the tag information of the new tags in the tag update suggestion to obtain a new version of the tag dictionary. The tag information includes at least one of tag name, tag definition, tag type, value range, description subject, whether multiple tags are selected, and whether tags are required; if the interactive operation indicates that the tag update suggestion of splitting tags is accepted, then the original tag dictionary is updated based on the original tag identifier and the tag information of each new tag after splitting to obtain a new version of the tag dictionary; if the interactive operation indicates that the tag update suggestion of merging tags is accepted, then the original tag dictionary is updated based on the original tag identifier to be merged and the merged tag information to obtain a new version of the tag dictionary; if the interactive operation indicates that the tag update suggestion of updating the tag value range is accepted, then the original tag dictionary is updated based on the tag identifier to be updated and the latest suggested value range to obtain a new version of the tag dictionary.

[0015] The tag generation method provided by this invention, by combining user interaction operations to process different types of tag update suggestions accordingly, can achieve dynamic and flexible updates of the tag dictionary. It can adapt to the iterative needs of the tag system under different business scenarios, effectively improving the efficiency and accuracy of tag dictionary updates, while reducing the time and manpower costs of tag system maintenance. Through the update methods of adding tags, splitting tags, merging tags, and updating tag value domains, it can ensure that the updated tag dictionary meets the actual usage needs of business personnel, further improving the usability and practicality of the tag system.

[0016] In one optional implementation, new common speech corpora and untagged corpora not covered by the original tag dictionary are supplemented with tags based on the new version of the tag dictionary to generate tag annotation data. This includes: filtering the historical untagged corpora in the candidate corpus by version and extracting untagged corpora whose tag version is identified as the original tag dictionary version or an earlier version to obtain corpora to be supplemented with tags; constructing a set of corpora to be supplemented with tags based on the corpora to be supplemented with tags and the new common speech corpora not covered by the original tag dictionary; and calling the new version of the tag dictionary to perform tag matching and annotation on each corpus in the set of corpora to be supplemented with tags to obtain tag annotation data.

[0017] The tag generation method provided by this invention filters and re-labels historical unlabeled corpora according to tag versions. This not only avoids missing unlabelable corpora under the old tag system, but also ensures that the corpus to be re-labeled covers both newly added business communication corpora and historically unlabeled corpora. It achieves synchronous completion of labeled corpora after the tag dictionary is updated, ensures the consistency between the tag labeling data and the new version of the tag dictionary, avoids mismatch problems caused by asynchronous labeling data after the tag system is updated, and also reduces the waste of resources caused by unlabeled corpora.

[0018] In an optional implementation, the method further includes: when the labeling result indicates that the newly added phonics data has been covered by the original label dictionary, generating label annotation data by taking the matching label, confidence level and version identifier of the original label dictionary corresponding to the labeling result.

[0019] The tag generation method provided by this invention handles different cases depending on whether the newly added discourse corpus is covered by the original tag dictionary. For corpora that can be matched by the original tag dictionary, the existing tagging results are directly reused to generate tag annotation data, without needing to repeat the tagging process using a new version of the tag dictionary. This approach avoids unnecessary repetitive calculations, effectively reducing the computational consumption of the annotation process and improving the overall processing efficiency of tag annotation. On the other hand, it can completely preserve the mature and accurate annotation results under the original tag system, ensuring the accuracy of the annotation data.

[0020] Secondly, the present invention provides a label generating apparatus, comprising: The corpus tagging module is used to tag newly added linguistic corpus using the original tag dictionary to obtain tagging results; The suggestion generation module is used to generate label update suggestions based on unlabeled information and unlabeled corpus in the backup corpus when the labeling result indicates that the newly added corpus is not covered by the original label dictionary. The unlabeled information is used to describe the reasons and corresponding semantic features of the newly added corpus that cannot be fully labeled by the original label dictionary. The dictionary update module is used to update the original tag dictionary based on tag update suggestions to obtain a new version of the tag dictionary; The tag generation module is used to add tags to newly added corpora and untagged corpora that are not covered by the original tag dictionary based on the new version of the tag dictionary, and generate tag annotation data.

[0021] Thirdly, the present invention provides an electronic device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the tag generation method of the first aspect or any corresponding embodiment described above.

[0022] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to perform the tag generation method of the first aspect or any corresponding embodiment thereof.

[0023] Fifthly, the present invention provides a computer program product, including computer instructions for causing a computer to execute the tag generation method of the first aspect or any corresponding embodiment thereof. Attached Figure Description

[0024] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0025] Figure 1 This is a schematic diagram of the first process of a label generation method according to an embodiment of the present invention; Figure 2 This is a schematic diagram of a second process for a label generation method according to an embodiment of the present invention; Figure 3 This is a data flow diagram of the label generation method according to an embodiment of the present invention; Figure 4 This is a structural block diagram of a label generation apparatus according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] It is understood that before using the technical solutions disclosed in the various embodiments of the present invention, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in the present invention and their authorization should be obtained in accordance with relevant laws and regulations through appropriate means.

[0028] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0029] According to an embodiment of the present invention, a label generation method embodiment is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0030] This embodiment provides a label generation method. Figure 1 This is a flowchart of a label generation method according to an embodiment of the present invention, such as... Figure 1 As shown, the process includes the following steps: Step S101: Tag the newly added utterance data using the original tag dictionary to obtain the tagging results.

[0031] The original tag dictionary refers to the effective tag dictionary used by the current application mode before the tag update operation is triggered.

[0032] In some optional implementations, when constructing the original tag dictionary, historical communication data within the target time period can be obtained; tag-related feature information in the historical communication data can be extracted, and the extracted tag-related feature information can be deduplicated, normalized, and organized. All the organized tag-related feature information can be integrated according to the preset tag hierarchy structure, and finally an original tag dictionary that can be used to tag new communication data can be generated, including tag identifier, tag name, tag definition, tag type, value range, description subject, whether multiple tags are selected, and whether tags are required.

[0033] Specifically, the range of the target time period can be flexibly set according to the update frequency of the actual business. For example, for call scenarios for vehicle services, the full historical call data within the corresponding period can be obtained according to the natural week or natural month. Alternatively, it can be combined with the iteration node of the business function to obtain all historical call data newly generated after the last tag update, ensuring that the original tag dictionary can cover the mainstream tag requirements under the current business.

[0034] Furthermore, during the process of organizing tag-related feature information, for tags with similar semantics but different expressions, the main tag can be retained based on the actual usage frequency in the business scenario, and the remaining synonymous tags can be mapped to the main tag as aliases. This avoids the problem of inconsistent tagging results caused by the dispersion of synonymous tags, and also accommodates the expression habits of different business personnel, improving the adaptability of tagging. The generated original tag dictionary can be simultaneously stored in the application cache and the update cache. The application cache is used to support the real-time tagging of newly added communication data, while the update cache is specifically used for comparison and verification of subsequent tag update operations. This avoids the tag update process from interfering with the daily tagging business, ensuring the continuity and stability of system operation.

[0035] In some optional implementations, when tagging the new communicative corpus using the original tag dictionary to obtain the tagging results, the new communicative corpus can be subjected to tag matching verification processing based on the original tag dictionary to obtain the matching verification results. The tag matching verification includes at least one of semantic matching, parameter extraction, value range verification, and conflict detection. If the matching verification results indicate that some or all of the content in the new communicative corpus does not match the corresponding tags in the original tag dictionary, then the tagging result is determined to be that the new communicative corpus is not covered by the original tag dictionary. If the matching verification results indicate that all of the content in the new communicative corpus is matched, then the tagging result is determined to be that the new communicative corpus has been covered by the original tag dictionary.

[0036] Specifically, semantic matching involves extracting semantic features from the preprocessed new communication corpus, then calculating the similarity between these features and the preset semantic features corresponding to each tag in the original tag dictionary. If the calculated maximum similarity is higher than a preset matching threshold, the corresponding tag is considered successfully matched. Parameter extraction targets tags with business parameters, locating and extracting the business parameters that meet the tag's requirements from the text content of the new communication corpus. If a parameter conforming to the specified format cannot be extracted, the match is considered unsuccessful. Value range verification verifies the range of values ​​for the extracted business parameters, confirming that the parameter values ​​fall within the preset legal value range of the corresponding tag. If the value exceeds the range, the match is considered invalid. Conflict detection checks whether multiple tags matched from the same new communication corpus have a preset mutually exclusive relationship.

[0037] As an example, if the newly added call data is pre-sales call text for vehicles, and the original tag dictionary has already included tags corresponding to vehicle model inquiries and trade-in policies in the pre-sales consultation scenario, the similarity obtained by semantic matching is higher than the preset threshold, parameter extraction and value range verification are both passed, and there are no mutually exclusive tag conflicts, it can be determined that the newly added call data has been covered by the original tag dictionary, and the corresponding tagging results can be directly output; if the newly added call data mentions user inquiries about the newly launched new energy vehicle rural subsidy, but the business has not been launched when the original tag dictionary was built and the corresponding tags have not been included, and the maximum similarity obtained by semantic matching is also lower than the preset matching threshold, it can be determined that the newly added call data contains content that has not been covered by the original tag dictionary.

[0038] As mentioned above, by using multi-dimensional tag matching verification such as semantic matching and parameter extraction, the problems of misjudgment and omission caused by single matching logic can be effectively avoided, which greatly improves the accuracy of identifying uncovered corpora and provides an accurate processing foundation for subsequent iterative updates of the tag dictionary. At the same time, this hierarchical judgment method only selects uncovered corpora that need to be updated, without having to process all newly added corpora in the entire process. While ensuring the accuracy of annotation, it effectively simplifies the processing flow, reduces the consumption of computing resources and the overall processing time, and enables the tag system to adapt to business changes more quickly, further improving the efficiency of the entire tag generation and update process.

[0039] Step S102: If the tagging result indicates that the newly added discourse data is not covered by the original tag dictionary, a tag update suggestion is generated based on the untagged information and the untagged data in the backup corpus.

[0040] Among them, the unlabeled information is used to describe the reasons and corresponding semantic features of the newly added discourse corpus that cannot be fully labeled by the original label dictionary.

[0041] In some optional implementations, when the tagging result indicates that the newly added communicative corpus is not covered by the original tag dictionary, when generating tag update suggestions based on untagged information and untagged corpus in the backup corpus, the untagged information is obtained. The untagged information includes one of the following: untagged reason, similarity of matched tags, uncovered semantic content, and tag version. The untagged reason includes at least one of the following: full semantic mismatch, partial semantic uncovering, and tag value exceeding the preset tag value range. Based on the target triggering condition, the corpus fragments corresponding to the untagged information are clustered with the untagged corpus to obtain at least one corpus cluster. The target triggering condition includes one of the following: timed triggering, quantity threshold triggering, untagged corpus growth rate triggering, and user operation command triggering. Based on the semantic features of each corpus cluster and the tag features of the original tag dictionary, tag update suggestions are generated. The tag update suggestions include at least one of the following: adding tags, splitting tags, merging tags, and updating tag value range.

[0042] Specifically, for each cluster of corpus obtained from clustering, a cluster semantic vector is extracted. The similarity between this cluster semantic vector and the semantic vectors of all tags in the original tag dictionary is calculated. If the highest calculated similarity is lower than a preset addition threshold, and the number of corpus fragments contained in the corpus cluster reaches a preset addition level, then a tag name, preliminary value range, and tag definition are generated based on the core semantics of the corpus cluster. These contents, along with the corresponding corpus examples, are then included as new tags in the tag update suggestion. If the cluster semantic vector of the corpus cluster has a similarity higher than a preset splitting threshold, and no... If the covered content belongs to an independent sub-business scenario under the existing tag, a tag update suggestion is generated to split the existing tag and add corresponding sub-tags; if it is detected that the semantic vectors of multiple different tags in the original tag dictionary all meet the preset merging conditions with the cluster semantic vectors of the same corpus cluster, and the business definitions of these tags themselves have ambiguous boundaries and often cause confusion during tagging, a tag update suggestion is generated to merge multiple tags into a new tag; if the reason for not tagging is that the parameter value exceeds the preset value range of the original tag, and the new value meets the requirements of the business rules, a tag update suggestion is generated to expand the value range of the tag.

[0043] As an example, when multiple newly added communication corpora reach a preset threshold in number, and all mention inquiries related to subsidies for new energy vehicles in rural areas, after clustering to obtain corresponding corpus clusters, if the highest similarity between the cluster semantic vector of this corpus cluster and the semantic vectors of all existing tags in the original tag dictionary is still lower than the preset addition threshold, and the number of corpora meets the addition requirements, then a tag update suggestion for adding a new tag for new energy vehicle subsidies in rural areas is generated, along with the core semantic description of the tag, corresponding corpus examples, and a preset value range suggestion. If the original tag dictionary already contains a tag for new energy subsidies, and the semantic similarity between the unlabeled corpus cluster obtained by clustering and the tag is higher than the preset splitting threshold, and the rural subsidy belongs to an independent local subsidy business scenario, meeting the splitting conditions, then a tag update suggestion for adding a sub-tag for new energy vehicle subsidies in rural areas under the original new energy subsidy consultation tag is generated.

[0044] As shown above, the system categorizes and records unlabeled information, and generates update suggestions by clustering the unlabeled corpus based on trigger conditions. It can automatically aggregate newly added semantics that are not covered, saving manual collection and reducing iteration workload. It also avoids label redundancy and confusion caused by scattered additions. Multiple trigger conditions can adapt to the needs of different businesses and update frequencies, balancing timeliness and resource allocation. Multiple types of suggestions match different reasons for not being labeled, and can provide adaptation solutions for semantic changes, maintaining the clarity and coverage of the label dictionary, improving the accuracy of subsequent automatic labeling, and making the label system flexibly adaptable to business changes, thereby improving the system's iteration efficiency and adaptability.

[0045] In some optional implementations, when generating tag update suggestions based on the semantic features of each corpus cluster and the tag features of the original tag dictionary, if the number of valid corpora within the corpus cluster is greater than or equal to a first quantity threshold, the semantic features do not match any tags in the original tag dictionary, and the proportion of the corpus cluster in unlabeled corpora within the statistical period is greater than or equal to a first proportion, then a tag update suggestion for adding a new tag is generated; if there is any original tag in the original tag dictionary that matches the semantic features, but the feature similarity between the semantic features and the features of the corpora already matched under the original tag is greater than or equal to a first similarity... If a similarity threshold is set, a tag update suggestion for splitting the tags is generated. If there are at least two original tags in the original tag dictionary that match the semantic features, and the overlap of the two original tags in the corpus within the statistical period is greater than or equal to the second similarity threshold, a tag update suggestion for merging the tags is generated. If there is any original tag in the original tag dictionary that matches the semantic features of the corpus cluster, but the tag value range of the semantic features exceeds the original value range of the original tags, and the proportion of the corpus in the super-value range in the unlabeled corpus within the statistical period is greater than or equal to the second proportion, a tag update suggestion for updating the tag value range is generated.

[0046] As an example, if 12 unlabeled corpora are clustered into a corpus cluster for "New Energy Vehicle Rural Subsidy Consultation" within a statistical period, and the first quantity threshold is 10, the corpus cluster contains 12 valid corpora, meeting the threshold requirement. The semantic features of this corpus cluster do not match any existing tags in the original tag dictionary, and this corpus cluster accounts for 26% of the total unlabeled corpora within the statistical period, exceeding the preset first proportion threshold of 20%. Therefore, a tag update suggestion is generated to add a new tag for "New Energy Vehicle Rural Subsidy Consultation". If the original tag dictionary already contains the original tag "New Energy Subsidy Consultation", and the semantic features of this corpus cluster have a feature similarity of 82% with the features of the matched corpora under the original tag, exceeding the preset first similarity threshold of 80%, then a tag update suggestion is generated to split the original "New Energy Subsidy Consultation" tag into two tags: "Regular New Energy Subsidy Consultation" and "New Energy Vehicle Rural Subsidy Consultation". If the original tag dictionary contains two original tags, "used car trade-in subsidy consultation" and "used car trade-in discount consultation," and both tags match the semantic features of the corpus cluster, and the overlap of the tagged corpus for the two tags reaches 78% within the statistical period, exceeding the preset 70% second similarity threshold, it indicates that the business boundaries of the two tags are blurred and easily confused during tagging. Therefore, a tag update suggestion is generated to merge the two original tags into a new tag, "used car trade-in subsidy consultation." If the original tag dictionary contains an original tag matching the semantic features of the corpus cluster, such as "number of accidents involving insured vehicles in the previous year," and the preset original value range for this original tag is 0-5 times, and the untagged corpus exceeding the value range (6 accidents) within the statistical period accounts for 15% of the total untagged corpus, exceeding the preset 10% second proportion threshold, and 6 accidents meet the current insurance business rules, a tag update suggestion is generated to update the tag value range to 0-6 times.

[0047] As shown above, various tag update suggestions are generated based on the rules, which can accurately identify the optimization needs of the tag system, eliminate the need for manual sorting and analysis, reduce maintenance costs, and avoid the decline in the applicability of the tag system caused by manual operation. By adding, splitting, merging tags and adjusting value ranges, the tag system can continuously adapt to business changes, maintain high coverage and accuracy, provide a reliable foundation for subsequent tasks, and improve tagging accuracy and efficiency.

[0048] Step S103: Update the original tag dictionary according to the tag update suggestion to obtain the new version of the tag dictionary.

[0049] The new version tag dictionary refers to the tag dictionary generated after being updated with tag optimization suggestions in the suggestion mode. Once generated, the new version tag dictionary will replace the original tag dictionary and become the currently effective tag dictionary.

[0050] In some optional implementations, when updating the original tag dictionary according to the tag update suggestion to obtain a new version of the tag dictionary, interactive operations related to the tag update suggestion can be detected; the original tag dictionary is updated based on the interactive operations to obtain a new version of the tag dictionary.

[0051] The interactive operations can include accepting all update suggestions, accepting some update suggestions, modifying the update suggestions and then confirming acceptance, and rejecting all updates.

[0052] Specifically, if the interaction is detected as accepting all or part of the update suggestion, the tag hierarchy, tag definitions, tag names, and value ranges of the original tag dictionary are adjusted accordingly, and a new version of the tag dictionary is generated. If the interaction is detected as accepting the modified update suggestion, the original tag dictionary is updated according to the modified content, and a new version of the tag dictionary is generated. If the interaction is detected as rejecting all update suggestions, the entire contents of the original tag dictionary are retained, and the original tag dictionary is directly used as the effective tag dictionary after this update. At the same time, all untagged information and untagged corpora cached in this update process are cleared, and the tag update process is restarted only after the next update trigger condition is met. In addition, after generating the new version of the tag dictionary, it can be synchronously written to the application cache and update cache, replacing the stored data of the original tag dictionary, so that the new version of the tag dictionary officially becomes the effective tag dictionary in the current application mode.

[0053] As shown above, by detecting interactive operations related to tag update suggestions, tag optimization recognition is combined with user demand judgment. This not only improves the efficiency of tag sorting and reduces the manpower cost of manually checking the entire tag system, but also allows for flexible control of the tag update process based on the specific needs of the actual business. This ensures that the updated tag system meets the actual needs of the current business, further improves the applicability and accuracy of the updated tag system, and thus better adapts to the tag system maintenance needs of different business scenarios.

[0054] In some optional implementations, when updating the original tag dictionary based on an interactive operation to obtain a new version of the tag dictionary, if the interactive operation indicates that the tag update suggestion of adding new tags is accepted, then the original tag dictionary is updated based on the tag information of the new tags in the tag update suggestion to obtain a new version of the tag dictionary. The tag information includes at least one of tag name, tag definition, tag type, value range, description subject, whether multiple tags are selected, and whether a tag is required. If the interactive operation indicates that the tag update suggestion of splitting tags is accepted, then the original tag dictionary is updated based on the original tag identifier and the tag information of each new tag after splitting to obtain a new version of the tag dictionary. If the interactive operation indicates that the tag update suggestion of merging tags is accepted, then the original tag dictionary is updated based on the original tag identifier to be merged and the merged tag information to obtain a new version of the tag dictionary. If the interactive operation indicates that the tag update suggestion of updating the tag value range is accepted, then the original tag dictionary is updated based on the tag identifier to be updated and the latest suggested value range to obtain a new version of the tag dictionary.

[0055] As an example, if the interactive operation confirms the adoption of the update suggestion to add a new label for rural new energy vehicle subsidy consultation, then the label information such as the label name, business definition, preset value range, and associated scenarios corresponding to the label will be written into the original label dictionary to complete the configuration of the new label; if the interactive operation confirms the adoption of the update suggestion to split the original new energy subsidy consultation label, then the identifier of the original parent label will be retained, and the identifiers and corresponding label information of the two sub-labels obtained from the splitting, namely "regular new energy subsidy consultation" and "new energy vehicle rural subsidy consultation", will be added, and the label hierarchy structure of the original label dictionary will be updated; if the interactive operation confirms the adoption of the update suggestion to merge "old car trade-in subsidy consultation" and "used car trade-in discount consultation", then the original identifiers of the two original labels to be merged will be cancelled, and the complete label information of the merged "new and old car trade-in subsidy consultation" will be written into the original label dictionary; if the interactive operation refuses to adopt the update suggestion to update the value range of the number of accidents of the insured vehicle in the previous year, then the original 0-5 value range setting of the label will be retained unchanged, and the final new version of the label dictionary will be generated after all adjustments are completed.

[0056] As shown above, by combining user interaction operations to process different types of tag update suggestions accordingly, the tag dictionary can be dynamically and flexibly updated. This adapts to the iterative needs of the tag system under different business scenarios, effectively improving the efficiency and accuracy of tag dictionary updates, while reducing the time and manpower costs of tag system maintenance. Through updating methods such as adding tags, splitting tags, merging tags, and updating tag value domains, the updated tag dictionary can be guaranteed to meet the actual usage needs of business personnel, further improving the usability and practicality of the tag system.

[0057] Step S104: Based on the new version of the tag dictionary, add tags to the newly added corpus and untagged corpus that were not covered by the original tag dictionary, and generate tag annotation data.

[0058] Specifically, all cached new speech corpora not covered by the original tag dictionary, as well as untagged corpora accumulated within the statistical period, can be extracted. Semantic features of each individual corpus are extracted, and then the cosine similarity between these extracted semantic features and the features of all tags in the new version of the tag dictionary is calculated sequentially. The tag with the highest similarity exceeding a preset matching threshold is identified as the matching tag for that corpus, completing the supplementary tagging of the corresponding corpus. Finally, all supplementary tagging results are summarized to obtain complete tag annotation data. If the similarity between a single corpus and multiple tag features exceeds the preset matching threshold, the corpus is sorted from high to low similarity, and the final tagging result is determined based on the preset multi-selection tag rules for the business scenario. If the similarity between the semantic features of the corpus and all tag features in the new version of the tag dictionary is lower than the preset matching threshold, the corpus and its extracted semantic features are retained for unified processing after the next tag update cycle, avoiding annotation errors caused by forced matching that could affect the accuracy of subsequent tag updates.

[0059] As an example, after generating a new version of the tag dictionary that includes the "New Energy Vehicle Rural Subsidy Consultation" tag, semantic features were extracted from each of the 12 previously cached, untagged corpora of this type. Then, the semantic features of each corpus were compared with the features of all tags in the new tag dictionary to calculate feature similarity. The feature similarity between each corpus and the newly added "New Energy Vehicle Rural Subsidy Consultation" tag exceeded the preset 85% matching threshold and was the highest among all tags. Therefore, all 12 corpora were tagged accordingly, and after completing the tagging process, they were all included in the final tag-annotated data.

[0060] The tag generation method provided by this invention performs preliminary screening and tagging of newly added communication corpora using an original tag dictionary. This allows for the rapid identification of new communication corpora that cannot be covered by existing tags due to changes in business scenarios. By generating tag update suggestions using untagged information and existing untagged corpora in a backup corpus, the method enables dynamic iterative updates of the tag dictionary. This allows for adaptation to new tag requirements without the need for manual rebuilding of the entire tag system. After generating a new version of the tag dictionary, the method uniformly supplements the tags of the relevant untagged corpora, ensuring the completeness of the full corpus annotation and allowing the tag system to be dynamically updated in response to changes in business scenarios. This improves the flexibility and efficiency of communication corpus tagging and reduces the manpower cost of maintaining the tag system.

[0061] In some optional implementations, when supplementing newly added discourse corpora and untagged corpora not covered by the original label dictionary with tags based on the new version of the label dictionary to generate label annotation data, the version of historical untagged corpora in the candidate corpus can also be filtered to extract untagged corpora whose tag version is identified as the original label dictionary version or an earlier version, thus obtaining the corpora to be supplemented with tags; based on the corpora to be supplemented with tags and the newly added discourse corpora not covered by the original label dictionary, a set of corpora to be supplemented with tags is constructed; the new version of the label dictionary is called to perform tag matching and annotation on each corpus in the set of corpora to be supplemented with tags, thus obtaining label annotation data.

[0062] Specifically, when storing the corpus to be processed, each untagged corpus can be bound to a version identifier of the currently effective tag dictionary, thus recording the tag dictionary version corresponding to the corpus when it enters the untagged cache. After completing the version filtering, the historical corpus to be re-tagged is merged with the newly added untagged corpus within the current statistical period to remove duplicates, resulting in a complete set of corpus to be re-tagged. Then, semantic feature extraction and tag similarity matching operations are performed on each corpus in the corpus, ultimately yielding the re-tagged results for the entire corpus.

[0063] As an example, the original tag dictionary version was v1.2.0, and the new version generated after this update is v1.3.0. When filtering historical untagged corpora, a total of 5 untagged corpora with version identifier v1.0 and 3 untagged corpora with version identifier v1.1.0 were obtained. These historical corpora to be re-tagged were merged with the 17 newly added untagged corpora in this statistical period. After removing 1 duplicate corpus, a total of 24 corpora to be re-tagged were obtained. After calling the new version tag dictionary v1.3.0 to match them one by one, 22 corpora matched the corresponding tags and were re-tagged. The feature similarity of the remaining 2 corpora with all tags did not reach the matching threshold. Therefore, these 2 corpora and their corresponding semantic features were retained and cached for processing in the next update period.

[0064] The tag generation method provided by this invention filters and re-labels historical unlabeled corpora according to tag versions. This not only avoids missing unlabelable corpora under the old tag system, but also ensures that the corpus to be re-labeled covers both newly added business communication corpora and historically unlabeled corpora. It achieves synchronous completion of labeled corpora after the tag dictionary is updated, ensures the consistency between the tag labeling data and the new version of the tag dictionary, avoids mismatch problems caused by asynchronous labeling data after the tag system is updated, and also reduces the waste of resources caused by unlabeled corpora.

[0065] This embodiment provides a label generation method. Figure 2 This is a flowchart of a label generation method according to an embodiment of the present invention, such as... Figure 2As shown, the process includes the following steps: Step S201: Tag the newly added discourse corpus using the original tag dictionary to obtain the tagging results. For details, please refer to [link to relevant documentation]. Figure 1 Step S101 of the illustrated embodiment will not be described again here.

[0066] Step S202: When the labeling result indicates that the newly added discourse data has been covered by the original label dictionary, the matching label, confidence score and version identifier of the original label dictionary corresponding to the labeling result are used to generate label annotation data.

[0067] Specifically, when the semantic features extracted from the newly added communication data have a similarity to the tag features of any tag in the original tag dictionary that exceeds the preset tag confidence threshold, it can be determined that the newly added communication data has been covered by the original tag dictionary. The matched tag number, tag name, and matching similarity confidence are directly bound and stored with the version identifier of the currently effective original tag dictionary and the original communication data. This directly generates tag annotation data that can be used for subsequent business calls or model training. There is no need to transfer the data to the unlabeled cache, which can effectively reduce the storage pressure of the unlabeled cache and improve the processing speed of the subsequent tag update process.

[0068] As an example, the original tag dictionary version is v1.2.0, with a preset tagging confidence threshold of 80%. The newly added corpus content is "I want to inquire about the subsidies I can receive for buying a new energy vehicle this year". After extracting semantic features, the similarity between the corpus and the new energy subsidy consultation tag in the original tag dictionary is calculated to be 92%, which exceeds the preset confidence threshold. Therefore, it can be determined that the corpus has been covered by the original tag dictionary. The corpus, the matched tag number 003, the tag name "new energy subsidy consultation", and the matching confidence of 92% are directly bound to the original tag dictionary version identifier v1.2.0 and stored to directly generate usable tag annotation data. There is no need to transfer the corpus to the unlabeled cache pool for subsequent processing.

[0069] The tag generation method provided in this embodiment handles different cases depending on whether the newly added discourse corpus is covered by the original tag dictionary. For corpora that can be covered and matched by the original tag dictionary, the existing tagging results are directly reused to generate tag annotation data, without needing to repeat the tagging process with a new version of the tag dictionary. This approach avoids unnecessary repetitive calculations, effectively reduces the computational consumption of the annotation process, and improves the overall processing efficiency of tag annotation. On the other hand, it can completely preserve the mature and accurate annotation results under the original tag system, ensuring the accuracy of the annotation data.

[0070] As one or more specific application embodiments of the present invention, such as Figure 3As shown, during the initialization phase, the initialization module retrieved the vehicle service communication data from the past three months, totaling 12,450 entries, covering sales, delivery, and service scenarios for multiple vehicle models. This vehicle service communication data was then input into a large language model to obtain an initial tag dictionary, which was subsequently approved as version v1.0.0.

[0071] In application mode, if the received new call data (call ID: CALL-20260515-001) is: "I am interested in the A11 model and would like to know if there are any cars in stock. Also, how does your zero-interest loan program work?", then the v1.0.0 tag dictionary and this data will be input into the large language model, and the model will perform tag matching verification. Semantic matching: The similarity between the corpus and the intended vehicle model label is 0.95, the similarity with the financial plan label is 0.91, and the similarity with the delivery cycle label is 0.88. Parameter extraction: The intended vehicle model A11 and the financial plan (zero-interest loan) are extracted from the corpus. Value range verification: Vehicle model A11 is within the T001 value range. Conflict detection for zero-interest loan (corresponding to T003): There is no semantic conflict between T001, T003, and T006, indicating that all content has passed the matching verification. Next, the results are written to the label result table, recording the call ID, matching label, confidence level (0.91), and version v1.0.0. Since the confidence level (0.91) is ≥ 0.8, this data directly participates in downstream system analysis.

[0072] If the newly received call data (Call ID: CALL-20260515-089) indicates that the financing plan for this model is 15% lower than competitors, with a down payment starting at 20%, and also includes lifetime free maintenance. However, I am concerned that since your brand is relatively new, there might be issues with its resale value later on. Then, the v1.0.0 tag dictionary and this data will be input into the large language model, and the model will perform tag matching verification. Semantic matching: Similarity with financial solutions is 0.85, similarity with price negotiations is 0.78, and similarity with competitors is 0.72; Parameter extraction: Down payment ratio is extracted as 20%, and price advantage is 15% lower than competitors; Value range verification: 20% is within a reasonable range, but there is no corresponding tag for lifetime free maintenance; Conflict detection: If there is no conflict, the matching verification result indicates that there is unmatched content. The following data was used: Corpus ID: CALL-20260515-089; Original Corpus: Complete call text; Semantic Feature Summary: Involves multiple core sales scenarios such as financial solutions, price advantages, down payment ratios, after-sales maintenance, and concerns about brand resale value; Reason for Matching Failure: The existing tag system can cover financial solutions and price negotiations, but cannot cover lifetime free maintenance and concerns about brand resale value; List of Matched Tags: Financial Solutions (0.85) Price Negotiation (0.78); Semantic Content Not Covered: Lifetime Free Maintenance, Concerns about Brand Resale Value; Code for Reason for Not Tagging: PARTIAL_COVER; Tag Version Information: v1.0.0; Entry Time: 202X-05-15 14:32:18, and stored in the backup corpus.

[0073] If the tag update suggestions are triggered by conditions including timed triggering (every Sunday at 02:00 AM), quantity threshold triggering (when the candidate corpus reaches 100 entries), and dynamic weighting parameters (R... high =50 items / hour, R low =5 entries / hour). From May 18th to May 20th, 2026, the rate of newly added untagged corpora was consistently higher than 60 entries / hour (peak period). The threshold was lowered from 100 entries to 50 entries, and the suggestion mode was triggered at 15:30 on May 20th. At this time, the candidate corpus contained 156 untagged corpora, all with tag version v1.0.0.

[0074] 156 corpora were vectorized and encoded, and cosine similarity between corpora was calculated. A dynamic threshold of 0.85 was set. Cluster analysis yielded the following corpus clusters: Cluster A: After-sales benefits (41 items); Core semantics: lifetime free maintenance, free roadside assistance, extended warranty service, battery warranty; Frequency: 41 items; Percentage in the last 7 days: 41 / 156 = 6.3%. Cluster B: Brand residual value (28 items); Core semantics: brand residual value, used car residual value, brand establishment time, market recognition; Frequency: 28 items; Percentage in the last 7 days: 17.9%. Cluster C: Intelligent driving functions (35 items); Core semantics: Huawei ADS, urban NCA, automatic parking, LiDAR; Frequency: 35 items; Percentage in the last 7 days: 22.4%. Cluster D: Charging pile installation (19 items); Core semantics: home charging pile, installation conditions, property coordination, charging power; Frequency: 19 items. Percentage in the last 7 days: 12.2%. Filtered by frequency (frequency ≥ 5 and percentage ≥ 3%).

[0075] The filtered corpus clusters are grouped and input into a large language model. Corpus cluster A triggers a new tag suggestion: Valid corpus quantity ≥ 41 entries, ≥ 5 entries; Existing after-sales benefit T007 is a boolean type, only indicating whether it applies, and cannot cover specific benefit types such as lifetime free maintenance; Recent 7-day percentage ≥ 3% (26.3%). For example: { "suggestion_type": "Add a new tag"; "reason": "The semantic focus of corpus A is on specific after-sales rights and benefits. The existing T007 only indicates whether after-sales service is involved, which cannot cover the subdivided semantics such as 'lifetime free maintenance' and 'extended warranty'. Moreover, it accounts for 26.3% in the past 7 days, indicating high business coverage." "new_tag": { "tag_id": "T009"; "tag_name": "Specific after-sales rights"; "description": "Specific after-sales service rights explicitly mentioned during the call"; "tag_type": "string"; "possible_values": "Lifetime free maintenance, extended warranty service, free roadside assistance, battery warranty, charging benefits, and others"; "entity_type": "call"; "is_multiple_choice": true; "is_required": false; }} Corpus cluster B triggered a suggestion to add a new tag: 28 valid corpora, none of which match existing tags; accounting for 17.9% ≥ 3%. For example: { "suggestion_type": "Add a new tag"; "new_tag": { "tag_id": "T010"; "tag_name": "Brand Confidence"; "description": "Customers' concerns or expressions of confidence regarding brand value retention, market recognition, and long-term development"; "tag_type": "string"; "possible_values": "Concerns about resale value, short brand history, high market recognition, trust in technology, other"; "entity_type": "customer"; "is_multiple_choice": true; "is_required": false; }} Corpus cluster C triggers a tag splitting suggestion: Semantically, it has no direct match with existing tags, but involves some technical comparisons in competitor comparison T004; analysis shows that the semantics of this cluster focus on intelligent driving functions, and its semantics differ significantly from the broad semantics of existing competitors; therefore, it should be determined as a new tag rather than an extension of the T004 value range. For example: { "suggestion_type": "Add a new tag"; "new_tag": { "tag_id": "T011"; "tag_name": "Focus on Intelligent Driving Functions"; "description": "Customer's concerns or inquiries regarding intelligent driving assistance functions (NCA, automatic parking, etc.)"; "tag_type": "string"; "possible_values": "Urban NCA, Highway NCA, Automatic Parking, LiDAR, Other"; "entity_type": "customer"; "is_multiple_choice": true; "is_required": false; }} Corpus cluster D triggers a new tag suggestion: 19 valid corpora, no matching with existing tags; percentage 12.2% ≥ 3%. For example: { "suggestion_type": "Add a new tag"; "new_tag": { "tag_id": "T012"; "tag_name": "Charging conditions"; "description": "Involves charging infrastructure conditions such as home charging pile installation, property coordination, and charging power"; "tag_type": "string"; "possible_values": "Can install home charger, property management obstruction, insufficient power, reliance on public charger, other"; "entity_type": "customer"; "is_multiple_choice": false; "is_required": false; } Furthermore, a confirmation page for tag optimization suggestions can be generated in the management backend, displaying the following content: Corpus cluster to be processed: 41 entries related to after-sales rights; Semantic feature summary: Customers frequently inquire about specific after-sales rights such as lifetime free maintenance, extended warranty, and roadside assistance; Example corpus: Does the lifetime free maintenance you mentioned mean it's given every year? Is the battery warranty 8 years or lifetime? If there's an accident, is roadside assistance available 24 hours a day? Suggested action: Add a new tag for specific after-sales rights T009; Reason for suggestion: The existing T007 only indicates whether after-sales service is involved, which is not granular enough; This corpus cluster accounts for 26.3%, with high coverage; Adding it can increase the tagging coverage by about 18%; Action options: Confirm adoption, reject and discard, temporarily accumulate.

[0076] Corpus cluster to be processed: 28 entries related to brand residual value; Semantic feature summary: Customers have concerns about the short establishment time of brands, residual value, and residual value of used cars; Suggested action: Add a tag "Brand Confidence T010"; Action options: Confirm adoption, reject and discard, temporarily accumulate.

[0077] If all 41 items related to after-sales rights are confirmed and adopted, T009 will be added, and the data will be removed from the candidate list. If all 28 items related to brand residual value are confirmed and adopted, T010 will be added, and the data will be removed from the candidate list. If all 35 items related to intelligent driving functions are confirmed and adopted, T011 will be added, and the data will be removed from the candidate list. If all 19 items related to charging installation are temporarily accumulated, the waiting timestamp will be updated, and the data will be retained in the candidate list. A v2.0.0 version tag dictionary will be generated, adding tags T009-T012, while retaining historical versions from v1.0.0. After v2.0.0 is generated, it will be automatically applied to the application mode. All subsequent new communication data will be tagged using v2.0.0. The tagged data for version v1.0.0 in the historical tag results table will retain the original version identifier and will not be updated retroactively.

[0078] After version v2.0.0 is generated, the system automatically identifies the target corpus for supplementary labeling. Target corpus A1: Historical unlabeled corpus (v1.0.0 version in the candidate corpus); the system filters unlabeled corpus with the tagged version v1.0.0 in the candidate corpus, with a total of 87 remaining (excluding processed and temporarily accumulated corpus). Target corpus B1: Newly added unlabeled corpus during the v1.0.0 period; including 156 corpus entries already in the database, such as CALL-20260515-089. These are merged to form a set of corpus entries to be supplemented with labels, totaling 243 entries.

[0079] During off-peak business hours (01:00-04:00), start batch asynchronous tagging tasks with a batch size of 50 tags.

[0080] Example of relabeling: CALL-20260515-089 (original PARTIAL_COVER corpus); The corpus "This car model's financing plan is 15% cheaper than competitors, and the down payment can start at 20%, plus lifetime free maintenance. However, I'm worried that since your brand is relatively new, will its resale value be affected?" is matched with the tag in the large language model input from the v2.0.0 tag dictionary for tag matching verification. Semantic matching: Similarity with T003 financial plan 0.85, similarity with T002 price negotiation 0.78, similarity with T009 specific after-sales benefits 0.92, similarity with T010 brand confidence 0.89; Parameter extraction: down payment ratio = 20%, specific after-sales benefits are lifetime free maintenance, brand confidence is concerns about resale value; Value range verification: all parameter values ​​are within the range of the corresponding label values; Conflict detection: no conflict. This result is written to the label result table, recording the call ID, matching label, confidence level 0.86, and version v2.0.0. This corpus is removed from the candidate corpus.

[0081] Furthermore, if on June 1, 202X, the business team discovers that the T011 intelligent driving function added in v2.0.0, in comparison with the existing T004 competitor, has annotation confusion in some corpora, affecting the accuracy of downstream data analysis, then the system backend can select the version rollback function to roll back the currently effective tag dictionary from v2.0.0 to v1.0.0. The tag dictionary table will mark v1.0.0 as the currently effective version; from the rollback time, all newly added corpora will be tagged using v1.0.0; historical data (v1.0.0 and v2.0.0 versions) in the tag results table will be retained without modification.

[0082] If the user selects the specified relabeling function: the corpus range is all corpora labeled using v2.0.0 from May 21, 202X to June 1, 202X; if version v1.0.0 is used, then the application mode will be re-executed for the corpora within the specified range according to v1.0.0, generating new label result data and marking the version as v1.0.0-re. The original v2.0.0 version's labeling results will still be retained in the label result table for comparison and auditing.

[0083] If, on June 18, 202X (during the 618 promotion period), the system monitors that the rate of newly added unlabeled corpus reaches 120 entries / hour, far exceeding R_high=50 entries / hour, then the Propose Mode quantity threshold will be reduced from 100 to 50 entries; the time window will be shortened from 7 days to 3 days; if the candidate corpus reaches 50 entries within 6 hours, the Propose Mode will be triggered immediately. If the server CPU load is monitored to reach 85%, exceeding the 80% threshold, the Propose Mode clustering calculation task will be paused, the current progress will be preserved, and it will automatically resume after the CPU load drops below 70%. The application mode real-time labeling service will not be affected, maintaining 99.99% availability.

[0084] This embodiment also provides a label generation apparatus for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0085] This embodiment provides a label generation device, such as... Figure 4 As shown, it includes: The corpus labeling module 401 is used to label the newly added corpus of speech using the original label dictionary to obtain the labeling results; The suggestion generation module 402 is used to generate label update suggestions based on unlabeled information and unlabeled corpus in the backup corpus when the labeling result indicates that the newly added corpus is not covered by the original label dictionary. The unlabeled information is used to describe the reasons and corresponding semantic features of the newly added corpus that cannot be fully labeled by the original label dictionary. Dictionary update module 403 is used to update the original tag dictionary based on tag update suggestions to obtain a new version of the tag dictionary; The tag generation module 404 is used to supplement the newly added corpus and untagged corpus that are not covered by the original tag dictionary with tags based on the new version of the tag dictionary, and generate tag annotation data.

[0086] In some optional implementations, the corpus tagging module 401 includes: The matching and verification unit is used to perform tag matching and verification processing on the newly added dictation corpus based on the original tag dictionary to obtain the matching and verification results. The tag matching and verification includes at least one of semantic matching, parameter extraction, value range verification and conflict detection. The first tagging unit is used to determine that the new communication corpus is not covered by the original tag dictionary if the matching verification result indicates that some or all of the content of the new communication corpus does not match the corresponding tag in the original tag dictionary. The second labeling unit is used to determine that if the matching verification result indicates that all the content of the newly added communication data has been matched, the labeling result is that the newly added communication data has been covered by the original label dictionary.

[0087] In some alternative implementations, it is suggested that generation module 402 include: The unlabeled information acquisition unit is used to acquire unlabeled information when the labeling result represents that the newly added discourse data is not covered by the original label dictionary. The unlabeled information includes one of the following: unlabeling reason, similarity of matched labels, uncovered semantic content, and label version. The unlabeling reason includes at least one of the following: full semantic mismatch, partial semantic uncovering, and label value exceeding the preset label value range. The untagged corpus clustering unit is used to cluster the corpus fragments corresponding to the untagged information with the untagged corpus based on the target triggering conditions to obtain at least one corpus cluster. The target triggering conditions include one of the following: timed triggering, quantity threshold triggering, untagged corpus speed-up triggering, and user operation command triggering. The tag update suggestion generation unit is used to generate tag update suggestions based on the semantic features of each corpus cluster and the tag features of the original tag dictionary. The tag update suggestions include at least one of adding tags, splitting tags, merging tags, and updating tag value domains.

[0088] In some optional implementations, the tag update suggestion generation unit includes: The new tag suggestion generation unit is used to generate tag update suggestions for new tags if the number of effective corpora in the corpus cluster is greater than or equal to the first quantity threshold, the semantic features do not match any tags in the original tag dictionary, and the proportion of the corpus cluster in the unlabeled corpora within the statistical period is greater than or equal to the first proportion. The split label suggestion generation unit is used to generate a split label update suggestion if there is any original label in the original label dictionary that matches the semantic features, but the semantic features have a feature similarity greater than or equal to the first similarity threshold with the features of the matched corpus under the original label. The merged label suggestion generation unit is used to generate a merged label update suggestion if there are at least two original labels in the original label dictionary that match the semantic features, and the overlap of the two original labels in the corpus within the statistical period is greater than or equal to the second similarity threshold. The update label value range suggestion unit is used to generate a label update suggestion for updating the label value range if there is any original label in the original label dictionary that matches the semantic features of the corpus cluster, but the label value range of the semantic features exceeds the original value range of the original label, and the proportion of the corpus in the excess value range in the unlabeled corpus within the statistical period is greater than or equal to the second proportion.

[0089] In some alternative implementations, the dictionary update module 403 includes: An interactive operation detection unit is used to detect interactive operations related to tag update suggestions; The original dictionary update unit is used to update the original tag dictionary based on interactive operations to obtain a new version of the tag dictionary.

[0090] In some optional implementations, the original dictionary update unit includes: The first update subunit is used to update the original tag dictionary based on the tag information of the new tag in the tag update suggestion if the interactive operation indicates that the tag update suggestion of the new tag is accepted, so as to obtain a new version of the tag dictionary. The tag information includes at least one of the following: tag name, tag definition, tag type, value range, description subject, whether multiple tags are selected, and whether a tag is required. The second update subunit is used to update the original tag dictionary based on the original tag identifier and the tag information of each new tag after splitting if the interactive operation indicates that the tag update suggestion of splitting the tags is accepted, so as to obtain a new version of the tag dictionary. The third update subunit is used to update the original tag dictionary based on the original tag identifiers to be merged and the merged tag information if the interactive operation indicates that the tag update suggestion of merging tags is accepted, so as to obtain a new version of the tag dictionary. The fourth update subunit is used to update the original label dictionary based on the label identifier to be updated and the latest suggested value range if the interactive operation indicates that the label update suggestion is adopted, so as to obtain a new version of the label dictionary.

[0091] In some alternative implementations, the label generation module 404 includes: The version filtering unit is used to filter the historical untagged corpus in the candidate corpus by version, extract the untagged corpus whose tag version is identified as the original tag dictionary version or an earlier version, and obtain the corpus to be tagged; The corpus construction unit is used to construct a corpus to be labeled based on the corpus to be labeled and the newly added corpus of speech not covered by the original label dictionary; The label annotation unit is used to call the new version of the label dictionary to perform label matching and annotation on each piece of text in the corpus to be tagged, and obtain the label annotation data.

[0092] In some optional implementations, it is suggested that the generation module 402 is also used to generate label annotation data by taking the matching label, confidence level and version identifier of the original label dictionary corresponding to the labeling result when the labeling result characterizes the newly added doctrinal data that has been covered by the original label dictionary.

[0093] The label generation apparatus provided in this embodiment of the invention can execute the label generation method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method. Further functional descriptions of the various modules and units described above are the same as in the corresponding embodiments described above, and will not be repeated here.

[0094] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0095] The following is a detailed reference. Figure 5 The diagram illustrates a structural schematic suitable for implementing an electronic device according to embodiments of the present invention. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 501, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 502 or a program loaded from memory 508 into random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of the electronic device. The processor 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0096] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 508 including, for example, magnetic tapes, hard disks, etc.; and communication devices 509. Communication device 509 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 5 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0097] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a memory 508, or installed from a ROM 502. When the computer program is executed by the processor 501, it performs the functions defined in the tag generation method of the embodiments of the present invention.

[0098] Figure 5The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0099] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the tag generation method shown in the above embodiments is implemented.

[0100] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0101] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A label generation method, characterized in that, The method includes: The newly added corpus of speech is tagged using the original tag dictionary to obtain the tagging results; When the labeling result indicates that the newly added common speech corpus is not covered by the original label dictionary, a label update suggestion is generated based on the unlabeled information and the unlabeled corpus in the backup corpus. The unlabeled information is used to describe the reasons and corresponding semantic features of the newly added common speech corpus that cannot be fully labeled by the original label dictionary. The original tag dictionary is updated according to the tag update suggestions to obtain a new version of the tag dictionary; Based on the new version of the tag dictionary, the newly added corpus and the untagged corpus that were not covered by the original tag dictionary are tagged to generate tag annotation data.

2. The method according to claim 1, characterized in that, The process of tagging the newly added discourse corpus using the original tag dictionary to obtain the tagging results includes: Based on the original tag dictionary, the newly added discourse corpus is subjected to tag matching and verification processing to obtain the matching and verification result. The tag matching and verification includes at least one of semantic matching, parameter extraction, value range verification and conflict detection. If the matching verification result indicates that some or all of the newly added communication data does not match the corresponding tags in the original tag dictionary, then the tagging result is determined to be that the newly added communication data is not covered by the original tag dictionary. If the matching verification result indicates that all content of the newly added communication corpus has been matched, then the tagging result is determined to be that the newly added communication corpus has been covered by the original tag dictionary.

3. The method according to claim 1, characterized in that, When the labeling result indicates that the newly added discourse corpus is not covered by the original label dictionary, a label update suggestion is generated based on the unlabeled information and the unlabeled corpus in the backup corpus, including: When the tagging result indicates that the newly added discourse data is not covered by the original tag dictionary, the untagged information is obtained. The untagged information includes at least one of the following: untagged reason, similarity of matched tags, uncovered semantic content, and tag version. The untagged reason includes at least one of the following: full semantic mismatch, partial semantic uncovering, and tag value exceeding the preset tag value range. Based on the target triggering condition, the corpus fragments corresponding to the untagged information are clustered with the untagged corpus to obtain at least one corpus cluster. The target triggering condition includes one of timed triggering, quantity threshold triggering, untagged corpus speed-up triggering, and user operation command triggering. Based on the semantic features of each corpus cluster and the tag features of the original tag dictionary, a tag update suggestion is generated. The tag update suggestion includes at least one of adding tags, splitting tags, merging tags, and updating tag value domains.

4. The method according to claim 3, characterized in that, The step of generating tag update suggestions based on the semantic features of each corpus cluster and the tag features of the original tag dictionary includes: If the number of valid corpora in the corpus cluster is greater than or equal to the first quantity threshold, the semantic features do not match any of the tags in the original tag dictionary, and the proportion of the corpus cluster in the untagged corpora within the statistical period is greater than or equal to the first proportion, then a tag update suggestion for the newly added tag is generated. If any original tag in the original tag dictionary matches the semantic feature, but the semantic feature has a feature similarity greater than or equal to the first similarity threshold with the feature of the matched corpus under the original tag, then a tag update suggestion for splitting the tag is generated. If there are at least two original tags in the original tag dictionary that match the semantic features, and the overlap of the two original tags in the corpus within the statistical period is greater than or equal to the second similarity threshold, then a tag update suggestion for the merged tags is generated. If there exists any original label in the original label dictionary that matches the semantic features of the corpus cluster, but the label value range of the semantic features exceeds the original value range of the original label, and the proportion of the corpus in the excess value range in the unlabeled corpus within the statistical period is greater than or equal to the second proportion, then a label update suggestion for the updated label value range is generated.

5. The method according to claim 1, characterized in that, The step of updating the original tag dictionary according to the tag update suggestion to obtain a new version of the tag dictionary includes: Detect interactive actions in response to the suggested label update; The original tag dictionary is updated based on the interactive operation to obtain a new version of the tag dictionary.

6. The method according to claim 5, characterized in that, The process of updating the original tag dictionary based on the interactive operation to obtain a new version of the tag dictionary includes: If the interactive operation indicates that the tag update suggestion of adding a new tag is adopted, the original tag dictionary is updated based on the tag information of the new tag in the tag update suggestion to obtain a new version of the tag dictionary. The tag information includes at least one of tag name, tag definition, tag type, value range, description subject, whether multiple tags are selected, and whether a tag is required. If the interactive operation indicates that the tag update suggestion of splitting tags is adopted, the original tag dictionary is updated based on the original tag identifier and the tag information of each new tag after splitting, so as to obtain the new version of the tag dictionary; If the interactive operation indicates that the tag update suggestion to merge tags is accepted, then the original tag dictionary is updated based on the original tag identifier to be merged and the merged tag information to obtain the new version of the tag dictionary; If the interactive operation indicates that the tag update suggestion to update the tag value range is adopted, the original tag dictionary is updated based on the tag identifier to be updated and the latest suggested value range to obtain the new version of the tag dictionary.

7. The method according to claim 1, characterized in that, The process of supplementing the newly added discourse corpus and the untagged corpus not covered by the original tag dictionary with tags based on the new version of the tag dictionary, and generating tag annotation data, includes: Version screening is performed on the historical untagged corpus in the candidate corpus, and untagged corpus with tag version identifiers that are the original tag dictionary version or earlier versions are extracted to obtain the corpus to be tagged; Based on the corpus to be supplemented and the newly added discourse corpus not covered by the original tag dictionary, construct a corpus set to be supplemented; The new version of the tag dictionary is invoked to perform tag matching and annotation on each piece of text in the corpus to be tagged, thereby obtaining the tag annotation data.

8. The method according to any one of claims 1 to 7, characterized in that, The method further includes: When the labeling result indicates that the newly added corpus has been covered by the original label dictionary, the matching label, confidence score, and version identifier of the original label dictionary corresponding to the labeling result are used to generate the label annotation data.

9. A label generating apparatus, characterized in that, The device includes: The corpus tagging module is used to tag newly added linguistic corpus using the original tag dictionary to obtain tagging results; The suggestion generation module is used to generate a tag update suggestion based on the unlabeled information and the unlabeled corpus in the backup corpus if the tagging result indicates that the newly added corpus is not covered by the original tag dictionary. The unlabeled information is used to describe the reasons and corresponding semantic features of the newly added corpus that cannot be fully labeled by the original tag dictionary. The dictionary update module is used to update the original tag dictionary according to the tag update suggestion to obtain a new version of the tag dictionary; The tag generation module is used to add tags to the newly added corpus and the untagged corpus that are not covered by the original tag dictionary based on the new version of the tag dictionary, and generate tag annotation data.

10. An electronic device, characterized in that, include: A memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, the processor executing the computer instructions to perform the method of any one of claims 1 to 8.

Images