An intelligent label generation method and device, computer equipment and storage medium

Abstract

The application relates to an intelligent label generation method and device, computer equipment and a storage medium. The method comprises the following steps: performing word segmentation calculation and text length judgment on filtered text, performing word segmentation weight calculation and keyword weight calculation on the filtered text according to the judgment result, extracting keywords from the filtered text according to the obtained word weight, judging the extracted keywords based on a deep learning BERT model, performing label feature word association matching by using the judgment result and a pre-set word library rule, performing weight summation calculation on an initial label by using a weight summation algorithm, performing label screening on a candidate label according to a pre-set rule, performing weight scaling calculation on the screened label, setting a rule word library based on a badcase and prior knowledge, performing mutual exclusion label processing on the scaled label according to the rule word library, and obtaining a final label. The method can improve the robustness of a label generation system.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method, apparatus, computer device and storage medium for generating a smart label. Background Art

[0002] Tags are used to categorize article types and content, allowing users to easily understand which industries or stocks a news article primarily covers and to visualize the price fluctuations of those stocks or sectors. Based on an article, such as news information, we generate words that best represent the article's theme or central idea. These words are then linked to a tag feature word mapping table to generate tags for different dimensions (stocks, themes, industries, etc.), and assign a credibility rating to each tag, i.e., the probability that the article will be labeled with that tag. Tags also serve other purposes, such as article classification, user profiling, and personalized recommendations.

[0003] Text labeling in existing label generation systems is divided into extractive, generative, and classification methods based on the implementation method. The extractive method is to extract the vocabulary in the original text by calculating the weight of the words in the original text; the generative method is to use the semantic features of the text to generate vocabulary through a generative model, and the vocabulary is not necessarily the words in the original text; a common approach is based on classification, that is, designing a multi-layer classification tree, the category is the label, and each text is annotated with a category as training data. A classification algorithm is used to classify the text using the semantic features of the text, and the output category is the label. However, the supervised method of text labeling in existing label generation systems requires a large amount of labeled data, and the label system needs to be retrained after being updated before it can be used. There are also problems such as inflexible label expansion and low robustness. Summary of the Invention

[0004] Based on this, it is necessary to provide an intelligent label generation method, device, computer equipment and storage medium that can improve the robustness of the label generation system in response to the above technical problems.

[0005] A method for generating a smart label, the method comprising:

[0006] Get information text;

[0007] Perform text cleaning and regular expression screening on the information text to obtain filtered text;

[0008] Perform word segmentation calculation and text length judgment on the filtered text, and calculate word segmentation weight and keyword weight based on the judgment results to obtain word weight;

[0009] Extract keywords from the filtered text based on word weights, perform ambiguous word judgment on the extracted keywords based on the deep learning BERT model, and use the judgment results and pre-set vocabulary rules to perform label feature word association matching to obtain the initial label;

[0010] The initial label is calculated by using a weight summation algorithm to obtain a candidate label;

[0011] The candidate label is screened according to a pre-set rule, and the screened label is calculated by weight scaling to obtain a scaled label;

[0012] A rule library is set based on badcase and prior knowledge, and the scaled label is processed by mutual exclusion label according to the rule library to obtain a final label.

[0013] In one embodiment, the filtered text is calculated by word segmentation and text length judgment, and the word weight is obtained by calculating the word weight and keyword weight of the filtered text according to the judgment result, including:

[0014] The filtered text is segmented by a segmentation tool, and the length of the segmented filtered text is judged. If the segmented filtered text is a short text, the weight is calculated according to the word order to obtain the word weight;

[0015] If the segmented filtered text is a long text, the textrank algorithm is used to calculate the keyword weight of the words in the segmented filtered text to obtain the word weight.

[0016] In one embodiment, the extracted keywords are judged based on a deep learning BERT model, and the initial label is obtained by using the judgment result and a pre-set word library rule for label feature word association matching, including:

[0017] The extracted keywords are classified based on a deep learning BERT model, and the keywords of general concepts are determined as ambiguous words. If the keyword is an ambiguous word, the extracted keyword is disambiguated by combining the context semantics to obtain the disambiguated keyword;

[0018] The disambiguated keyword is associated and matched with the label feature word according to a pre-set word library rule to obtain the initial label;

[0019] If the keyword is not an ambiguous word, the keyword is associated and matched with the label feature word according to a pre-set word library rule to obtain the initial label.

[0020] In one embodiment, the information text is cleaned and regularly screened to obtain a filtered text, including:

[0021] The HTML format in the information text is removed, and the full-width and half-width symbols are unified to obtain the cleaned text;

[0022] A regular table is designed for the titles with naming rules in the cleaned text, and different types of information are judged by the titles, entering different processing flows to obtain filtered text.

[0023] In one embodiment, a weighted sum algorithm is used to perform weighted sum calculation on the initial tags to obtain candidate tags, including:

[0024] The weighted sum algorithm is used to calculate the weight of the initial label. The weight of the initial label is weight + = weight * pow (a, b), where a and b are the word weights of different keywords corresponding to the initial label, and pow (a, b) represents the attenuation coefficient.

[0025] The initial labels are selected according to their weights and pre-set thresholds to obtain candidate labels.

[0026] In one embodiment, candidate tags are screened according to pre-set rules, and weight scaling calculation is performed on the screened tags to obtain scaled tags, including:

[0027] The commonly used words in metaphors are processed into exclusion words and essential words. If the candidate label and the exclusion word appear at the same time in the text, the candidate label is deleted; if the candidate label and the essential word do not appear at the same time in the text, the candidate label is deleted.

[0028] All filtered labels are uniformly processed by the scaling function, and the final accumulated weight value of each label is scaled to between (0,1] to obtain the scaled label.

[0029] In one embodiment, a rule vocabulary is set based on badcase and prior knowledge, and mutually exclusive label processing is performed on the scaled labels according to the rule vocabulary to obtain the final labels, including:

[0030] A rule vocabulary is set based on badcase and prior knowledge. The rule vocabulary stipulates that when the first label is set to be mutually exclusive with the second label, when a news article contains feature words of both the first label and the second label, only the second label is retained and the first label is deleted.

[0031] A smart label generating device, comprising:

[0032] The preprocessing module is used to obtain information text; the information text is cleaned and regularized to obtain filtered text;

[0033] The word weight calculation module is used to perform word segmentation calculation and text length judgment on the filtered text, and calculate the word segmentation weight and keyword weight of the filtered text according to the judgment results to obtain the word weight;

[0034] The feature word association matching module is configured to extract keywords from the filtered text according to the word weight, judge the extracted keywords based on a deep learning BERT model, perform label feature word association matching by using the judgment result and a pre-set word library rule, and obtain initial labels;

[0035] The label screening module is configured to perform weight summation calculation on the initial labels by using a weight summation algorithm, obtain candidate labels, perform label screening on the candidate labels according to a pre-set rule, perform weight scaling calculation on the screened labels, and obtain scaled labels.

[0036] The final label generation module is configured to set a rule word library based on badcases and prior knowledge, perform mutual exclusion label processing on the scaled labels according to the rule word library, and obtain final labels.

[0037] A computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0038] Obtain information text;

[0039] Perform text cleaning and regular screening on the information text to obtain filtered text;

[0040] Perform word segmentation calculation and text length judgment on the filtered text, perform word segmentation weight calculation and keyword weight calculation on the filtered text according to the judgment result, and obtain word weights;

[0041] Extract keywords from the filtered text according to the word weight, judge the extracted keywords based on a deep learning BERT model, perform label feature word association matching by using the judgment result and a pre-set word library rule, and obtain initial labels;

[0042] Perform weight summation calculation on the initial labels by using a weight summation algorithm to obtain candidate labels;

[0043] Perform label screening on the candidate labels according to a pre-set rule, perform weight scaling calculation on the screened labels, and obtain scaled labels.

[0044] Set a rule word library based on badcases and prior knowledge, perform mutual exclusion label processing on the scaled labels according to the rule word library, and obtain final labels.

[0045] A computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the following steps:

[0046] Obtain information text;

[0047] Perform text cleaning and regular screening on the information text to obtain filtered text;

[0048] Perform word segmentation calculation and text length judgment on the filtered text, and calculate word segmentation weight and keyword weight based on the judgment results to obtain word weight;

[0049] Extract keywords from the filtered text based on word weights, perform ambiguous word judgment on the extracted keywords based on the deep learning BERT model, and use the judgment results and pre-set vocabulary rules to perform label feature word association matching to obtain the initial label;

[0050] Use the weighted sum algorithm to perform weighted sum calculation on the initial labels to obtain candidate labels;

[0051] The candidate tags are screened according to the pre-set rules, and the weighted tags are scaled to obtain the scaled tags.

[0052] A rule vocabulary is set based on badcase and prior knowledge, and the scaled labels are processed with mutually exclusive labels according to the rule vocabulary to obtain the final labels.

[0053] The above-mentioned intelligent label generation method, device, computer equipment and storage medium first perform text cleaning and regular screening on the information text to avoid the incorrect typing of individual stock and institutional labels; perform word segmentation calculation and text length judgment on the filtered text, and perform word segmentation weight calculation and keyword weight calculation on the filtered text according to the judgment result to obtain word weight; extract keywords from the filtered text according to the word weight, perform ambiguous word judgment on the extracted keywords based on the deep learning BERT model, use the judgment result and the pre-set vocabulary rules to perform label feature word association matching to obtain the initial label, and use the weight addition algorithm to perform weight addition calculation on the initial label to obtain the candidate label. The label is the final output result, which represents the vocabulary of the content of the information text. The label weight is calculated by accumulating the scores of the keywords related to the label. Since the cumulative score of keywords on the label is positively correlated with the number of keywords, it is necessary to attenuate the summation to weaken the influence of the number on the summation, obtain more accurate labels, perform label screening on the candidate labels according to pre-set rules, and perform weight scaling calculation on the screened labels to obtain scaled labels; set a rule vocabulary based on badcase and prior knowledge, perform mutually exclusive label processing on the scaled labels according to the rule vocabulary, circumvent the labels that should not appear at the same time through the rules set by this application, and obtain accurate final labels. The present invention adopts an extractive method, which is an unsupervised learning method with stronger interpretability, faster calculation speed, no need for training data, and more flexible expansion of labels than supervised algorithms such as generation algorithms or classification algorithms, without the need for data retraining. The extractive method extracts words from the original text. Some words are not topical. To ensure that the output tags are topical and professional in the field, a keyword-to-label mapping calculation is introduced. Keywords are words in the original text, and labels are professional terms defined by the business. Multiple optimizations have been made to the data processing details, and texts with different characteristics can be processed in different ways, which is more adaptable and improves the robustness of the system. The system opens multiple operational entrances, and regular expressions or vocabulary can be adjusted according to business needs to achieve immediate effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 A schematic flow chart of a method for generating a smart label in one embodiment;

[0055] Figure 2 A flowchart of a method for generating a smart label according to an embodiment;

[0056] Figure 3 This is a structural block diagram of a smart label generating device according to an embodiment;

[0057] Figure 4 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0059] In one embodiment, Figure 1 As shown, a method for generating a smart label is provided, comprising the following steps:

[0060] Step 102: Obtain information text; perform text cleaning and regular expression screening on the information text to obtain filtered text.

[0061] Text cleaning involves removing HTML formatting from text and unifying traditional and simplified Chinese characters, as well as full-width and half-width symbols. Regular expression filtering involves creating a regular expression table for titles with naming patterns. This allows us to identify different types of information based on the title and process them accordingly. Regular expression filtering is primarily used to prevent the incorrect typing of individual stock or institutional labels, such as institutional opinions and disclaimers.

[0062] Step 104 , performing word segmentation calculation and text length judgment on the filtered text, and performing word segmentation weight calculation and keyword weight calculation on the filtered text according to the judgment results to obtain word weight.

[0063] Perform word segmentation calculation and text length judgment on the filtered text, and calculate word weights according to the length of the text to improve calculation efficiency.

[0064] Step 106: extract keywords from the filtered text according to the word weights, perform ambiguous word judgment on the extracted keywords based on the deep learning BERT model, and use the judgment results and pre-set vocabulary rules to perform label feature word association matching to obtain the initial label.

[0065] Based on word weights, words with weights greater than a certain threshold are extracted from the filtered text as keywords. The extracted keywords are then classified and identified as ambiguous using the deep learning BERT model. Because the final labels are displayed based on different dimensions, such as individual stock labels, regional labels, and person labels, noun disambiguation ensures that labels are accurate. For example, only references to the company "Laobing" will be labeled; those identified as general concepts will be ignored. Ambiguous words are ambiguous feature words, such as "Laobing" and "Saturday," which can refer to listed companies or general concepts. For general concepts, they can be: "The lives of our common people are getting better and better." For company names, they can be: "Laobing surpasses 10,000 stores! Listed chain integration is accelerating! The landscape of off-premises product categories is undergoing a major shift!" If the keyword is determined to be non-ambiguous, the keyword is matched against pre-set vocabulary rules for label feature word associations to assign a label to the keyword, resulting in the initial label for the information text.

[0066] Step 108: Use a weighted sum algorithm to perform weighted sum calculation on the initial tags to obtain candidate tags; filter the candidate tags according to pre-set rules, and perform weighted scaling calculation on the filtered tags to obtain scaled tags.

[0067] A weighted summation algorithm is used to weight the initial tags to generate candidate tags. Tags are the final output, representing the main theme of the news text. Tag weights are calculated by summing the scores of keywords associated with the tag. Since the cumulative score of a keyword for a tag is positively correlated with its number, a decaying summation is used to mitigate the effect of keyword number on the summation, resulting in more accurate candidate tags. For words commonly used in metaphors, a necessary exclusion process is used to determine whether they should be considered valid keywords. For example, "Nongfu Spring is like Moutai in water." Moutai is merely a metaphor, not a reference. Therefore, the metaphor "Moutai" is constrained, and "like" is excluded as an exclusion word. Exclusion: This addresses the issue of certain words appearing simultaneously with tags in the text, preventing us from typing the tag. Necessary: ​​Certain tags must appear with specific words in order to be typed. Due to the keyword extraction algorithm and the weighted summation of keywords, the range of word weights varies across each text. Therefore, a unified weight range is required to facilitate uniform threshold adjustments after the system goes live. All weight values ​​are uniformly processed by the scaling function, and the final accumulated weight value of each label is scaled to between (0, 1] to facilitate label management.

[0068] Step 110 : Setting a rule vocabulary based on badcase and prior knowledge, and performing mutually exclusive label processing on the scaled labels according to the rule vocabulary to obtain final labels.

[0069] To avoid ambiguity in text, mutually exclusive tags are required. For example, "US stock market dynamics" and "A-share dynamics" should not appear at the same time. A rule vocabulary is set based on badcase and prior knowledge, and the mutual exclusion function must be configured under the tag that is not printed. If label A is set to be mutually exclusive with label B, then when a news article contains feature words of both labels A and B, the system will print only label B and not label A, resulting in the final label.

[0070] In the above-mentioned intelligent label generation method, the information text is first cleaned and regularly screened to avoid the incorrect typing of individual stock and institutional labels; the filtered text is segmented and the text length is judged, and the filtered text is segmented and weighted according to the judgment results to obtain word weights; keywords are extracted from the filtered text according to the word weights, and the extracted keywords are judged as ambiguous words based on the deep learning BERT model. The judgment results and the pre-set vocabulary rules are used to perform label feature word association matching to obtain initial labels, and the weighted summation algorithm is used to perform weighted summation on the initial labels to obtain candidate labels. The label is the final output result, which represents the vocabulary of the content of the information text. The label weight is calculated by accumulating the scores of the keywords related to the label. Since the cumulative score of keywords on the label is positively correlated with the number of keywords, it is necessary to attenuate the summation to weaken the influence of the number on the summation, obtain more accurate labels, perform label screening on the candidate labels according to pre-set rules, and perform weight scaling calculation on the screened labels to obtain scaled labels; set a rule vocabulary based on badcase and prior knowledge, perform mutually exclusive label processing on the scaled labels according to the rule vocabulary, circumvent the labels that should not appear at the same time through the rules set by this application, and obtain accurate final labels. The present invention adopts an extractive method, which is an unsupervised learning method with stronger interpretability, faster calculation speed, no need for training data, and more flexible expansion of labels than supervised algorithms such as generation algorithms or classification algorithms, without the need for data retraining. The extractive method extracts words from the original text. Some words are not topical. To ensure that the output tags are topical and professional in the field, a keyword-to-label mapping calculation is introduced. Keywords are words in the original text, and labels are professional terms defined by the business. Multiple optimizations have been made to the data processing details, and texts with different characteristics can be processed in different ways, which is more adaptable and improves the robustness of the system. The system opens multiple operational entrances, and regular expressions or vocabulary can be adjusted according to business needs to achieve immediate effect.

[0071] In one embodiment, the filtered text is segmented and the text length is judged, and the segmentation weight and keyword weight are calculated based on the judgment results to obtain the word weight, including:

[0072] The filter text is segmented using the word segmentation tool, and the length of the segmented filter text is judged. If the segmented filter text is a short text, the weight is calculated according to the word order to obtain the word weight;

[0073] If the filtered text after word segmentation is a long text, the keyword weight of the words in the filtered text after word segmentation is calculated according to the textrank algorithm to obtain the word weight.

[0074] In one of the embodiments, the extracted keywords are subjected to ambiguity judgment based on a deep learning BERT model, and the judgment result and a pre-set word library rule are used for label feature word association matching to obtain an initial label, including:

[0075] The extracted keywords are subjected to classification processing based on a deep learning BERT model, and the keywords of a general concept are determined as ambiguous words. If the keywords are ambiguous words, the extracted keywords are subjected to disambiguation processing by combining context semantics to obtain disambiguated keywords.

[0076] The disambiguated keywords are subjected to label feature word association matching according to a pre-set word library rule to obtain an initial label.

[0077] If the keywords are not ambiguous words, the keywords are subjected to label feature word association matching according to a pre-set word library rule to obtain an initial label.

[0078] In one of the embodiments, the information text is subjected to text cleaning and regular screening to obtain filtered text, including:

[0079] The HTML format in the information text is removed, and full-width and half-width symbols are unified to obtain cleaned text.

[0080] A regular table is designed for the title with a naming rule in the cleaned text, different types of information are determined by the title, and different processing procedures are entered to obtain filtered text.

[0081] In one of the embodiments, a weight summation algorithm is used to perform weight summation calculation on the initial label to obtain a candidate label, including:

[0082] The weight summation algorithm is used to perform weight summation calculation on the initial label, and the weight of the initial label is weight+=weight*pow(a,b), where a and b are different word weights of the keywords corresponding to the initial label, and pow(a,b) represents a decay coefficient.

[0083] The initial label is selected according to the weight of the initial label and a pre-set threshold value to obtain a candidate label.

[0084] In specific embodiments, the pre-set threshold value is set according to actual business requirements. For example, if the recall rate is required to be low, the threshold value is set to be below 0.8, and if the accuracy is required to be high, the threshold value is set to be above 0.8.

[0085] In one of the embodiments, the candidate label is subjected to label screening according to a pre-set rule, and the screened label is subjected to weight scaling calculation to obtain a scaled label, including:

[0086] The commonly used words in metaphors are processed into exclusion words and essential words. If the candidate label and the exclusion word appear at the same time in the text, the candidate label is deleted; if the candidate label and the essential word do not appear at the same time in the text, the candidate label is deleted.

[0087] All filtered labels are uniformly processed by the scaling function, and the final accumulated weight value of each label is scaled to between (0,1] to obtain the scaled label.

[0088] In a specific embodiment, exclusion words are used to solve the problem that some special words and tags appear at the same time in the text, and we do not need to type the tag. Exclusion words cannot exist alone and need to be used in conjunction with feature words. They have the same principle as feature words but different functions. When a sentence hits a feature word but also hits an exclusion word, the system will not type the tag. It should be noted that all sentences with feature words must contain exclusion words. As long as one of the sentences does not have an exclusion word, the system will still type the tag normally.

[0089] Essential words are used in certain tags and must appear with specific words to be displayed. Essential words cannot exist alone and must be used in conjunction with feature words. They work the same way as feature words, but their function is different. If a sentence contains a feature word but not an essential word, the system will not display the tag. The system will only display the tag if both the feature word and the essential word are present. It's important to note that if the essential word appears in any one of the sentences containing multiple feature words, the tag will be displayed.

[0090] In one embodiment, a rule vocabulary is set based on badcase and prior knowledge, and mutually exclusive label processing is performed on the scaled labels according to the rule vocabulary to obtain the final labels, including:

[0091] A rule vocabulary is set based on badcase and prior knowledge. The rule vocabulary stipulates that when the first label is set to be mutually exclusive with the second label, when a news article contains feature words of both the first label and the second label, only the second label is retained and the first label is deleted.

[0092] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 1At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0093] In one embodiment, Figure 3 As shown, an intelligent tag generation device is provided, including: a pre-processing module 302, a word weight calculation module 304, a feature word association matching module 306, a tag screening module 308 and a final tag generation module 310, wherein:

[0094] The pre-processing module 302 is used to obtain information text; perform text cleaning and regular expression filtering on the information text to obtain filtered text;

[0095] The word weight calculation module 304 is used to perform word segmentation calculation and text length judgment on the filtered text, and perform word segmentation weight calculation and keyword weight calculation on the filtered text according to the judgment results to obtain word weight;

[0096] The feature word association matching module 306 is used to extract keywords from the filtered text according to the word weights, perform ambiguous word judgment on the extracted keywords based on the deep learning BERT model, and use the judgment results and pre-set vocabulary rules to perform label feature word association matching to obtain the initial label;

[0097] The tag screening module 308 is configured to perform weighted sum calculation on the initial tags using a weighted sum algorithm to obtain candidate tags; filter the candidate tags according to pre-set rules, and perform weighted scaling calculation on the filtered tags to obtain scaled tags;

[0098] The final label generation module 310 is used to set a rule vocabulary based on badcase and prior knowledge, and perform mutually exclusive label processing on the scaled labels according to the rule vocabulary to obtain the final label.

[0099] In one embodiment, the word weight calculation module 304 is further used to perform word segmentation calculation and text length judgment on the filtered text, and perform word segmentation weight calculation and keyword weight calculation on the filtered text according to the judgment results to obtain word weights, including:

[0100] The filter text is segmented using the word segmentation tool, and the length of the segmented filter text is judged. If the segmented filter text is a short text, the weight is calculated according to the word order to obtain the word weight;

[0101] If the filtered text after word segmentation is a long text, the keyword weight of the words in the filtered text after word segmentation is calculated according to the textrank algorithm to obtain the word weight.

[0102] In one embodiment, the feature word association matching module 306 is further configured to perform ambiguous word judgment on the extracted keywords based on the deep learning BERT model, and perform tag feature word association matching using the judgment result and pre-set vocabulary rules to obtain an initial tag, including:

[0103] The extracted keywords are classified based on the deep learning BERT model, and keywords of general concepts are determined to be ambiguous words. If the keywords are ambiguous words, the extracted keywords are disambiguated by combining the contextual semantics to obtain the disambiguated keywords.

[0104] According to the pre-set vocabulary rules, the disambiguated keywords are matched with label feature words to obtain the initial label;

[0105] If the keyword is not an ambiguous word, the keyword is matched with the label feature word association according to the pre-set vocabulary rules to obtain the initial label.

[0106] In one embodiment, the pre-processing module 302 is further configured to perform text cleaning and regular expression filtering on the information text to obtain filtered text, including:

[0107] Remove the HTML format in the information text, unify the traditional and simplified Chinese characters, and unify the full-width and half-width symbols to obtain the cleaned text;

[0108] A regular table is designed for the titles with naming rules in the cleaned text, and different types of information are judged by the titles, entering different processing flows to obtain filtered text.

[0109] In one embodiment, the tag screening module 308 is further configured to perform weighted sum calculation on the initial tags using a weighted sum algorithm to obtain candidate tags, including:

[0110] The weighted sum algorithm is used to calculate the weight of the initial label. The weight of the initial label is weight + = weight * pow (a, b), where a and b are the word weights of different keywords corresponding to the initial label, and pow (a, b) represents the attenuation coefficient.

[0111] The initial labels are selected according to their weights and pre-set thresholds to obtain candidate labels.

[0112] In one embodiment, the tag screening module 308 is further configured to screen candidate tags according to pre-set rules, and perform weight scaling calculation on the screened tags to obtain scaled tags, including:

[0113] The commonly used words in metaphors are processed into exclusion words and essential words. If the candidate label and the exclusion word appear at the same time in the text, the candidate label is deleted; if the candidate label and the essential word do not appear at the same time in the text, the candidate label is deleted.

[0114] All filtered labels are uniformly processed by the scaling function, and the final accumulated weight value of each label is scaled to between (0,1] to obtain the scaled label.

[0115] In one embodiment, the final label generation module 310 is further configured to set a rule vocabulary based on badcase and prior knowledge, and perform mutually exclusive label processing on the scaled labels according to the rule vocabulary to obtain the final labels, including:

[0116] A rule vocabulary is set based on badcase and prior knowledge. The rule vocabulary stipulates that when the first label is set to be mutually exclusive with the second label, when a news article contains feature words of both the first label and the second label, only the second label is retained and the first label is deleted.

[0117] For the specific definition of a smart label generation device, please refer to the definition of a smart label generation method above, which will not be repeated here. The various modules in the above-mentioned smart label generation device can be implemented in whole or in part by software, hardware, or a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above modules.

[0118] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 4As shown in the figure. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The network interface of the computer device is used to communicate with the external terminal through the network connection. The computer program is executed by the processor to implement an intelligent tag generation method. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the computer device, or an external keyboard, touchpad or mouse, etc.

[0119] Those skilled in the art can understand that, Figure 4 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0120] In one embodiment, a computer device is provided, including a memory and a processor, the memory stores a computer program, and the processor executes the computer program to implement the steps of the method in the above-mentioned embodiments.

[0121] In one embodiment, a computer storage medium is provided, which stores a computer program, and the computer program is executed by the processor to implement the steps of the method in the above-mentioned embodiments.

[0122] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0123] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0124] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A method for generating a smart label, characterized in that: The method comprises: Get information text; Perform text cleaning and regular expression screening on the information text to obtain filtered text; Perform word segmentation calculation and text length judgment on the filtered text, and perform word segmentation weight calculation and keyword weight calculation on the filtered text according to the judgment results to obtain word weight; Extract keywords from the filtered text according to the word weights, perform ambiguous word judgment on the extracted keywords based on the deep learning BERT model, and use the judgment results and pre-set vocabulary rules to perform label feature word association matching to obtain initial labels; Use the weighted sum algorithm to perform weighted sum calculation on the initial labels to obtain candidate labels; The candidate tags are screened according to pre-set rules, and weight scaling calculation is performed on the screened tags to obtain scaled tags; Setting a rule vocabulary based on badcase and prior knowledge, and performing mutually exclusive label processing on the scaled labels according to the rule vocabulary to obtain final labels; The candidate tags are screened according to pre-set rules, and weight scaling calculation is performed on the screened tags to obtain scaled tags, including: The words commonly used in metaphors are processed into exclusion words and essential words. If the candidate tag and the exclusion word appear at the same time in the text, the candidate tag is deleted; if the candidate tag and the essential word do not appear at the same time in the text, the candidate tag is deleted; All filtered labels are uniformly processed by the scaling function, and the final accumulated weight value of each label is scaled to between (0,1] to obtain the scaled label.

2. The method according to claim 1, characterized in that Perform word segmentation calculation and text length judgment on the filtered text, and perform word segmentation weight calculation and keyword weight calculation on the filtered text according to the judgment results to obtain word weight, including: Segmenting the filtered text according to a word segmentation tool, determining the length of the segmented filtered text, and if the segmented filtered text is a short text, calculating the weight according to the order of the words to obtain a word weight; If the filtered text after word segmentation is a long text, the keyword weight of the words in the filtered text after word segmentation is calculated according to the textrank algorithm to obtain the word weight.

3. The method according to claim 1, characterized in that Based on the deep learning BERT model, the extracted keywords are ambiguous and matched with each other using the judgment results and pre-set vocabulary rules to obtain the initial labels, including: The extracted keywords are classified based on the deep learning BERT model, and keywords of general concepts are determined to be ambiguous words. If the keywords are ambiguous words, the extracted keywords are disambiguated by combining the contextual semantics to obtain the disambiguated keywords. According to the pre-set vocabulary rules, the disambiguated keywords are matched with label feature words to obtain the initial label; If the keyword is not an ambiguous word, the keyword is matched with the label feature word association according to the pre-set vocabulary rules to obtain the initial label.

4. The method according to any one of claims 1 to 3, characterized in that The information text is cleaned and regularized to obtain filtered text, including: The HTML format in the information text is cleared, the traditional and simplified Chinese characters are unified, and the full-width and half-width symbols are unified to obtain the cleaned text; A regular table is designed for the titles with naming rules in the cleaned text, and different types of information are judged by the titles, entering different processing flows to obtain filtered text.

5. The method according to claim 4, characterized in that The weighted sum algorithm is used to perform weighted sum calculation on the initial labels to obtain candidate labels, including: The weighted sum algorithm is used to calculate the weighted sum of the initial labels, and the weight of the initial labels is obtained as , where a and b are the word weights of different keywords corresponding to the initial labels, and pow(a,b) represents the attenuation coefficient; The initial labels are selected according to their weights and pre-set thresholds to obtain candidate labels.

6. The method according to claim 5, characterized in that A rule vocabulary is set based on badcase and prior knowledge, and mutually exclusive label processing is performed on the scaled labels according to the rule vocabulary to obtain final labels, including: A rule word library is set based on badcase and prior knowledge. The rule word library stipulates that when a first tag is set to be mutually exclusive with a second tag, when a news article contains feature words of both the first tag and the second tag, only the second tag is retained and the first tag is deleted.

7. A smart label generating device, characterized in that: The device comprises: The preprocessing module is used to obtain information text; perform text cleaning and regularization filtering on the information text to obtain filtered text; A word weight calculation module is used to perform word segmentation calculation and text length judgment on the filtered text, and perform word segmentation weight calculation and keyword weight calculation on the filtered text according to the judgment results to obtain word weight; A feature word association matching module is used to extract keywords from the filtered text according to the word weights, perform ambiguous word judgment on the extracted keywords based on the deep learning BERT model, and use the judgment results and pre-set vocabulary rules to perform label feature word association matching to obtain initial labels; The tag screening module is used to perform weighted sum calculation on the initial tags using a weighted sum algorithm to obtain candidate tags; filter the candidate tags according to pre-set rules, and perform weighted scaling calculation on the filtered tags to obtain scaled tags, including: The words commonly used in metaphors are processed into exclusion words and essential words. If the candidate tag and the exclusion word appear at the same time in the text, the candidate tag is deleted; if the candidate tag and the essential word do not appear at the same time in the text, the candidate tag is deleted; All filtered labels are uniformly processed by the scaling function, and the final cumulative weight value of each label is scaled to between (0,1] to obtain the scaled label; The final label generation module is used to set a rule vocabulary based on badcase and prior knowledge, and perform mutually exclusive label processing on the scaled labels according to the rule vocabulary to obtain the final label.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images