A method, device, medium, and program product for tagging text content.

Abstract

A method, device, medium, and program product for text content tagging are disclosed, relating to the field of natural language processing computing. Because the device employs a technique of first performing preliminary semantic classification using a pre-defined tag model combined with context, and then performing precise vectorized matching within a specific knowledge database corresponding to that classification, it can first utilize the model's contextual understanding capabilities to confine the true meaning of words to a clear semantic category, thereby selecting a highly relevant knowledge database. This effectively avoids the semantic confusion caused by blindly searching the global knowledge base, overcomes the obstacle of traditional methods in distinguishing polysemous words in professional texts, and thus improves the accuracy and reliability of tag recognition.

Description

Technical Field

[0001] This application relates to the field of natural language processing computing, and more particularly to a method, apparatus, medium, and program product for tagging text content. Background Technology

[0002] With the rapid development of artificial intelligence technology, the demand for intelligent processing of unstructured content is growing. Especially in professional fields such as healthcare, massive amounts of text content need to be accurately understood and transformed into structured tagging information to support subsequent precise retrieval and analysis applications. This has led to widespread attention and application of automated content tagging technology.

[0003] In related technologies, rule-based content tagging systems are typically used to convert text into tags. These systems employ three core modules: a rule base, content matching, and tag output. The rule base pre-defines mapping rules between keywords and tags (e.g., "target X" corresponds to the tag "target name"). Keywords are identified in the input content based on string matching, and finally, the rule base is invoked to output the corresponding tags based on the matched keywords.

[0004] However, with the continuous innovation of medical technology, new professional terms for diseases and drugs are constantly emerging, and medical texts contain a large number of professional terms with multiple meanings (such as "cardiovascular" which may refer to a disease or department in different contexts). Existing technologies rely on preset string matching rules, which will lead to an inability to accurately understand the actual semantics of words in different contexts, resulting in confusion in label recognition and poor label-based retrieval performance. Summary of the Invention

[0005] This application provides a method, apparatus, medium, and program product for tagging text content, which improves the accuracy of converting text content into tag processing and achieves accurate tag recognition of professional text content.

[0006] Firstly, this application provides a method for tagging text content. The method includes: upon receiving source data text to be processed, performing word segmentation on the source data text to obtain candidate words and context data of the candidate words; inputting the candidate words and context data into a preset tagging model to obtain a preliminary category of the candidate words; determining a target knowledge database corresponding to the preliminary category, and vectorizing the candidate tags in the target knowledge database to obtain candidate tag vectors; vectorizing the candidate words to obtain candidate word vectors; performing vector matching between the candidate word vectors and the candidate tag vectors to obtain a matching degree; and determining the candidate tag corresponding to the candidate tag vector with the highest matching degree as the tag of the candidate word.

[0007] By adopting the above technical solution, the device first performs preliminary semantic classification using a pre-set label model combined with context, and then performs vectorized precise matching within the specific knowledge database corresponding to that classification. Therefore, it can first utilize the model's contextual understanding ability to limit the true meaning of words to a clear semantic category, thereby selecting a highly relevant knowledge database. This effectively avoids the semantic confusion caused by blindly searching the global knowledge base, solves the obstacle of traditional methods in distinguishing polysemous words in professional texts, and thus achieves accurate text labeling, improving the accuracy and reliability of label recognition.

[0008] In conjunction with some embodiments of the first aspect, in some embodiments, the method further includes: determining a pending tag type in a preset tag type group that does not match the candidate tag; the preset tag type group includes a preliminary category and other tag types; inputting the candidate word, the context data, and the pending tag type into a preset tag model to obtain the tag to which the candidate word belongs and the matching degree; determining the tag to which the candidate word belongs with the highest matching degree as the tag of the corresponding pending tag type; and summarizing all tags to obtain a tag group.

[0009] By adopting the above technical solution, after completing the first round of tag matching, the device actively identifies undefined tag types not yet covered in the preset tag type group, and then performs targeted and directional supplementary extraction by calling the preset tag model again. This enables multi-dimensional and in-depth analysis of the text content. This mechanism effectively solves the technical problem that relying solely on candidate word matching may lead to an incomplete tag system, allowing for multi-faceted capture of text information. It not only focuses on the accuracy of individual words but also on the systematic completeness of the final tag set, ensuring that the generated tag set more comprehensively reflects the connotation of the source data text.

[0010] In conjunction with some embodiments of the first aspect, in some embodiments, the method further includes: performing hierarchical mapping in a tag library according to the tag type to which the candidate word belongs, to obtain a tag group with a mapping relationship.

[0011] By adopting the above technical solution, after generating tags, the device further utilizes a hierarchical tag library to trace and locate each tag, thereby establishing an inherent mapping relationship between tags. This processing step transforms the originally flat and isolated tag list into a structured knowledge system. It not only clarifies the specific location of each tag in the professional domain knowledge graph but also reveals the subordinate and related relationships between tags. This greatly enhances the semantic value and usability of the tagging results, providing a solid structured foundation for subsequent advanced data analysis, knowledge reasoning, or precise retrieval applications, and improving the depth and quality of the tagging results.

[0012] In conjunction with some embodiments of the first aspect, in some embodiments, after the step of performing vector matching between the candidate word vector and the candidate tag vector to obtain the matching degree, the method further includes: forming a preliminary matching result set by assembling a number of candidate tag vectors whose matching degree is higher than a preset retrieval threshold; inputting the candidate word, the context data, and the preliminary matching result set into a preset Rerank model to obtain a reranked matching result set, and selecting the candidate tag with the highest ranking from the reranked matching result set as the tag of the candidate word.

[0013] By employing the above technical solution, the device quickly filters out a preliminary matching result set from a massive number of candidate labels through vector matching. Then, using a pre-defined Rerank model with stronger expressive power, it performs in-depth analysis and re-ranking of this small result set in conjunction with the context. The Rerank model can more precisely capture the deep semantic relationships between candidate words and candidate labels, effectively distinguishing those interference items that are close in distance in the vector space but have different actual meanings. This approach balances efficiency and accuracy, significantly improving the accuracy of the final label selection, especially in complex scenarios with a large number of similar or synonymous candidate labels.

[0014] In conjunction with some embodiments of the first aspect, in some embodiments, before the step of performing word segmentation on the source data text to obtain candidate words and context data of the candidate words upon receiving the source data text to be processed, the method further includes: when a data change event of the source data platform is detected, extracting the change type identifier and the changed data content of the data change event; determining a corresponding data processing strategy based on the change type identifier, and converting the changed data content into the source data text according to the data processing strategy.

[0015] By adopting the above technical solution, the device constructs an intelligent data access and preprocessing layer at the forefront of the tagging process. By proactively monitoring data change events on the source platform and matching different data processing strategies based on the change type identifier, it achieves automated and standardized conversion of heterogeneous and dynamic data. This mechanism allows the entire tagging method to flexibly adapt to various data sources and business scenarios without requiring customized development for each data format. This ensures that the source data text fed into the core tagging process is consistent and of high quality, thereby enhancing the stability of the processing flow and the accuracy of the final result.

[0016] In conjunction with some embodiments of the first aspect, in some embodiments, after the step of inputting the candidate word and the context data into a preset label model to obtain the preliminary category of the candidate word, the method further includes: if it is determined that the number of subcategories included in the preliminary category is higher than a preset threshold, obtaining the target category information corresponding to the subcategory from a pre-built local label library; the local label library contains standard labels that have been manually confirmed during the historical labeling process, as well as the category and category information corresponding to the standard labels; the target category information includes the category definition and core feature word set of the subcategory; based on the target category information, calculating the semantic similarity between the context data and the target category information to obtain the category matching score of the subcategory; and determining the subcategory with the highest category matching score as the updated preliminary category.

[0017] By adopting the above technical solution, the device employs a category refinement mechanism based on a local high-quality tag library to address scenarios where the preset label model may output broad initial categories. When the initial category contains too many subcategories, a second, more precise discrimination can be performed by calculating the semantic similarity between the context data and the definitions of each subcategory and the core feature word set. This step utilizes standard label data that has been manually verified, essentially introducing expert knowledge to calibrate and refine the model's initial results. It effectively solves the problem of insufficient model classification granularity, ensuring more accurate selection of the subsequent target knowledge database, thereby improving the accuracy of the entire labeling process from specific points to a broader scope.

[0018] In conjunction with some embodiments of the first aspect, in some embodiments, before the step of performing word segmentation on the source data text to obtain candidate words and context data of the candidate words upon receiving the source data text to be processed, the method further includes: counting the number of source data texts received within a preset time window and calculating the data processing frequency; storing the source data text in a processing buffer when the data processing frequency exceeds a preset processing threshold; performing similarity clustering on the source data text in the processing buffer when the number of source data texts in the processing buffer reaches a preset processing scale, and obtaining the similarity of the source data texts in the processing buffer; identifying source data texts with similarity higher than a preset clustering threshold as units to be processed, and identifying one source data text in the unit to be processed as the source data text to be processed.

[0019] By adopting the above technical solution, the device introduces traffic shaping and batch processing mechanisms for high-concurrency scenarios. It temporarily stores frequently arriving source data text in a buffer, and after reaching a certain processing scale, performs similarity clustering to group a large number of repetitive or highly similar texts into units to be processed. Finally, only one representative is labeled. This greatly avoids redundant calculations of essentially identical content and effectively solves the problem of wasted computing resources under high data throughput. Without sacrificing labeling effectiveness, it improves the system's processing efficiency and carrying capacity, enhancing the scalability and cost-effectiveness of the solution in practical applications.

[0020] In a second aspect, this application provides an apparatus comprising: one or more processors and a memory; the memory being coupled to the one or more processors, the memory being used to store computer program code including computer instructions, the one or more processors invoking the computer instructions to cause the apparatus to perform the method as described in the first aspect and any possible implementation thereof.

[0021] Thirdly, this application provides a computer program product containing instructions that, when run on a device, cause the device to perform the method described in the first aspect and any possible implementation thereof.

[0022] Fourthly, this application provides a computer-readable storage medium including instructions that, when executed on a device, cause the device to perform the method described in the first aspect and any possible implementation thereof.

[0023] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0024] 1. By employing a technique that first uses a pre-defined model to perform preliminary semantic classification of candidate words in conjunction with the context, and then performs vectorized precise matching within the target knowledge database defined by the classification, the technical problem of low labeling accuracy caused by blind matching in the global knowledge base is effectively solved. This achieves accurate label recognition of professional text content and improves the accuracy of the labels.

[0025] 2. By adopting the technical means of actively verifying the coverage of preset tag types and selectively extracting missing types after the initial tag extraction is completed, the technical problem of existing technologies that only focus on matching a single word and ignore the integrity of the tag system, resulting in a single dimension of tagging results and incomplete information coverage, is effectively solved. This enables multi-dimensional and all-round tagging of text content and ensures the system integrity of the final tag group.

[0026] 3. By employing a technique that utilizes expert knowledge from a local high-quality tag library to perform secondary discrimination and refinement of categories through semantic similarity calculation when the initial categories output by the model are too broad, the technical problem of the pre-set model outputting initial categories that are too coarse and inaccurate when facing complex text in the existing technology is effectively solved. This achieves precise refinement of the initial categories and enhances the accuracy of the entire tagging process. Attached Figure Description

[0027] Figure 1 This is a flowchart illustrating a method for tagging text content in an embodiment of this application;

[0028] Figure 2 This is another flowchart illustrating a text content tagging method in an embodiment of this application;

[0029] Figure 3 This is a schematic diagram of an exemplary hardware structure of the device in the embodiments of this application. Detailed Implementation

[0030] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to and includes any or all possible combinations of one or more of the listed items.

[0031] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0032] Please see Figure 1 This is a flowchart illustrating a text content tagging method in an embodiment of this application.

[0033] S101. Upon receiving the source data text to be processed, perform word segmentation on the source data text to obtain candidate words and their context data.

[0034] Among them, the source data text to be processed represents the original text content that needs to be tagged, which can be medical reports, academic papers or other professional text materials; word segmentation refers to the process of dividing continuous text into meaningful basic units (words); context data refers to the text content before and after the candidate words in the original text.

[0035] After receiving the source text data to be processed, the device performs word segmentation, dividing the continuous, unstructured text string into a series of independent word sequences with basic semantic units, i.e., candidate words. Simultaneously, for each generated candidate word, the device determines and extracts its contextual data within the original text. Contextual data refers to other words, phrases, or sentence fragments that appear around a specific candidate word. For example, contextual data can be defined as the N words preceding and following the candidate word (i.e., a fixed-size sliding window), or the entire sentence containing the candidate word.

[0036] It is understandable that the source data text can be diverse in its origin, format, and language. For example, it can be a plain text file, a database field, web page content, or a data stream transmitted via an API interface. The granularity of word segmentation can also be adjusted according to the specific application scenario. For example, it can be divided into coarse-grained word segmentation (which tends to combine words into longer terms) or fine-grained word segmentation (which tends to segment words into smaller pieces). No specific limitation is made here.

[0037] In some embodiments, before performing general word segmentation, the device loads a domain-specific entity dictionary and uses the entity dictionary to pre-scan the source data text to identify and mark all entity strings that completely match the dictionary entries. After marking, the device treats these marked entity strings as a whole as an independent, indivisible processing unit and performs general word segmentation on the remaining part of the text except for these marked entities.

[0038] In some embodiments, the device acquires data change events from the source data platform via API interfaces, database triggers, or message queues, based on its connection to the source data platform. When a data change event is detected, the device captures the event and extracts two elements from the event information: a change type identifier and changed data content. The change type identifier typically includes types such as add, modify, and delete, indicating the nature of the data change; the changed data content contains the specific changed data fields and values. After acquiring this information, the device matches corresponding processing strategies for different change types according to a pre-configured data processing strategy mapping table. For example, for add types, a complete extraction strategy may be used; for modify types, only the changed fields may be extracted; for delete types, a deletion identifier may be recorded, etc. Then, according to the determined processing strategy, the device performs necessary transformation processing on the changed data content, such as format conversion, field extraction, and data cleaning, ultimately generating standardized source data text.

[0039] In some embodiments, the device performs real-time statistics on the received source data text, calculating the number of texts received within a preset time window (e.g., 1 minute or 5 minutes) to obtain the current data processing frequency. The device compares this frequency with a preset processing threshold to determine whether the current data flow exceeds the system's direct processing capacity. If the data processing frequency exceeds the preset threshold, newly received source data text is temporarily stored in a processing buffer. The device continuously monitors the accumulated number of texts in the processing buffer, and triggers a batch processing flow when the number reaches a preset processing scale. Before batch processing, the device performs similarity calculation and cluster analysis on the texts in the buffer, using algorithms such as TF-IDF, Word2Vec, or BERT to calculate the similarity between texts, and grouping texts with similarity higher than a preset clustering threshold (e.g., 0.8 or 0.85) into the same processing unit. For each formed processing unit, the device selects a representative text (e.g., the earliest received or the most complete) as the source data text to be processed into subsequent word segmentation and tagging processes, avoiding duplicate processing of highly similar content.

[0040] S102. Input the candidate words and context data into the preset label model to obtain the preliminary category of the candidate words.

[0041] The preset labeling model refers to a large-scale language model (LLM) used to understand text semantics and perform classification, such as GPT-4 or an equivalent open-source model. Candidate words are word units to be labeled, obtained through word segmentation. The device provides the generated candidate words and their associated contextual data as a combined input to the preset labeling model. This model analyzes the candidate words themselves and their context to predict and output a preliminary category to which the candidate word is most likely to belong. The preliminary category represents the model's coarse-grained semantic classification result for the candidate words, such as predefined categories like disease name, drug name, examination item, department, or treatment method. For example, for the candidate word "cardiovascular," by analyzing its contextual data (such as "cardiovascular internal medicine" or "cardiovascular disease"), the labeling model can determine whether its preliminary category in the current context should be "department" or "disease."

[0042] Specifically, the preset labeling model is a Large Language Model (LLM). The device guides this LLM with prompts, enabling it to focus on a specific classification task. These prompts define the model's role (e.g., a senior medical expert), clarify the target category system (e.g., disease, drug, examination, department, treatment method), and specify in detail the decision rules, output format, and handling of special cases for each category. After receiving complete input containing context, candidate words, and prompts, the LLM performs deep semantic analysis on the entire text, thereby outputting one or more preliminary categories to which the candidate words belong.

[0043] In some embodiments, the preliminary classification of candidate words can be achieved in several ways: Optionally, the device can employ a zero-shot or few-shot inference method based on cue engineering, select a large language model with general strong semantic understanding capabilities, and dynamically construct structured cue text, including role setting (you are a professional medical information analyst), task description (based on the provided context, determine which predefined category the specified candidate word belongs to), category definition (list all possible preliminary categories and their definitions), input format specification (indicate the position of candidate words and context data in the input), and output format constraints (requiring the model to return results in strict JSON format); then this cue word is compared with specific candidate words. The input is concatenated with the context data and sent to a large language model via API calls. Optionally, the device can employ a domain model fine-tuning approach, selecting a medium-sized pre-trained language model (such as BERT, RoBERTa, or a smaller LLM) and fine-tuning it on a domain-specific labeled dataset. This dataset consists of numerous (candidate words, context data, preliminary category) triples. By training on this supervised dataset, the model parameters are tuned to specifically optimize performance for this classification task. After training, this fine-tuned model becomes a dedicated pre-labeled model. During inference, the device directly inputs the candidate words and context data into the model, which efficiently outputs the predicted preliminary category. It is understood that other deep learning-based semantic analysis methods can also be used to achieve this step; this is not limited here.

[0044] S103. Determine the target knowledge database corresponding to the preliminary category, and vectorize the candidate tags in the target knowledge database to obtain candidate tag vectors.

[0045] The target knowledge database refers to the structured knowledge base of the professional field corresponding to the preliminary category, which contains the standard terminology and normative expressions under that category.

[0046] The device selects a target knowledge database corresponding to the initial category from multiple pre-built professional knowledge databases. For example, if the initial category is a disease name, a disease knowledge database is selected; if it is a drug name, a drug knowledge database is selected. These target knowledge databases store standard terms and related information that have been certified by professional institutions or are generally recognized in the industry. The device extracts all possible candidate labels from the selected target knowledge database and uses a pre-trained text vectorization model (such as Word2Vec, BERT, etc.) to convert these candidate labels into high-dimensional numerical vectors.

[0047] Understandably, the device will access and integrate multiple authoritative data sources. In the medical field, this might include the complete import of the World Health Organization's ICD-10 or ICD-11 coding system (including codes, official names, common aliases, etc.). Simultaneously, the device will parse the publicly available drug instruction manual database released by the National Medical Products Administration, extracting and storing key fields such as the generic name, brand name, indications, and contraindications for each drug through structured information extraction technology. Furthermore, during vectorized matching, standardized terms and descriptions in the knowledge base are used as the corpus for generating baseline vectors, ensuring that the matching target is a standard concept within the domain rather than vague text; during hierarchical mapping, the inherent hierarchical structure in the knowledge base (such as the chapter structure of ICD coding) directly provides the path information required for mapping.

[0048] In some embodiments, the determination of the target knowledge database and the vectorization of candidate labels can be achieved in several ways: Optionally, the device can adopt a rule-based knowledge base mapping method. First, a preliminary mapping table from categories to knowledge bases is established, such as {disease name: disease knowledge base, drug name: drug knowledge base, ...}. Then, based on the preliminary category of candidate words, the corresponding target knowledge database is directly retrieved from the table. Next, all standard terms are extracted from the target knowledge database as candidate labels. Finally, a pre-trained general word embedding model (such as Word2Vec or GloVe) is used to convert each candidate label into a fixed-dimensional vector representation, and the vectors are normalized for subsequent calculations. Optionally, the device can adopt a semantic-based knowledge base retrieval method. First, the semantic similarity between the preliminary categories and the domain descriptions of each knowledge base is calculated. Then, the knowledge base with the highest similarity is selected as the target knowledge database. Next, based on the characteristics of candidate words, a subset of standard terms with high semantic relevance are retrieved from the target knowledge database as candidate labels, rather than all standard terms. Finally, a context-sensitive language model (such as BERT, RoBERTa, etc.) is used to vectorize these candidate labels, generating a high-dimensional vector representation that can capture deep semantic information. No limitations are imposed here.

[0049] Understandably, the target knowledge database can be a locally stored structured database or an external professional knowledge service accessed via an API.

[0050] S104. Vectorize the candidate words to obtain candidate word vectors.

[0051] The device converts candidate words into vector representations using the same vectorization model as candidate labels, ensuring that candidate word vectors and candidate label vectors are in the same vector space.

[0052] S105. Perform vector matching between the candidate word vectors and the candidate label vectors to obtain the matching degree.

[0053] Vector matching refers to the process of calculating the similarity between two vectors in a vector space.

[0054] The device performs a one-to-many similarity calculation between the candidate word vectors and all candidate label vectors.

[0055] Specifically, the device can use cosine similarity as the primary similarity metric, calculating the cosine of the angle between the candidate word vector and each candidate tag vector. The cosine similarity formula is the dot product of two vectors divided by the product of their respective magnitudes. When two vectors are identical, the cosine similarity is 1; when two vectors are in completely opposite directions, the cosine similarity is -1; and when two vectors are orthogonal, the cosine similarity is 0.

[0056] Understandably, the device can also calculate similarity by calculating other distance metrics such as Euclidean distance and Manhattan distance, or combine multiple similarity indicators to obtain a comprehensive matching score through a weighted method, which is not limited here.

[0057] S106. The candidate label corresponding to the candidate label vector with the highest matching degree is determined as the label of the candidate word.

[0058] The device sorts all calculated match scores and obtains the candidate tag with the highest match score. The device checks whether the highest match score exceeds a preset acceptable threshold (usually set between 0.7 and 0.9). If the highest match score exceeds the threshold, the corresponding candidate tag is determined as the final tag for that candidate word; if the highest match score does not exceed the threshold, it may indicate that the candidate word does not have a suitable standard expression in the knowledge base, and the device may retain the original word or mark it as an unknown category.

[0059] In some embodiments, the device first filters all matching scores obtained from vector matching, selecting candidate tags with matching scores higher than a preset retrieval threshold to form a preliminary matching result set. The device then inputs the candidate words, their contextual data (such as text fragments before and after the candidate words, document topics, etc.), and the preliminary matching result set into a pre-trained Rerank model. The Rerank model is typically based on a deep learning architecture, such as BERT or RoBERTa, and is fine-tuned for a specific task, comprehensively considering text semantics, contextual relationships, and domain knowledge. The model analyzes the specific meaning of the candidate words in the context, evaluates the degree of matching between each candidate tag and the context, and generates new matching scores. Based on these scores, the device reorders the candidate tags in the preliminary matching result set, forming a reordered matching result set. Finally, the device selects the candidate tag with the highest score from the reordered matching result set as the final tag for the candidate word.

[0060] In this embodiment, a hierarchical label matching mechanism based on semantic understanding is employed. This mechanism involves first performing semantic classification using a large model and then directing the matching to a professional knowledge base for precise matching. Therefore, the system can accurately understand the professional meaning of words in specific contexts and perform label matching within the corresponding professional fields. This effectively solves the semantic confusion problem in traditional technologies, thereby improving the standardization processing capability in the labeling of professional texts such as medical texts.

[0061] In the above embodiments, the text tagging device can achieve high-precision tagging of specific words in text by fusing technical features that perform preliminary semantic classification followed by targeted vector matching, effectively solving the problem of semantic confusion. However, in practical applications, when executing the above method, which mainly focuses on the accurate mapping of identified candidate words, it may suffer from insufficient tag coverage when processing complex text. This method can solve this technical problem by fusing technical features that perform integrity verification and targeted supplementation of the tagging results, ensuring that the final generated tag set is more complete.

[0062] Please see Figure 2 This is another flowchart illustrating a text content tagging method in an embodiment of this application.

[0063] S201. Upon receiving the source data text to be processed, perform word segmentation on the source data text to obtain candidate words and their context data.

[0064] S202. Input the candidate words and context data into the preset label model to obtain the preliminary category of the candidate words.

[0065] The device takes candidate words and their contextual data as input and passes them to a pre-trained labeling model. This pre-trained labeling model is typically a multi-classification model, possibly built based on machine learning algorithms (such as support vector machines or random forests) or deep learning architectures (such as pre-trained language models like BERT and RoBERTa). The model comprehensively analyzes the features of the candidate words themselves (such as word form, word frequency, and part of speech) and the contextual information provided by the contextual data (such as surrounding words, syntactic structure, and semantic relationships), calculating the probability distribution of the word belonging to various predefined categories. Based on the probability distribution output by the model, the device determines one or more categories with the highest probabilities as the initial categories of the candidate words. These initial categories may be coarse-grained broad categories (such as diseases, drugs, and symptoms) or may include finer-grained subcategories (such as cardiovascular diseases, antibiotics, and neurological symptoms).

[0066] S203. If the number of subcategories included in the preliminary category is higher than a preset threshold, obtain the target category information corresponding to the subcategories from the pre-built local tag library.

[0067] First, the device checks the number of subcategories in the initial category. It compares this number to a pre-set threshold, typically determined by system processing power and label accuracy requirements; this threshold might be 3, 5, or another appropriate value. If the number of subcategories exceeds the threshold, it indicates high ambiguity or complexity in the initial category. In this case, the device accesses a pre-built local label library, a structured knowledge base containing various categories, subcategories, their corresponding standardized descriptions, key features, and examples. Based on the subcategory identifiers in the initial category, the device retrieves the corresponding target category information from the local label library, including the category's standard definition, key feature descriptions, typical examples, and semantic feature vectors.

[0068] S204. Based on the target category information, calculate the semantic similarity between the context data and the target category information to obtain the category matching score of the subcategory.

[0069] The device first preprocesses the contextual data, including text cleaning, word segmentation, and stop word removal, to extract key information with semantic value. Then, it converts the processed contextual data and target category information into semantic vector representations, which can be done using pre-trained language models (such as BERT or Word2Vec). Next, the device calculates the semantic similarity between these two sets of vectors, using common methods such as cosine similarity, Euclidean distance, or dot product. During the calculation, the device considers the weights of different semantic features, such as keyword matching, syntactic structure similarity, and topic relevance. For each subcategory, the device generates a category matching score between 0 and 1; a higher score indicates a stronger semantic match between the subcategory and the context.

[0070] In some embodiments, the device may employ a deep learning-based semantic matching method. First, it encodes the text using a pre-trained BERT model; then, it extracts deep semantic features through a multi-layer neural network; next, it calculates the similarity score between feature vectors; and finally, it normalizes the score to the 0-1 range using a softmax function. The device may also combine knowledge graphs for semantic reasoning, mapping the text to entities and relationships in the knowledge graph; then, it calculates the shortest path distance between entities; next, it combines the semantic relationship strength along the path; and finally, it converts the results into a standardized matching score.

[0071] In other embodiments, the device can employ a context-focused matching method based on differential semantic probes. First, it performs a horizontal comparison of multiple target category information, extracting a unique differential feature set for each category. Then, it uses these differential features as probes to apply targeted attention weighting to the context data, generating multiple focused context vectors. Next, it calculates the basic similarity score between each focused vector and its corresponding target category information. Finally, it combines the probe's hit rate in the context to calibrate the score, obtaining the final category matching score. The generation of the differential feature set can be achieved using a mutually exclusive information entropy algorithm, which identifies semantic units that frequently appear in one category but rarely in others. For example, for the puncture biopsy category, differential features such as {tissue, pathology, cytology} might be extracted; while for the puncture drainage category, differential features such as {effusion, abscess, decompression} might be extracted. In the context attention weighting stage, the device uses the differential feature set of each category as a query vector and calculates attention scores with each word in the context, thereby generating a context representation biased towards a specific category perspective. It is suitable for processing complex medical texts containing multiple similar concepts, and improves the accuracy of disambiguation by capturing subtle semantic differences.

[0072] S205. Determine the subcategory with the highest category matching score as the updated preliminary category.

[0073] S206. Determine the target knowledge database corresponding to the preliminary category, and vectorize the candidate tags in the target knowledge database to obtain candidate tag vectors.

[0074] S207. Vectorize the candidate words to obtain candidate word vectors.

[0075] S208. Perform vector matching between the candidate word vector and the candidate label vector to obtain the matching degree.

[0076] S209. The candidate label corresponding to the candidate label vector with the highest matching degree is determined as the label of the candidate word.

[0077] Steps S206 to S209 are similar to steps S103 to S106, and will not be described in detail here.

[0078] S210. Determine the pending tag types in the preset tag type group that do not have matching candidate tags.

[0079] Among them, the preset tag type group represents the set of standardized tag types predefined by the system.

[0080] After completing the candidate tag matching process, the device first acquires all tag types from a preset tag type group. These types typically include multiple dimensions such as topic type, attribute type, and relationship type. Then, the device iterates through each tag type, checking whether a matching candidate tag has been found in the previous steps. For each tag type, the device maintains a matching status flag to record whether the type has been successfully matched. If a tag type's matching status is unmatched, it is marked as a pending tag type. This process identifies which preset tag types have not yet found suitable matching tags.

[0081] In some embodiments, the determination of pending label types can be achieved in several ways: Optionally, the device can employ a fast matching method based on a mapping table, first establishing a mapping relationship table between preset types and candidate labels; then traversing the preset types to check the mapping status; next, marking types without established mapping relationships as pending; and finally outputting a list of pending types. Optionally, the device can employ a dynamic matching method based on semantic similarity, first calculating the semantic similarity matrix between preset types and candidate labels; then setting a similarity threshold for filtering; next, identifying preset types whose similarity does not meet the threshold; and finally determining these types as pending types. It is understood that other methods for determining pending types can also be used, such as rule-based logical judgment, statistical probability evaluation, etc., which are not limited here.

[0082] S211. Input the candidate words, context data, and undetermined label types into the preset label model to obtain the label to which the candidate words belong and the matching degree.

[0083] Once the potential tag types are determined, the text tagging device performs deep semantic understanding and tag matching using a pre-defined tag model. Specifically, the device first constructs an input prompt containing candidate words, contextual data, and a structured description of the potential tag type. Then, the device inputs this information into the pre-defined tag model for processing. The model analyzes the relationship between the text content and various tags based on deep semantic understanding. During processing, the model considers the professional characteristics of medical texts, including a multi-dimensional tag system encompassing diseases, medications, examinations, and departments. For each possible tag type, the model calculates a matching score, reflecting the semantic relevance of the tag to the text content. Finally, the device standardizes the model output, converting it into a structured JSON format for easy use in subsequent applications.

[0084] Specifically, the device can adopt a precise matching method based on prompt engineering. First, a prompt template containing role definition, task description and constraints is designed; then, the text content and tag information are filled into the template; next, semantic understanding and matching are performed through a large model; finally, the output results are standardized.

[0085] S212. Determine the candidate word tag type with the highest matching degree as the tag of the corresponding pending tag type.

[0086] For each candidate label type, the device first sorts the matching degree of all candidate words and selects the candidate word label type with the highest matching degree that exceeds a preset threshold as the final label for that type. If there are multiple candidate words with similar matching degrees, the device will further consider contextual information and domain rules for filtering.

[0087] S213. Summarize all tags to obtain a tag group.

[0088] After identifying the tags for each pending tag type, the device integrates these newly identified tags with the previously identified tags to form a complete tag group. During the aggregation process, the device maintains the hierarchical relationship between tags and ensures the structural integrity and semantic consistency of the tag group.

[0089] In some embodiments, the device iterates through each tag in the tag group, using the tag name as the search key to query a pre-defined tag library. This tag library is typically designed as a tree or graph structure, storing standardized classification systems for various fields (such as diseases and drugs). For example, for the tag "aspirin," the query operation finds the corresponding node in the tag library and backtracks to extract all its parent nodes, ultimately obtaining a complete hierarchical path, such as "chemical drugs," "antipyretic analgesics," "anti-inflammatory drugs," and "aspirin." If a thesaurus exists in the tag library, the device will first standardize the tags, converting them into standard terms in the library before mapping them. For tags that cannot be directly found in the tag library, the device can place them under an "uncategorized" default node or attach them to the root node of the corresponding major category based on their initial category. Finally, the device integrates all tags and their corresponding hierarchical path information into a new, hierarchical tag group, such as a JSON object, thereby providing richer structured information for subsequent precise retrieval and data analysis applications.

[0090] By employing a technique that identifies unmatched tag types within a pre-defined tag type group after the first round of tag matching and then uses a pre-defined tag model for targeted supplementation, secondary deep semantic analysis can uncover and fill in tag dimensions that may have been missed in the initial matching round. This effectively solves the technical problem that relying solely on knowledge base matching makes it difficult to cover all tag dimensions, leading to an incomplete tag system. This, in turn, enables multi-dimensional and comprehensive analysis of text content, ensuring the completeness and richness of the final generated tag group.

[0091] The exemplary device 300 provided in the embodiments of this application is described below. Figure 3This is an exemplary hardware structure diagram of the device 300 provided in the embodiments of this application.

[0092] In some embodiments, the device 300 is a computer device or includes a computer device. The computer device includes a processor, memory, and a network interface connected via a system bus. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database of the computer device stores data. The network interface of the computer device is used to communicate with other external terminals or servers via a network connection. In some embodiments, the network interface can be a wired network interface; in some embodiments, the network interface can also be a wireless network interface. When the computer program is executed by the processor, it implements the methods in the embodiments of this application.

[0093] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0094] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0095] As used in the above embodiments, depending on the context, the term "when..." can be interpreted as meaning "if...", "after...", "in response to determining...", or "in response to detecting...". Similarly, depending on the context, the phrase "when determining..." or "if (the stated condition or event) is interpreted as meaning "if determining...", "in response to determining...", "when (the stated condition or event) is detected", or "in response to detecting (the stated condition or event)".

[0096] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0097] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A method of tagging text content, characterized by, The method includes: Upon receiving the source data text to be processed, the source data text is segmented to obtain candidate words and their context data. The candidate words and the context data are input into a preset labeling model to obtain the preliminary category of the candidate words; If the number of subcategories included in the preliminary category is higher than a preset threshold, the target category information corresponding to the subcategory is obtained from a pre-built local tag library; the local tag library contains standard tags that have been manually confirmed during the historical tagging process, as well as the category and category information corresponding to the standard tags; the target category information includes the category definition and core feature word set of the subcategory; Based on the target category information, the target category information is compared horizontally, and a unique differential feature set for each subcategory is extracted. The differential feature set includes semantic units that appear more frequently in one subcategory but less frequently in other subcategory information than the preset value. Attention weighting is applied to the context data based on the differential feature set to generate a focus context vector corresponding to the sub-category; Calculate the semantic similarity between the focusing context vector and the corresponding sub-category semantic vector to obtain the category matching score of the sub-category; The subcategory with the highest category matching score is determined as the updated initial category; Determine the target knowledge database corresponding to the preliminary category, and vectorize the candidate tags in the target knowledge database to obtain candidate tag vectors; The candidate words are vectorized to obtain candidate word vectors; The candidate word vectors are matched with the candidate label vectors to obtain the matching degree. The candidate label corresponding to the candidate label vector with the highest matching degree is determined as the label of the candidate word.

2. The method of claim 1, wherein, The method further includes: A pending tag type is determined that does not match the candidate tag in the preset tag type group; the preset tag type group includes preliminary categories and other tag types; Input the candidate words, the context data, and the undetermined label type into a preset label model to obtain the label and matching degree of the candidate words; The tag to which the candidate word with the highest matching degree belongs is determined as the tag of the corresponding pending tag type; Summarize all tags to obtain a tag group.

3. The method according to claim 2, characterized in that, The method further includes: Based on the tag type to which the candidate words belong, a hierarchical mapping is performed in the tag library to obtain a tag group with a mapping relationship.

4. The method according to claim 1, characterized in that, After the step of performing vector matching between the candidate word vector and the candidate label vector to obtain the matching degree, the method further includes: A preliminary matching result set is formed by combining several candidate tag vectors with a matching degree higher than a preset retrieval threshold; The candidate words, the context data, and the preliminary matching result set are input into a preset Rerank model to obtain a reranked matching result set. From the reranked matching result set, the candidate label with the highest ranking is selected as the label of the candidate word.

5. The method according to claim 1, characterized in that, Before the step of performing word segmentation on the source data text upon receiving it to obtain candidate words and context data of the candidate words, the method further includes: When a data change event is detected on the source data platform, the change type identifier and the changed data content of the data change event are extracted; The corresponding data processing strategy is determined based on the change type identifier, and the changed data content is converted into the source data text according to the data processing strategy.

6. The method according to claim 1, characterized in that, Before the step of performing word segmentation on the source data text upon receiving it to obtain candidate words and context data of the candidate words, the method further includes: The number of source data texts received within a preset time window is counted, and the data processing frequency is calculated. If the data processing frequency exceeds a preset processing threshold, the source data text is stored in the processing buffer. When the number of source data texts in the processing buffer reaches a preset processing scale, similarity clustering is performed on the source data texts in the processing buffer to obtain the similarity of the source data texts in the processing buffer. Source data texts with similarity higher than a preset clustering threshold are identified as units to be processed, and one source data text in the unit to be processed is identified as the source data text to be processed.

7. A device, characterized in that, The device includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, the one or more processors invoking the computer instructions to cause the device to perform the method as described in any one of claims 1-6.

8. A computer program product containing instructions, characterized in that, When the computer program product is run on the device, the device causes the device to perform the method as described in any one of claims 1-6.

9. A computer-readable storage medium comprising instructions, characterized in that, When the instructions are executed on the device, the device causes the device to perform the method as described in any one of claims 1-6.

Images