Tag system construction method based on large language model
Through the large language model combining deep learning and natural language processing methods, the tag generation and update are automated, which solves the problems of subjectivity, update lag and poor scalability of traditional tag system construction, and realizes efficient and accurate tag system construction, which is suitable for Various text types and application scenarios.
Patent Information
- Application Number
- CN202410891650.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-04
- Publication Date
- 2024-11-26
AI Technical Summary
The construction of traditional label system relies on manual experience, and has problems such as strong subjectivity, lag in updates, poor scalability, lack of flexibility, limited ability to process complex texts, sparse data and high cost, making it difficult to meet the needs of modern data processing.
The tag system construction method based on large language models is adopted, combined with deep learning and natural language processing, and the tag generation and update are automated through pre-training large models, which is suitable for a variety of text types, realizing dynamic updates and personalized services.
It significantly improves the efficiency and accuracy of label system construction, enhances context understanding and generalization capabilities, reduces manual intervention, reduces costs, adapts to diversified data types and application scenarios, and improves user experience and data governance efficiency.
Smart Images

Figure CN119025672A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing technology, and in particular to a labeling system construction method based on a large language model. Background Art
[0002] Labeling text is an important text processing technique that involves extracting key information from text and using labels to represent the categories or attributes of this information. The main purpose of labeling is to make text easier to calculate, index, and represent, thereby facilitating subsequent tasks such as text analysis, information retrieval, or machine learning.
[0003] Tags are used to identify the characteristics of an object. In order to better describe the object, the characteristics of the object can be highlighted from different dimensions. By summarizing and generalizing, the main characteristics of an object can be highlighted, based on which the personality and differences of the object can be highlighted. The tag system is a system established based on multiple tags. In order to be able to systematically and normatively sort out the tags of the objects included in the entire enterprise or group, the tag system can better understand the content characteristics and user characteristics of the platform. A fuller understanding also means better satisfaction of user needs.
[0004] The significance of labeling system construction is far-reaching and extensive. It not only helps the internal information management and business process optimization of the organization, but also improves the user experience, strengthens the data-driven decision-making ability, and promotes business innovation and growth. The following are the specific significances of labeling system construction: (1) Optimizing information management. The labeling system can classify and label massive amounts of data and information in an orderly manner, making information retrieval and search more efficient. Unified labeling standards can help reduce information redundancy and confusion and improve the accuracy and consistency of information. (2) Improving user experience. For users, the labeling system can help them find content or products of interest more quickly, improving search efficiency and satisfaction. Personalized label recommendations can improve user experience and make users feel more intimate and accurate service. (3) Strengthening data-driven decision-making. The labeling system can label multi-dimensional data such as user behavior and product attributes, so as to better analyze key information such as user portraits and product features. These analysis results provide strong support for data-driven decision-making and help organizations formulate more accurate market strategies and product optimization plans. In short, labeling has a wide range of applications. For example, in information retrieval, labels can help users find relevant content faster; in recommendation systems, labels can be used for personalized recommendations; in machine learning, labels can be used as features for tasks such as classification or clustering.
[0005] Text tagging can be achieved in many ways, some common traditional methods include:
[0006] (1) Dictionary matching: Use a predefined tag vocabulary to identify tags from text by dictionary matching. This method is fast, but may misidentify some very short tags or words that are not real tags in the context.
[0007] (2) Named Entity Recognition (NER): NER is a more complex tagging method that uses machine learning models to identify specific entities in text, such as names of people, places, organizations, etc. NER models can be trained to recognize custom tags and solve some misidentification problems caused by context.
[0008] (3) Rules or templates: Extract tags from text according to specific rules or templates. This method requires manual definition of rules or templates and is suitable for some fixed-format texts.
[0009] However, the establishment of the traditional labeling system mainly relies on the experience of business personnel, which can ensure the accuracy and practicality of the labels to a certain extent, but it also has some shortcomings, which can no longer meet the needs of today's society. The shortcomings are as follows:
[0010] Highly subjective: Since it is based on human experience, it is greatly influenced by personal cognition and bias. Different people may have different understandings of the same concept, resulting in poor label consistency.
[0011] Update lag: The update of traditional labeling systems usually requires manual intervention. When new concepts or trends emerge, the system update may not keep up with the speed of change, resulting in the labeling system being unable to reflect the latest information in a timely manner.
[0012] Poor scalability: Faced with massive data and diverse information, the traditional labeling system has poor scalability and often requires a lot of manpower and time costs to add new labels.
[0013] Lack of flexibility: Once a traditional labeling system is established, it is difficult to modify and adjust. Customized labeling systems may be required for specific tasks, and traditional methods are difficult to quickly adapt to these needs.
[0014] Limited ability to process complex text: The ability to process complex text content such as implicit information, puns, slang, etc. is limited, which may lead to reduced classification accuracy.
[0015] Data sparsity problem: In the traditional labeling system, some uncommon labels may be difficult to train effective models due to insufficient sample data.
[0016] Dependence on expertise: Establishing and maintaining a labeling system usually requires the participation of domain experts, which limits the popularity and scope of application of the labeling system.
[0017] Cost issue: The establishment and maintenance of the traditional labeling system requires a lot of manpower and material resources, and the cost is relatively high.
[0018] Therefore, it is urgent to study a more advanced labeling system construction method to solve these technical problems.
[0019] It should be noted that the above introduction to the technical background is only for the convenience of providing a clear and complete description of the technical solutions of the present invention and for the convenience of understanding by those skilled in the art. It cannot be considered that the above technical solutions are well known to those skilled in the art simply because they are described in the technical background section of the present invention. Summary of the invention
[0020] The purpose of the present invention is to provide a label system construction method based on a large language model, which combines the latest progress in deep learning, natural language processing and large-scale pre-trained models. Compared with the traditional label system construction based on artificial experience and the label system construction based on traditional machine learning, the efficiency of label system construction is significantly improved, the context understanding ability is enhanced, the generalization ability is strong, and the label system can be updated in time by only fine-tuning the model or continuing to train. In addition, based on many large language models, it solves the problem that the current text classification needs to pre-establish a label system and the label system cannot fully reflect the text content classification. It is applicable to a variety of text types, solves the drawbacks of the existing technology, and has important research significance and use value.
[0021] To achieve the above object, the present invention provides the following technical solutions:
[0022] The present invention provides a method for constructing a label system based on a large language model, comprising the following steps:
[0023] Step 1: Organize and store the text to be processed, including data collection, data cleaning, text preprocessing, text library establishment, verification and testing.
[0024] Data collection includes:
[0025] Determine the data source: clarify the source of the text to be processed;
[0026] Collect text: Use crawlers or API interfaces to batch download or import target texts to local computers;
[0027] Data cleaning includes:
[0028] Remove irrelevant content: Remove content that is irrelevant to tagging, including advertisements, comments, and HTML tags;
[0029] Text formatting: unify the format of the text to be processed;
[0030] Remove duplicate text: Use hash value method to detect and remove duplicate text to avoid redundancy in subsequent processing;
[0031] The text preprocessing includes:
[0032] Word segmentation: Based on the characteristics of Chinese, use appropriate word segmentation tools to segment the text and divide the continuous text into words or phrases;
[0033] Stop word filtering: remove stop words that are not helpful for tagging;
[0034] Stemming or lemmatization: For English text, words are converted to their basic form through stemming or lemmatization to facilitate subsequent tagging operations;
[0035] Building a text library includes
[0036] Select storage method: According to the amount and size of the text to be processed, select the storage method, which can be a relational database, NoSQL database or file system;
[0037] Design the data table structure: If you use a relational database, you need to design the table structure to store text data, including text ID, content, source, and timestamp fields;
[0038] Importing data: importing the texts after data cleaning and text preprocessing into the text library in batches to ensure the integrity and accuracy of the data;
[0039] Verification and testing include:
[0040] Random sampling: randomly extract a portion of text from the text library for verification to ensure the effectiveness of data cleaning and text preprocessing;
[0041] Performance testing: testing the reading and writing performance of the text library to ensure that data can be processed efficiently in the subsequent tagging process;
[0042] Step 2: Call the pre-trained large model to use its generation capability to generate labels, including calling the pre-trained large model, preparing input data, calling the pre-trained large model to generate labels, obtaining and processing the generated results, and verifying and evaluating the label quality;
[0043] The calling of a suitable large model includes:
[0044] Based on model capabilities: select large models with strong generation capabilities as alternatives to pre-trained large models;
[0045] Domain adaptability: If the text to be processed belongs to a specific field, select a model that has been trained in the specific field as the pre-trained large model, or select a highly versatile model as an alternative to the pre-trained large model;
[0046] The preparation of input data comprises:
[0047] Text preprocessing: ensure that the text to be processed has been preprocessed, such as word segmentation and removal of stop words;
[0048] Format adjustment: adjusting the text format according to the input requirements of the pre-trained large model, such as setting an appropriate length limit and adding special tags;
[0049] The calling of the pre-trained large model to generate labels includes:
[0050] API call: Through the API interface provided by the pre-trained large model, the pre-processed text is used as input, and the pre-trained large model is called to generate labels;
[0051] Parameter setting: Set parameters according to requirements, such as the number and diversity of generated labels.
[0052] The obtaining and processing of the generated results includes:
[0053] Result analysis: parse the generated labels from the output of the pre-trained large model;
[0054] Post-processing: Post-process the generated tags, such as removing duplicates and merging similar tags;
[0055] The verification and assessment of label quality includes:
[0056] Manual verification: Randomly select a portion of the generated labels for manual verification to ensure the accuracy and relevance of the labels;
[0057] Quantitative evaluation: Use appropriate evaluation indicators to quantitatively evaluate the generated labels, including accuracy and recall;
[0058] Step 3: Post-process all labels obtained in step 2, including label cleaning, merging, and standardization.
[0059] Step 4: Cluster the labels obtained after label post-processing in step 3; the clustering process includes: selecting a suitable clustering algorithm, data preprocessing, executing the clustering algorithm, and evaluating the clustering results;
[0060] The selection of a suitable clustering algorithm includes:
[0061] Choose the appropriate clustering algorithm according to your needs;
[0062] The data preprocessing includes:
[0063] Label cleaning: Ensure that the label data has been cleaned to remove irrelevant, duplicate or obviously wrong labels;
[0064] Label encoding: convert labels into numerical data suitable for clustering algorithm processing;
[0065] The executing clustering algorithm comprises:
[0066] Set clustering parameters: according to the algorithm requirements, set appropriate parameters, such as the number of clusters and distance measurement method;
[0067] Execute clustering: call the clustering algorithm to cluster the label data;
[0068] The evaluating clustering results includes: evaluating according to the silhouette coefficient or the Calinski-Harabasz index;
[0069] Silhouette coefficient: evaluates the closeness and separation of clustering results;
[0070] Calinski-Harabasz index: evaluates the ratio of variance within a cluster to variance between clusters;
[0071] The manual verification includes:
[0072] Randomly select a part of clustering results for manual verification to ensure the rationality and accuracy of clustering;
[0073] Step 5: Use the pre-trained large model to obtain the label system for the clustering results obtained in the fourth step; including: clustering result preparation, selection of pre-trained large model, design of title generation strategy, calling the pre-trained large model to generate titles, post-processing and optimization, result output and application, manual intervention and correction;
[0074] The clustering result preparation includes:
[0075] Ensure that clustering results have been evaluated for accuracy and reasonableness;
[0076] Organize the clustering results into a format suitable for processing by a pre-trained large model, such as a list or text form;
[0077] The selection of the pre-trained large model includes:
[0078] Select a large model with strong text generation and summarization capabilities as the pre-trained large model;
[0079] Ensure that the large model has been trained and has a deep understanding of the text content;
[0080] The design title generation strategy includes:
[0081] Design a title generation strategy based on the characteristics and needs of the clustering results;
[0082] Determine the hierarchy of titles, which include first-level titles, second-level titles, and third-level titles;
[0083] Evaluate the title's conciseness, accuracy, and representativeness, and optimize it to ensure it accurately summarizes the content of the text;
[0084] The calling of the pre-trained large model to generate a title includes:
[0085] The clustering results are used as input and the pre-trained large model is called to generate titles;
[0086] According to the designed title generation strategy, guide the pre-trained large model to generate titles at different levels;
[0087] Set the number and diversity of generated titles to meet different needs;
[0088] The post-processing and optimization include:
[0089] Post-process the generated titles, such as removing duplicates and merging similar titles;
[0090] Optimize titles as needed to ensure accuracy and readability;
[0091] The result output and application include:
[0092] Output the generated titles in a preset way, such as a text file, a database or a visual interface;
[0093] Apply the generated titles to text display, directory navigation, and content recommendation scenarios to improve user experience and information acquisition efficiency;
[0094] The manual intervention and correction are manual intervention and correction of the constructed labeling system;
[0095] Step 6: Label system storage;
[0096] The label system after manual intervention and correction is saved in the database, including: selecting a suitable database system, designing the database table structure, importing label data, establishing indexes and constraints, optimizing database performance, and providing interfaces and tools;
[0097] The selection of a suitable database system includes:
[0098] Choose a suitable database system, including relational database or NoSQL database, based on the data volume of the tag system, query performance requirements, and the team's familiarity;
[0099] The design database table structure includes:
[0100] According to the hierarchical structure and attributes of the label system, a reasonable database table structure is designed, a main table is created to store the first-level title, second-level title and third-level title, and titles of different levels are associated through foreign keys;
[0101] Consider the attributes of tags, including creation time, modification time, and usage frequency, and add corresponding fields to the table structure;
[0102] The imported tag data includes:
[0103] Import the sorted, clustered and summarized label data into the database, using a batch import tool or writing a script to import;
[0104] Data cleaning and conversion: to ensure that the imported data matches the database table structure, and to perform data cleaning and conversion;
[0105] The indexing and constraint creation includes:
[0106] Create indexes for key fields based on query requirements to improve query performance;
[0107] According to business needs, set appropriate constraints, such as unique constraints or non-null constraints, to ensure the accuracy and consistency of data;
[0108] The optimization of database performance includes:
[0109] Optimize database performance based on the amount of data and the complexity of queries. Performance optimization includes adjusting cache strategies and optimizing query statements.
[0110] Perform database maintenance at preset times, including cleaning up useless data and updating statistical information to keep the database running efficiently;
[0111] The provided interfaces and tools include:
[0112] Provide an interface for other systems or applications to access the tag system. The interface is REST API or GraphQL, which facilitates data acquisition and use;
[0113] Development or integration tools, including visual interfaces or command line tools, for querying, editing and managing the tag system;
[0114] Step 7: Use the constructed labeling system to label the text library obtained in the first step; including: loading the labeling system, using the large model for preliminary label prediction, using vector similarity for label matching, using the text matching algorithm for fine-tuning, post-processing and label determination, and outputting and storing the results;
[0115] The loading label system includes:
[0116] Loading the constructed label system from the database, including the first-level title, the second-level title, and the third-level title;
[0117] Convert the label system into a format suitable for processing by the preprocessing large model, in the form of embedding vectors or label encodings;
[0118] Convert the text in the text library after the first step of preprocessing into an embedding vector;
[0119] The use of a large model for preliminary label prediction includes:
[0120] Inputting the embedding vector of the text library into the preprocessing large model, and using the generation capability of the preprocessing large model to perform preliminary label prediction;
[0121] Get the label candidate list output by the preprocessing large model as input for subsequent steps;
[0122] The label matching using vector similarity includes:
[0123] Calculate the similarity between the text embedding vector and the tag system embedding vector using the cosine similarity method;
[0124] Sort by similarity and select the most matching tag as the candidate tag for the text;
[0125] The fine adjustment using the text matching algorithm includes:
[0126] For each of the texts, searching for the most relevant tag in the tag system using a text matching algorithm;
[0127] Combined with the results of vector similarity, the candidate labels are fine-tuned to ensure their accuracy and relevance;
[0128] The post-processing and label determination include:
[0129] Post-process the matched tags, including deduplication and merging similar tags;
[0130] Label determination is to set a threshold or perform manual verification as needed to determine the final label for the text;
[0131] The result output and storage include:
[0132] Associate the labeling results with the corresponding text and output them in a structured format, which is JSON or CSV.
[0133] Store the results in a suitable storage system for subsequent analysis, retrieval, or visualization applications.
[0134] In summary, the method for constructing a label system based on a large language model is a process that combines natural language processing (NLP) and label management techniques. The present invention uses a large language model for label extraction, and processes the preprocessed text data using a pre-trained large language model (such as GPT, BERT, etc.). Utilize the text generation and understanding capabilities of the model to extract keywords or phrases related to the labels from the text. Establish a label system based on the extracted keywords or phrases. The label system should be able to clearly reflect the structure and characteristics of the data and facilitate the use and management by business personnel.
[0135] Optionally, clarifying the source of the text to be processed in the first step includes: clarifying that the source of the text to be processed is crawled from a website, extracted from a database, or obtained from a partner.
[0136] Optionally, unifying the format of the text to be processed is to convert the text to be processed into plain text format to ensure consistency in subsequent processing. The words removed in stop word filtering include: de (的), le (了), a (啊).
[0137] Optionally, label cleaning in the third step includes removing irrelevant symbols: deleting symbols that are irrelevant to the text content; removing duplicate labels: removing duplicate identical labels; removing English; removing labels that exceed a preset length.
[0138] The construction of a label system based on a large language model has several significant advantages, which make this method particularly important in modern data processing and artificial intelligence applications. The following are several main advantages of the construction of a label system based on a large language model:
[0139] Powerful semantic understanding ability:
[0140] Large language models are trained on a large amount of text data and can deeply understand the semantic information in the text.
[0141] This understanding ability enables the label system to more accurately capture the core content and intent of the text data, thereby generating more accurate and relevant labels.
[0142] Automated label generation:
[0143] The construction of traditional label systems usually requires manual participation, manually annotating data and generating labels.
[0144] However, the construction of a label system based on a large language model can automate this process, greatly reducing the burden of manual annotation and improving work efficiency.
[0145] Scalability and flexibility:
[0146] As new data is continuously generated, the labeling system needs to be constantly updated and expanded.
[0147] The construction of a labeling system based on a large language model can adapt to new data changes through simple model updates or retraining, thereby maintaining the timeliness and accuracy of the labeling system.
[0148] Adapt to various data types:
[0149] Large language models are able to process text data of various types and formats, including social media texts, news articles, user comments, etc.
[0150] Therefore, the construction of a labeling system based on a large language model can be applied to a variety of application scenarios and meet different business needs.
[0151] Efficient data processing and analysis:
[0152] By converting text data into numerical vector representations, large language models make data processing and analysis more efficient.
[0153] The construction of the labeling system can perform rapid calculations and clustering based on these numerical vectors, thereby achieving rapid processing and accurate analysis of large-scale data.
[0154] Improve user experience and personalized services:
[0155] The construction of a labeling system based on a large language model can deeply understand user needs and interests, thereby providing users with more accurate and personalized services.
[0156] For example, in a recommendation system, the construction of a tag system can help us better understand user preferences and recommend content that is more in line with their interests.
[0157] Enhanced data governance and security:
[0158] The construction of a labeling system helps to achieve effective management and classification of data and improve the efficiency and accuracy of data governance.
[0159] At the same time, through the review and optimization of labels, the compliance and security of data can be ensured and the risk of data leakage and abuse can be reduced.
[0160] In short, the construction of a labeling system based on a large language model has the advantages of powerful semantic understanding, automatic label generation, scalability and flexibility, adaptability to diverse data types, efficient data processing and analysis, improved user experience and personalized services, and enhanced data governance and security. These advantages make the construction of a labeling system based on a large language model an indispensable part of modern data processing and artificial intelligence applications.
[0161] Optionally, the process of selecting a suitable clustering algorithm includes:
[0162] Distance-based algorithms: such as K-means and hierarchical clustering, which are suitable for situations where there are a large number of labels and the relationship between labels is relatively simple;
[0163] Density-based algorithms: such as DBSCAN, which are suitable for situations where there is a certain density difference between labels;
[0164] Model-based algorithms: such as Gaussian mixture models, which are suitable for situations where the labeled data conforms to a specific distribution;
[0165] Methods based on word vectorization include stella-large-zh-v3-1792d and Dmeta-embedding-zh.
[0166] Optionally, the relational database is MySQL or PostgreSQL, and the NoSQL database is MongoDB or Cassandra.
[0167] The establishment of traditional labeling systems mainly relies on the experience of business personnel, which can ensure the accuracy and practicality of labels to a certain extent, but there are also some shortcomings, which can no longer meet the needs of today's society. The shortcomings are as follows: Strong subjectivity: Because it is based on human experience, it is greatly affected by personal cognition and prejudice. Different people may have different understandings of the same concept, resulting in poor consistency of labels. Update lag: The update of traditional labeling systems usually requires manual intervention. When new concepts or trends emerge, the update of the system may not keep up with the speed of change, resulting in the labeling system being unable to reflect the latest information in a timely manner. Poor scalability: In the face of massive data and diverse information, the scalability of traditional labeling systems is poor, and it often requires a lot of manpower and time costs to add new labels. Lack of flexibility: Once the traditional labeling system is established, it is difficult to modify and adjust. For specific tasks, a customized labeling system may be required, and traditional methods are difficult to quickly adapt to these needs. Limited ability to handle complex text: The limited ability to handle complex text content such as implicit information, puns, and slang may lead to a decrease in classification accuracy. Data sparsity problem: In the traditional labeling system, some uncommon labels may be difficult to train effective models due to insufficient sample data. Dependence on professional knowledge: The establishment and maintenance of the labeling system usually requires the participation of domain experts, which limits the popularity and application scope of the labeling system. Cost problem: The establishment and maintenance of the traditional labeling system requires a lot of manpower and material resources, and the cost is high.
[0168] The label system construction method based on a large language model provided by the present invention combines the latest progress in deep learning, natural language processing and large-scale pre-training models. Compared with the traditional label system construction based on artificial experience and the label system construction based on traditional machine learning, it has the following advantages:
[0169] Automation and efficiency: Large language models can automatically learn patterns and knowledge from large amounts of text without requiring extensive human intervention, significantly improving the efficiency of labeling system construction.
[0170] Contextual understanding ability: Large language models are able to understand complex contextual information, including context, semantics, and sentiment, thereby generating more accurate and fine-grained labels.
[0171] Generalization: Since large language models are pre-trained on a wide range of text, they usually have better generalization capabilities and are able to recognize and process unseen text data.
[0172] Dynamic Update: Large language models can quickly adapt to new data and concepts. The label system can be updated in a timely manner by simply fine-tuning the model or continuing training.
[0173] Reduced bias: By training on diverse datasets, large language models can reduce human bias and generate more objective labels.
[0174] Cost-effectiveness: Once large language models are trained, they can be used for various labeling tasks, thereby amortizing the fixed cost and being cost-effective for large-scale data processing.
[0175] Flexibility and Customization: Large language models can be fine-tuned according to the needs of specific tasks to generate customized labeling systems to meet the needs of different application scenarios.
[0176] Real-time processing capability: Large language models can quickly process large amounts of text data and achieve real-time labeling, which is very important for application scenarios that require instant feedback. BRIEF DESCRIPTION OF THE DRAWINGS
[0177] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0178] Figure 1 It is a flow chart of a method for constructing a label system based on a large language model according to an embodiment of the present invention. DETAILED DESCRIPTION
[0179] To make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be described in detail below. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other implementation methods obtained by ordinary technicians in this field without creative work belong to the scope of protection of the present invention.
[0180] Text tagging can be achieved in many ways, some common traditional methods include:
[0181] (1) Dictionary matching: Use a predefined tag vocabulary to identify tags from text by dictionary matching. This method is fast, but may misidentify some very short tags or words that are not real tags in the context.
[0182] (2) Named Entity Recognition (NER): NER is a more complex tagging method that uses machine learning models to identify specific entities in text, such as names of people, places, organizations, etc. NER models can be trained to recognize custom tags and solve some misidentification problems caused by context.
[0183] (3) Rules or templates: Extract tags from text according to specific rules or templates. This method requires manual definition of rules or templates and is suitable for some fixed-format texts.
[0184] However, the establishment of the traditional labeling system mainly relies on the experience of business personnel, which can ensure the accuracy and practicality of the labels to a certain extent, but it also has some shortcomings, which can no longer meet the needs of today's society. The shortcomings are as follows:
[0185] Highly subjective: Since it is based on human experience, it is greatly influenced by personal cognition and bias. Different people may have different understandings of the same concept, resulting in poor label consistency.
[0186] Update lag: The update of traditional labeling systems usually requires manual intervention. When new concepts or trends emerge, the system update may not keep up with the speed of change, resulting in the labeling system being unable to reflect the latest information in a timely manner.
[0187] Poor scalability: Faced with massive data and diverse information, the traditional labeling system has poor scalability and often requires a lot of manpower and time costs to add new labels.
[0188] Lack of flexibility: Once a traditional labeling system is established, it is difficult to modify and adjust. Customized labeling systems may be required for specific tasks, and traditional methods are difficult to quickly adapt to these needs.
[0189] Limited ability to process complex text: The ability to process complex text content such as implicit information, puns, slang, etc. is limited, which may lead to reduced classification accuracy.
[0190] Data sparsity problem: In the traditional labeling system, some uncommon labels may be difficult to train effective models due to insufficient sample data.
[0191] Dependence on expertise: Establishing and maintaining a labeling system usually requires the participation of domain experts, which limits the popularity and scope of application of the labeling system.
[0192] Cost issue: The establishment and maintenance of the traditional labeling system requires a lot of manpower and material resources, and the cost is relatively high.
[0193] In order to overcome these shortcomings, modern text classification systems are increasingly using technologies such as machine learning, natural language processing, and knowledge graphs to build and manage label systems in an automated and intelligent way to improve the objectivity, real-time, and scalability of label systems. Although the construction of new label systems using technologies such as machine learning, natural language processing, and knowledge graphs has brought many advantages, there are also some disadvantages and challenges:
[0194] 1. Obvious data dependence: The performance of machine learning models is highly dependent on the quality and quantity of data. If the training data is biased or insufficient, the model may produce inaccurate or biased labels.
[0195] 2. Model complexity: Modern text classification models are often very complex, and it can be difficult to understand and explain the model’s decision-making process, which leads to the so-called “black box” problem.
[0196] 3. Computing resource requirements: Training large-scale machine learning models requires a lot of computing resources, including high-performance hardware and a long training process.
[0197] 4. Cold start problem: For completely new fields or data sets, machine learning models may require a lot of time and data to train, which may encounter cold start problems in the early stages.
[0198] 5. Insufficient generalization ability: The model may overfit the training data, resulting in poor performance on new or unseen data, that is, insufficient generalization ability.
[0199] 6. Relatively poor knowledge acquisition and integration: The construction of a knowledge graph requires acquiring knowledge from multiple sources and integrating it into a consistent knowledge system. This process may face inconsistency and uncertainty in knowledge sources.
[0200] The present invention provides a label system construction method based on a large language model, which combines the latest progress in deep learning, natural language processing and large-scale pre-training models. Compared with the traditional label system construction based on artificial experience and the label system construction based on traditional machine learning, it has many advantages and solves the drawbacks of the existing technology. The specific contents are as follows:
[0201] The present invention provides a method for constructing a label system based on a large language model, such as Figure 1 As shown, the following steps are included:
[0202] Step 1: Organize and store the text to be processed, including data collection, data cleaning, text preprocessing, text library establishment, verification and testing.
[0203] Data collection includes:
[0204] Determine the data source: clarify the source of the text to be processed;
[0205] Collect text: Use crawlers or API interfaces to batch download or import target texts to local computers;
[0206] Data cleaning includes:
[0207] Remove irrelevant content: Remove content that is irrelevant to tagging, including advertisements, comments, and HTML tags;
[0208] Text formatting: unify the format of the text to be processed;
[0209] Remove duplicate text: Use hash value method to detect and remove duplicate text to avoid redundancy in subsequent processing;
[0210] Text preprocessing includes:
[0211] Word segmentation: Based on the characteristics of Chinese, use appropriate word segmentation tools to segment the text and divide the continuous text into words or phrases;
[0212] Stop word filtering: remove stop words that are not helpful for tagging;
[0213] Stemming or lemmatization: For English text, words are converted to their basic form through stemming or lemmatization to facilitate subsequent tagging operations;
[0214] Building a text library includes
[0215] Select storage method: According to the amount and size of the text to be processed, select the storage method, which can be a relational database, NoSQL database or file system;
[0216] Design the data table structure: If you use a relational database, you need to design the table structure to store text data, including text ID, content, source, and timestamp fields;
[0217] Importing data: importing the texts after data cleaning and text preprocessing into the text library in batches to ensure the integrity and accuracy of the data;
[0218] Verification and testing include:
[0219] Random sampling: randomly extract a portion of text from the text library for verification to ensure the effectiveness of data cleaning and text preprocessing;
[0220] Performance testing: testing the reading and writing performance of the text library to ensure that data can be processed efficiently in the subsequent tagging process;
[0221] Step 2: Call the pre-trained large model to use its generation capability to generate labels, including calling the pre-trained large model, preparing input data, calling the pre-trained large model to generate labels, obtaining and processing the generated results, and verifying and evaluating the label quality;
[0222] Calling suitable large models includes:
[0223] Based on model capabilities: select large models with strong generation capabilities as alternatives to pre-trained large models;
[0224] Domain adaptability: If the text to be processed belongs to a specific field, select a model that has been trained in the specific field as the pre-trained large model, or select a highly versatile model as an alternative to the pre-trained large model;
[0225] Preparing input data includes:
[0226] Text preprocessing: ensure that the text to be processed has been preprocessed, such as word segmentation and removal of stop words;
[0227] Format adjustment: adjusting the text format according to the input requirements of the pre-trained large model, such as setting an appropriate length limit and adding special tags;
[0228] Calling the pre-trained large model for label generation includes:
[0229] API call: Through the API interface provided by the pre-trained large model, the pre-processed text is used as input, and the pre-trained large model is called to generate labels;
[0230] Parameter setting: Set parameters according to requirements, such as the number and diversity of generated labels.
[0231] Obtaining and processing the generated results includes:
[0232] Result analysis: parse the generated labels from the output of the pre-trained large model;
[0233] Post-processing: Post-process the generated tags, such as removing duplicates and merging similar tags;
[0234] Verifying and assessing label quality includes:
[0235] Manual verification: Randomly select a portion of the generated labels for manual verification to ensure the accuracy and relevance of the labels;
[0236] Quantitative evaluation: Use appropriate evaluation indicators to quantitatively evaluate the generated labels, including accuracy and recall;
[0237] Step 3: Post-process all labels obtained in step 2, including label cleaning, merging, and standardization.
[0238] Step 4: Cluster the labels obtained after label post-processing in step 3; the clustering process includes: selecting a suitable clustering algorithm, data preprocessing, executing the clustering algorithm, and evaluating the clustering results;
[0239] Choosing an appropriate clustering algorithm involves:
[0240] Choose the appropriate clustering algorithm according to your needs;
[0241] Data preprocessing includes:
[0242] Label cleaning: Ensure that the label data has been cleaned to remove irrelevant, duplicate or obviously wrong labels;
[0243] Label encoding: convert labels into numerical data suitable for clustering algorithm processing;
[0244] Execution of the clustering algorithm includes:
[0245] Set clustering parameters: according to the algorithm requirements, set appropriate parameters, such as the number of clusters and distance measurement method;
[0246] Execute clustering: call the clustering algorithm to cluster the label data;
[0247] Evaluation of clustering results includes: evaluation based on silhouette coefficient or Calinski-Harabasz index;
[0248] Silhouette coefficient: evaluates the closeness and separation of clustering results;
[0249] Calinski-Harabasz index: evaluates the ratio of variance within a cluster to variance between clusters;
[0250] Manual verification includes:
[0251] Randomly select a part of clustering results for manual verification to ensure the rationality and accuracy of clustering;
[0252] Step 5: Use the pre-trained large model to obtain the label system for the clustering results obtained in the fourth step; including: clustering result preparation, selection of pre-trained large model, design of title generation strategy, calling the pre-trained large model to generate titles, post-processing and optimization, result output and application, manual intervention and correction;
[0253] Clustering result preparation includes:
[0254] Ensure that clustering results have been evaluated for accuracy and reasonableness;
[0255] Organize the clustering results into a format suitable for processing by a pre-trained large model, such as a list or text form;
[0256] Choices for pre-trained large models include:
[0257] Select a large model with strong text generation and summarization capabilities as the pre-trained large model;
[0258] Ensure that the large model has been trained and has a deep understanding of the text content;
[0259] Designing a headline generation strategy includes:
[0260] Design a title generation strategy based on the characteristics and needs of the clustering results;
[0261] Determine the hierarchy of titles, which include first-level titles, second-level titles, and third-level titles;
[0262] Evaluate the title's conciseness, accuracy, and representativeness, and optimize it to ensure it accurately summarizes the content of the text;
[0263] Calling the pre-trained large model to generate titles includes:
[0264] The clustering results are used as input and the pre-trained large model is called to generate titles;
[0265] According to the designed title generation strategy, guide the pre-trained large model to generate titles at different levels;
[0266] Set the number and diversity of generated titles to meet different needs;
[0267] Post-processing and optimization include:
[0268] Post-process the generated titles, such as removing duplicates and merging similar titles;
[0269] Optimize titles as needed to ensure accuracy and readability;
[0270] Results output and applications include:
[0271] Output the generated titles in a preset way, such as a text file, a database or a visual interface;
[0272] Apply the generated titles to text display, directory navigation, and content recommendation scenarios to improve user experience and information acquisition efficiency;
[0273] Manual intervention and correction is to manually intervene and correct the constructed labeling system;
[0274] Step 6: Label system storage;
[0275] The label system after manual intervention and correction is saved in the database, including: selecting a suitable database system, designing the database table structure, importing label data, establishing indexes and constraints, optimizing database performance, and providing interfaces and tools;
[0276] Choosing the right database system involves:
[0277] Choose a suitable database system, including relational database or NoSQL database, based on the data volume of the tag system, query performance requirements, and the team's familiarity;
[0278] Characteristics of NoSQL databases include:
[0279] Flexible scalability: The relational characteristics of relational databases are removed, and there is no relationship between data, so it is very easy to expand.
[0280] Flexible data model: It uses non-relational models such as key / value and column family to allow different types of data to be stored in one data element.
[0281] High performance: NoSQL databases have very high read and write performance, especially when dealing with large amounts of data.
[0282] High availability: NoSQL databases were designed to meet the needs of "horizontal expansion" and therefore have good horizontal expansion capabilities.
[0283] NoSQL databases are widely used in big data, real-time computing, distributed systems and other fields, such as log analysis, real-time recommendation, real-time monitoring, social networks, e-commerce platforms and other scenarios. In the specific embodiments of the present invention, NoSQL databases have many advantages, all of which are within the specific protection scope of the present invention.
[0284] Designing the database table structure includes:
[0285] According to the hierarchical structure and attributes of the label system, a reasonable database table structure is designed, a main table is created to store the first-level title, second-level title and third-level title, and titles of different levels are associated through foreign keys;
[0286] Consider the attributes of tags, including creation time, modification time, and usage frequency, and add corresponding fields to the table structure;
[0287] Importing label data includes:
[0288] Import the sorted, clustered and summarized label data into the database, using a batch import tool or writing a script to import;
[0289] Data cleaning and conversion: to ensure that the imported data matches the database table structure, and to perform data cleaning and conversion;
[0290] Creating indexes and constraints includes:
[0291] Create indexes for key fields based on query requirements to improve query performance;
[0292] According to business needs, set appropriate constraints, such as unique constraints or non-null constraints, to ensure the accuracy and consistency of data;
[0293] Optimizing database performance includes:
[0294] Optimize database performance based on the amount of data and the complexity of queries. Performance optimization includes adjusting cache strategies and optimizing query statements.
[0295] Perform database maintenance at preset times, including cleaning up useless data and updating statistical information to keep the database running efficiently;
[0296] The interfaces and tools provided include:
[0297] Provide an interface for other systems or applications to access the tag system. The interface is REST API or GraphQL, which facilitates data acquisition and use;
[0298] Development or integration tools, including visual interfaces or command line tools, for querying, editing and managing the tag system;
[0299] Step 7: Use the constructed labeling system to label the text library obtained in the first step; including: loading the labeling system, using the large model for preliminary label prediction, using vector similarity for label matching, using the text matching algorithm for fine-tuning, post-processing and label determination, and outputting and storing the results;
[0300] The loading label system includes:
[0301] Load the constructed label system from the database, including first-level title, second-level title, and third-level title;
[0302] Convert the label system into a format suitable for processing by the preprocessing large model, in the form of embedding vectors or label encodings;
[0303] Convert the text in the text library after the first step of preprocessing into an embedding vector;
[0304] Using a large model to make preliminary label predictions includes:
[0305] Input the embedding vector of the text library into the preprocessing large model, and use the generation ability of the preprocessing large model to make preliminary label predictions;
[0306] Get the label candidate list output by the preprocessing large model as input for subsequent steps;
[0307] Label matching using vector similarity includes:
[0308] Calculate the similarity between the text embedding vector and the tag system embedding vector using the cosine similarity method;
[0309] Sort by similarity and select the most matching tag as the candidate tag for the text;
[0310] Fine-tuning using text matching algorithms includes:
[0311] For each of the texts, searching for the most relevant tag in the tag system using a text matching algorithm;
[0312] Combined with the results of vector similarity, the candidate labels are fine-tuned to ensure their accuracy and relevance;
[0313] Post-processing and label determination include:
[0314] Post-process the matched tags, including deduplication and merging similar tags;
[0315] Label determination is to set a threshold or perform manual verification as needed to determine the final label for the text;
[0316] Result output and storage include:
[0317] Associate the labeling results with the corresponding text and output them in a structured format, which is JSON or CSV.
[0318] Store the results in a suitable storage system for subsequent analysis, retrieval, or visualization applications.
[0319] It should be noted that the technical solution of the present invention uses vector similarity for label matching, which is different from the traditional method in natural language processing (NLP) and recommendation systems. The basic idea of the method of using vector similarity for label matching in the technical solution of the present invention is to convert the label (usually text) into a vector representation, and then calculate the similarity between these vectors to find matching labels.
[0320] The specific steps for label matching using vector similarity are as follows:
[0321] Label text preprocessing:
[0322] Clean text: remove punctuation, stop words, special characters, etc.
[0323] Tokenization: Breaking text into words or subword units.
[0324] Stemming / lemmatization: Converting words to their base form.
[0325] Vector representation:
[0326] Use word embeddings (such as Word2Vec, GloVe, FastText, etc.) to convert each word or subword unit into a fixed-length vector.
[0327] For label text, you can use average word embedding (for example, take the average of all word vectors) to get the vector representation of the entire label. You can also consider using more complex methods such as TF-IDF weighted word embedding or models such as BERT.
[0328] Calculate vector similarity:
[0329] Compute the similarity between two label vectors using a metric such as cosine similarity, Euclidean distance, or Manhattan distance. Cosine similarity is a commonly used metric because it takes into account the orientation of the vectors, not just their lengths.
[0330] Tag matching:
[0331] Given a query tag, calculate its vector similarity with all candidate tags.
[0332] Select several tags with the highest similarity as matching results.
[0333] Optimization and Evaluation:
[0334] You can use machine learning algorithms to optimize vector representations or similarity calculations, such as by training a classifier to predict matching relationships between labels. Use a validation set or test set to evaluate the accuracy of label matching and adjust model parameters or select the best algorithm.
[0335] application:
[0336] In a recommendation system, similar tags or content can be recommended to users based on their interests or historical behaviors. In NLP tasks, such as text classification and information retrieval, tag matching can be used to improve classification accuracy or search efficiency.
[0337] It should be noted that the above steps are a simplified overview, and more factors and optimization techniques may need to be considered in the actual application of the technical solution of the present invention. In addition, with the development of deep learning technology, more complex models (such as BERT) have been widely used in text representation and label matching tasks, which can generally provide more accurate and robust results.
[0338] In short, labeling text is an important text processing technology, which involves extracting key information from text and using labels to represent the category or attribute of this information. The main purpose of labeling is to make text easier to calculate, index and represent, so as to facilitate subsequent tasks such as text analysis, information retrieval or machine learning. The present invention provides a labeling system construction method based on a large language model, which combines the latest progress in deep learning, natural language processing and large-scale pre-training models. Compared with the traditional labeling system construction based on artificial experience and the labeling system construction based on traditional machine learning, the efficiency of labeling system construction is significantly improved, the context understanding ability is enhanced, the generalization ability is strong, and the labeling system can be updated in time by only fine-tuning the model or continuing to train. In addition, based on many large language models, the current text classification needs to establish a labeling system in advance and the labeling system cannot fully reflect the text content classification problem. It is applicable to a variety of text types, solves the drawbacks of the prior art, and has important research significance and use value.
[0339] In a specific embodiment of the present invention, clarifying the source of the text to be processed in the first step includes: clarifying that the source of the text to be processed is crawling from a website, extracting from a database, or obtaining from a partner, etc.
[0340] In a specific embodiment of the present invention, the format of the text to be processed is unified in the label system construction method based on a large language model by converting the text to be processed into a plain text format to ensure the consistency of subsequent processing. The words removed in the stop word filtering include: de, le, ah, and other similar characters or words.
[0341] In a specific embodiment of the present invention, the third step of label cleaning includes removing irrelevant symbols: deleting symbols irrelevant to the text content; removing duplicate labels: removing duplicate identical labels; removing English; removing labels exceeding a preset length. Label cleaning, merging, and standardization operations are all used to improve the quality and consistency of labels. The following are detailed steps and considerations:
[0342] Remove irrelevant symbols: Delete symbols that are irrelevant to the text content;
[0343] Remove duplicate tags: remove duplicate identical tags;
[0344] Remove English;
[0345] Remove labels that are too long.
[0346] In a specific embodiment of the present invention, the process of selecting a suitable clustering algorithm includes:
[0347] Distance-based algorithms: such as K-means and hierarchical clustering, which are suitable for situations where there are a large number of tags and the relationship between tags is relatively simple;
[0348] Density-based algorithms: such as DBSCAN, which are suitable for situations where there is a certain density difference between labels;
[0349] Model-based algorithms: such as Gaussian mixture models, which are suitable for situations where the labeled data conforms to a specific distribution;
[0350] Methods based on word vectorization include stella-large-zh-v3-1792d and Dmeta-embedding-zh.
[0351] In order to improve the storage efficiency of data, in the specific embodiment of the present invention, the relational database in the label system construction method based on the large language model is a variety of types such as MySQL or PostgreSQL, and the NoSQL database is a variety of types such as MongoDB or Cassandra. It should be noted that these are commonly used methods or types by those skilled in the art, and can also be replaced by other types, all within the specific protection scope of the present invention.
[0352] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.
Claims
1. A method for building a labeling system based on a large language model, characterized in that: The steps include: Step 1: Organize and store the text to be processed, including data collection, data cleaning, text preprocessing, text library establishment, verification and testing. The data collection includes: Determine the data source: Identify the source of the text to be processed; Collect text: Use crawlers or API interfaces to batch download or import target text to the local computer; The data cleaning includes: Remove irrelevant content: Remove content that is not related to tagging, including advertisements, comments, and HTML tags; Text formatting: unify the format of the text to be processed; Remove duplicate text: Use hashing to detect and remove duplicate text to avoid redundancy in subsequent processing; The text preprocessing includes: Word segmentation: Based on the characteristics of Chinese, use appropriate word segmentation tools to segment the text and divide the continuous text into words or phrases; Stop word filtering: remove stop words that are not helpful for tagging; Stemming or lemmatization: For English text, stemming or lemmatization is used to convert words into their basic form, facilitating subsequent tagging operations. Described establishing text library comprises: Select storage method: Based on the amount and size of the text to be processed, select a storage method, which can be a relational database, NoSQL database, or file system; Design the data table structure: If using a relational database, you need to design a table structure to store text data, including text ID, content, source, and timestamp fields; Importing data: importing the texts that have undergone data cleaning and text preprocessing into the text library in batches to ensure the integrity and accuracy of the data; The verification and testing include: Random sampling: randomly extract a portion of text from the text library for verification to ensure the effectiveness of data cleaning and text preprocessing; Performance testing: Testing the read and write performance of the text library to ensure efficient data processing in the subsequent tagging process; Step 2: Call the pre-trained large model to use its generation capabilities to generate labels, including calling the pre-trained large model, preparing input data, calling the pre-trained large model to generate labels, obtaining and processing the generated results, and verifying and evaluating the label quality; The calling of the appropriate large model includes: Based on model capabilities: select large models with strong generative capabilities as an alternative to pre-trained large models; Domain adaptability: If the text to be processed belongs to a specific domain, choose a model trained in that specific domain as the pre-trained large model, or choose a highly versatile model as an alternative to the pre-trained large model; The preparation of input data includes: Text preprocessing: Ensure that the text to be processed has undergone the aforementioned text preprocessing, such as word segmentation and stop word removal; Format adjustment: Adjust the text format according to the input requirements of the pre-trained large model, such as setting appropriate length limits and adding special tags; Calling the pre-trained large model to generate labels includes: API call: Through the API interface provided by the pre-trained large model, the pre-processed text is used as input and the pre-trained large model is called to generate labels; Parameter setting: Set parameters according to requirements, such as the number and diversity of generated labels. The obtaining and processing of the generated results includes: Result parsing: Parsing the generated labels from the output of the pre-trained large model; Post-processing: Perform post-processing on the generated tags, such as removing duplicates and merging similar tags; The verification and assessment of label quality include: Manual verification: Randomly select a portion of the generated labels for manual verification to ensure the accuracy and relevance of the labels; Quantitative evaluation: Use appropriate evaluation metrics to quantitatively evaluate the generated labels, including precision and recall. Step 3: Post-process all labels obtained in step 2, including label cleaning, merging, and standardization. Step 4: Cluster the labels obtained after label post-processing in step 3. The clustering process includes: selecting a suitable clustering algorithm, data preprocessing, executing the clustering algorithm, and evaluating the clustering results. The selection of a suitable clustering algorithm includes: Select the appropriate clustering algorithm according to your needs; The data preprocessing includes: Label cleaning: Ensure that the label data has been cleaned to remove irrelevant, duplicate, or obviously incorrect labels; Label encoding: convert labels into numerical data suitable for clustering algorithm processing; The executing clustering algorithm comprises: Set clustering parameters: According to the algorithm requirements, set appropriate parameters, such as the number of clusters and distance measurement method; Perform clustering: call the clustering algorithm to cluster the label data; The evaluating clustering results includes: evaluating according to the silhouette coefficient or the Calinski-Harabasz index; Silhouette coefficient: evaluates the closeness and separation of clustering results; Calinski-Harabasz index: evaluates the ratio of the variance within a cluster to the variance between clusters; The manual verification includes: Randomly select a part of the clustering results for manual verification to ensure the rationality and accuracy of the clustering; Step 5: Use the pre-trained large model to obtain a label system for the clustering results obtained in step 4. This includes: clustering result preparation, pre-trained large model selection, title generation strategy design, calling the pre-trained large model to generate titles, post-processing and optimization, result output and application, and manual intervention and correction. The clustering result preparation includes: Ensure that clustering results have been evaluated for accuracy and rationality; Organize the clustering results into a format suitable for processing by a pre-trained large model, such as a list or text form; The selection of the pre-trained large model includes: Select a large model with strong text generation and summarization capabilities as the pre-trained large model; Ensure that the large model has been trained and has a deep understanding of the text content; The design title generation strategy includes: Design a title generation strategy based on the characteristics and needs of the clustering results; Determine the hierarchical structure of titles, which includes first-level titles, second-level titles, and third-level titles; Evaluate the title's conciseness, accuracy, and representativeness, and optimize it to ensure it accurately summarizes the text content; The calling of the pre-trained large model to generate a title includes: Take the clustering results as input and call the pre-trained large model to generate titles; According to the designed title generation strategy, guide the pre-trained large model to generate titles at different levels; Set the number and diversity of generated titles to meet different needs; The post-processing and optimization include: Post-process the generated titles, such as removing duplicates and merging similar titles; Optimize titles as needed to ensure accuracy and readability; The result output and application include: Output the generated titles in a preset way, such as a text file, a database or a visual interface; Apply the generated titles to text display, directory navigation, and content recommendation scenarios to improve user experience and information acquisition efficiency; The manual intervention and correction are manual intervention and correction of the constructed labeling system; Step 6: Label system storage; Saving the label system after manual intervention and correction into the database, including: selecting a suitable database system, designing the database table structure, importing label data, establishing indexes and constraints, optimizing database performance, and providing interfaces and tools; The selection of a suitable database system includes: Choose an appropriate database system, including relational or NoSQL databases, based on the data volume, query performance requirements, and team familiarity with the tag system. The design database table structure includes: Design a reasonable database table structure based on the hierarchical structure and attributes of the tag system. Create a master table to store first-level, second-level, and third-level titles, and associate titles at different levels through foreign keys. Consider the attributes of the tag, including creation time, modification time, and usage frequency, and add corresponding fields to the table structure; The imported tag data includes: Import the sorted, clustered, and summarized labeled data into the database using a batch import tool or scripting. Data cleaning and conversion: To ensure that the imported data matches the database table structure, and to perform data cleaning and conversion; The indexing and constraint creation includes: Create indexes for key fields based on query requirements to improve query performance; Set appropriate constraints based on business needs, such as unique constraints or non-null constraints, to ensure data accuracy and consistency; Optimizing database performance includes: Optimize database performance based on data volume and query complexity. This includes adjusting cache strategies and optimizing query statements. Perform database maintenance according to the preset schedule, including cleaning up useless data and updating statistical information to keep the database running efficiently; The provided interfaces and tools include: Provide an interface for other systems or applications to access the tag system. The interface is REST API or GraphQL, which facilitates data acquisition and use. Development or integration tools, including visual interfaces or command-line tools, for querying, editing, and managing the tag system; Step 7: Use the constructed labeling system to label the text library obtained in the first step. This includes: loading the labeling system, using the large model for preliminary label prediction, using vector similarity for label matching, using the text matching algorithm for fine-tuning, post-processing and label determination, and outputting and storing the results. The loading label system includes: Loading the constructed label system from the database, including the first-level title, the second-level title, and the third-level title; Convert the label system into a format suitable for processing by the pre-processing large model, in the form of embedding vectors or label encodings; Convert the text in the text library after the first step of preprocessing into an embedding vector; The use of a large model for preliminary label prediction includes: Inputting the embedding vector of the text library into the preprocessing large model, and using the generation capability of the preprocessing large model to perform preliminary label prediction; Obtain the label candidate list output by the preprocessing large model as input for subsequent steps; The label matching using vector similarity includes: Calculate the similarity between the text embedding vector and the tag system embedding vector using the cosine similarity method; Sort by similarity and select the best matching tag as the candidate tag for the text; The fine adjustment using the text matching algorithm includes: For each of the texts, searching for the most relevant tag in the tag system using a text matching algorithm; Combined with the results of vector similarity, the candidate labels are fine-tuned to ensure their accuracy and relevance; The post-processing and label determination include: Post-process the matched tags, including deduplication and merging similar tags; Label determination involves setting thresholds or performing manual verification as needed to determine the final label for the text; The result output and storage include: Associate the labeling results with the corresponding text and output them in a structured format, which can be JSON or CSV. Store the results in a suitable storage system for subsequent analysis, retrieval, or visualization applications.
2. The method for constructing a label system based on a large language model according to claim 1, characterized in that: The first step of clarifying the source of the text to be processed includes: clarifying that the source of the text to be processed is crawled from a website, extracted from a database, or obtained from a partner.
3. The method for constructing a label system based on a large language model according to claim 1, characterized in that: The format of the text to be processed is unified by converting it into plain text format to ensure the consistency of subsequent processing. The words removed in the stop word filtering include: de, le, ah.
4. The method for building a label system based on a large language model according to claim 1, characterized in that: The third step of label cleaning includes removing irrelevant symbols: deleting symbols that are irrelevant to the text content; removing duplicate labels: removing repeated identical labels; removing English; removing labels that exceed the preset length.
5. The method for constructing a label system based on a large language model according to claim 1, characterized in that: The process of selecting a suitable clustering algorithm includes: Distance-based algorithms: such as K-means and hierarchical clustering, are suitable for situations where there are a large number of labels and the relationships between labels are relatively simple; Density-based algorithms: such as DBSCAN, which are suitable for situations where there is a certain density difference between labels; Model-based algorithms: such as Gaussian mixture models, which are suitable for situations where labeled data conforms to a specific distribution; Methods based on word vectorization include: stella-large-zh-v3-1792d and Dmeta-embedding-zh.
6. The method for building a label system based on a large language model according to claim 1, characterized in that: The relational database is MySQL or PostgreSQL, and the NoSQL database is MongoDB or Cassandra.
Citation Information
Patent Citations
Power user grouping method based on text mining
CN106709754A
Data information label processing method of large language model
CN117453921A
Intelligent information processing method and server
CN117540036A
Method for establishing content label based on e-commerce data and social media marketing content data
CN117786103A
Text classification method, apparatus, computer device, and storage medium
WO2022227207A1
Cited By
Short text clustering method and device based on large language model
CN119669476A
Supervisory control and data acquisition (SCADA) system label generation method and system based on structured multi-source information fusion
CN120144567A
SCADA system label generation method and system based on structured multi-source information fusion
CN120144567B
Hierarchical classification method based on large language model, electronic equipment and computer readable storage medium
CN120179821A
Method and system for multi-dimensionally summarizing classroom live-recorded texts
CN120319248A
