Automatic labeling method, device and equipment based on large model and clustering algorithm

Abstract

The present application relates to the technical field related to data processing, and specifically to an automatic labeling method, apparatus, and device based on a large model and a clustering algorithm. The method comprises: using natural language processing technology to perform a preliminary analysis on the original data set and generate an initial label for each piece of data in the original data set; applying a clustering algorithm to perform cluster analysis on the generated initial labels, grouping the initial labels based on the similarity or correlation between the labels to obtain multiple initial label groups; selecting the most representative label from each initial label group to replace all labels in the initial label group to simplify the label system and reduce redundancy; determining the similarity between any two pieces of data in the original data set; and adjusting the label system based on the similarity.

Description

Technical Field

[0001] The present application relates to the technical field related to data processing, and specifically to an automatic labeling method, device and equipment based on a large model and a clustering algorithm. Background Art

[0002] With the rapid development of technology, the amount of data has shown explosive growth, and the demand for data annotation has also increased accordingly.

[0003] However, current data annotation largely relies on manual annotation, which presents significant efficiency issues. Manual annotation is not only time-consuming and labor-intensive, but also limited by speed and accuracy, making it difficult to meet the demands of large-scale datasets. Furthermore, manual annotation is costly and prone to fatigue and errors when processing large amounts of data, compromising annotation quality. Summary of the Invention

[0004] In view of this, the embodiments of the present application are dedicated to providing an automatic labeling method, apparatus and device based on a large model and a clustering algorithm.

[0005] This application provides an automatic annotation method based on a large model and a clustering algorithm, including:

[0006] Use natural language processing technology to perform preliminary analysis on the original data set and generate an initial label for each data in the original data set;

[0007] Applying a clustering algorithm to perform cluster analysis on the generated initial labels, grouping the initial labels based on the similarity or correlation between the labels to obtain multiple initial label groups;

[0008] Select the most representative label from each initial label group to replace all labels in the initial label group to simplify the label system and reduce redundancy;

[0009] Determine the similarity between any two data in the original data set;

[0010] The labeling system is adjusted based on the similarity.

[0011] In some embodiments, the natural language processing technology includes: technology corresponding to the llama3.1 model of DSPy technology.

[0012] In some embodiments, a clustering algorithm is applied to perform cluster analysis on the generated initial tags, and the initial tags are grouped based on similarity or correlation between the tags, including:

[0013] Determine the similarity between any two initial labels;

[0014] For each initial tag, determine the n other initial tags with the highest similarity to the initial tag; accumulate the similarities between the n other initial tags and the initial tag to obtain the score corresponding to the initial tag;

[0015] Filter out the m initial labels with the highest scores as the target initial labels;

[0016] For each target initial label, the target initial label and n other initial labels with the highest similarity to the target initial label are determined to form a group of labels to achieve initial label grouping.

[0017] In some embodiments, determining the similarity between any two initial tags includes:

[0018] Determine the similarity between the word vectors corresponding to the two initial labels as the first similarity value;

[0019] Determine the similarity between the data corresponding to the two initial labels as a second similarity value;

[0020] It is determined that a weighted sum of the first similarity value and the second similarity value is the similarity between the two initial labels.

[0021] In some embodiments, determining the similarity between any two pieces of data in the original data set;

[0022] Calculate the cosine similarity of any two data based on the gte-large model and milvus database.

[0023] In some embodiments, adjusting the tag system based on the similarity includes:

[0024] Determine the two data with cosine similarity greater than 0.9 as the target data pair;

[0025] Verify whether the two pieces of data corresponding to the target data pair belong to the same label under the simplified label system;

[0026] If not, the llama3.1 model based on DSPy again targets the data pair to determine whether the two corresponding data should belong to the same label;

[0027] If so, modify the labels of the two data corresponding to the target data pair.

[0028] In some embodiments, if the llama3.1 model based on DSPy again determines whether the two data corresponding to the target data pair should belong to the same label, and it is determined that they do not belong to the same label, then training data is generated based on the target data pair;

[0029] The training data is used to retrain the gte-large model.

[0030] In some embodiments, after modifying the labels of the two pieces of data corresponding to the target data pair, the method further includes:

[0031] Based on the target data pair, training data is generated; the training data is used to retrain the gte-large model.

[0032] The present application also provides an automatic labeling device based on a large model and a clustering algorithm, comprising:

[0033] The preliminary analysis module is used to perform preliminary analysis on the original data set using natural language processing technology and generate an initial label for each data in the original data set;

[0034] A clustering module is used to apply a clustering algorithm to perform cluster analysis on the generated initial labels, grouping the initial labels based on the similarity or correlation between the labels to obtain multiple initial label groups;

[0035] A simplification module is used to select the most representative label from each initial label group to replace all labels in the initial label group, so as to simplify the label system and reduce redundancy;

[0036] The judgment module is used to judge the similarity between any two data in the original data set;

[0037] An adjustment module is used to adjust the label system based on the similarity.

[0038] The present application also provides an electronic device, comprising:

[0039] A processor, and a memory for storing a program executable by the processor;

[0040] The processor is used to implement the automatic labeling method of the above-mentioned large model and clustering algorithm by running the program in the memory.

[0041] The present application provides an automatic annotation method based on a large model and a clustering algorithm, comprising: using natural language processing technology to perform a preliminary analysis of the original data set and generate an initial label for each data item in the original data set; applying a clustering algorithm to perform cluster analysis on the generated initial labels, grouping the initial labels based on the similarity or correlation between the labels to obtain multiple initial label groups; selecting the most representative label from each initial label group to replace all labels in the initial label group to simplify the label system and reduce redundancy; determining the similarity between any two data items in the original data set; and adjusting the label system based on the similarity. With this configuration, the automatic annotation method uses natural language processing technology to perform a preliminary analysis of the original data set and generate initial labels, significantly reducing the workload of manual annotation. This method can quickly process large amounts of data and improve the efficiency of data annotation. The application of a clustering algorithm helps to group similar labels, further simplifying the label system and reducing redundancy, thereby improving the accuracy and consistency of annotation. Traditional manual annotation methods are costly, while automatic annotation methods can reduce reliance on manual labor, thereby reducing overall annotation costs. By preprocessing data using clustering algorithms, patterns and structures can be discovered, enabling subsequent machine learning algorithms to more effectively process and analyze the data. This method is adaptable to different data types and application scenarios, offering excellent flexibility. Whether it's text, images, or other types of data, this automatic annotation method can be applied to improve efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The above and other purposes, features, and advantages of the present application will become more apparent through a more detailed description of the embodiments of the present application in conjunction with the accompanying drawings. The accompanying drawings are intended to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the drawings, the same reference numerals generally represent the same components or steps.

[0043] Figure 1 This is a flowchart of an automatic labeling method based on a large model and clustering algorithm provided in one embodiment of the present application.

[0044] Figure 2 This is a flowchart of an automatic labeling method based on a large model and clustering algorithm provided in another embodiment of the present application.

[0045] Figure 3 It is a structural diagram of an automatic labeling device based on a large model and a clustering algorithm provided by an embodiment of the present application.

[0046] Figure 4 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0047] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0048] To effectively address issues such as large sample sizes, low answer matching, and cognitive differences between manual annotation and large models during labeling, we need to integrate efficient and intelligent technical means and strategy optimization to build a labeling system that is both accurate and automated.

[0049] First, we leverage the latest advances in natural language processing technology, DSPy, specifically its applications in transfer learning and few-shot learning. These techniques enable large models to effectively learn and generalize to new problems using limited samples, significantly reducing reliance on large amounts of manually labeled data. This approach not only improves annotation efficiency but also, to a certain extent, addresses the limitations of large models in understanding complex business scenarios.

[0050] Next, we used a large model for preliminary labeling, then applied label clustering techniques to group similar questions under the same label. During this process, we focused on leveraging the business insights gained from manual labeling. By repeatedly sorting and adjusting similar questions, we ensured that questions with different wording but the same meaning were correctly categorized.

[0051] This comprehensive approach leverages the advantages of large models for automatic labeling while also incorporating the precise classification principles of traditional methods based on similarity metrics. By combining these two approaches, we achieve precise and automated labeling and answer generation, providing support for online question-answering systems.

[0052] After introducing the basic principles of the present application, various non-limiting embodiments of the present application will be described in detail with reference to the accompanying drawings.

[0053] Exemplary Methods

[0054] Reference Figure 1 and Figure 2 , the method provided in this application includes the following contents.

[0055] Step S110: Using natural language processing technology, perform a preliminary analysis on the original data set and generate an initial label for each piece of data in the original data set;

[0056] In this step, natural language processing (NLP) technology is applied to analyze the raw dataset. NLP techniques include, but are not limited to, text segmentation, part-of-speech tagging, and named entity recognition to understand the text content and extract key information. The goal of this step is to generate an initial label for each piece of data in the dataset. This label is a preliminary classification of the data content. For example, if the dataset contains user reviews, NLP technology will identify the sentiment of the reviews and assign an initial positive or negative label accordingly.

[0057] Step S120, applying a clustering algorithm to perform cluster analysis on the generated initial tags, grouping the initial tags based on similarity or correlation between the tags to obtain multiple initial tag groups;

[0058] After generating the initial labels, the next step is to analyze them using a clustering algorithm. Clustering algorithms, such as K-means, hierarchical clustering, or DBSCAN, group labels based on similarities or associations. The goal of this step is to group similar labels together, thereby identifying underlying patterns and structures in the data. For example, if the initial labels include "satisfied," "happy," and "pleased," a clustering algorithm might group these labels into the "positive sentiment" group.

[0059] Step S130, selecting the most representative tag from each initial tag group to replace all tags in the initial tag group to simplify the tag system and reduce redundancy;

[0060] After cluster analysis is complete, the most representative label is selected from each initial label group. This representative label is used to replace all labels in the group to simplify the entire label system and reduce redundancy. Methods for selecting representative labels include calculating the center point of the labels within the group, frequency analysis, or metrics based on semantic similarity. For example, if a group contains "satisfied," "happy," and "pleased," "satisfied" is selected as the representative label because it appears most frequently in the group or best represents the common characteristics of the labels in the group.

[0061] Step S140, determining the similarity between any two pieces of data in the original data set;

[0062] After simplifying the labeling system, the next step is to determine the similarity between any two data points in the original dataset. This can be achieved by calculating the distance between the data vectors (such as Euclidean distance or cosine similarity). This step is crucial for identifying and processing similar data points, helping to further optimize the accuracy and consistency of the labels. For example, if two reviews are very similar in semantics, they will be assigned the same label.

[0063] Step S150: adjusting the label system based on the similarity.

[0064] The final step involves adjusting the labeling system based on the similarity between data points. If two pieces of data are found to be very similar but assigned different labels, the system will adjust these labels to ensure consistency. This involves re-evaluating and adjusting representative labels, or introducing new labels when necessary to better reflect the actual content of the data. This step ensures that the labeling system can flexibly adapt to the actual characteristics of the data, improving the accuracy and reliability of the annotations.

[0065] Through these steps, the automatic labeling method not only improves the efficiency of data processing, but also enhances the accuracy and consistency of labeling, providing a solid foundation for subsequent data analysis and machine learning tasks.

[0066] In some embodiments, the natural language processing technology includes: technology corresponding to the llama3.1 model of DSPy technology.

[0067] DSPy is a framework that uses language models and retrieval models to solve high-level tasks. It allows for rapid iterative construction of modular AI systems and provides algorithms to optimize their hints and weights. The core advantages of the DSPy framework are:

[0068] Modular AI system building: DSPy allows builders to quickly iterate and build modular AI systems, which makes the system more flexible and scalable.

[0069] Optimizing hints and weights: DSPy provides algorithms to optimize the hints and weights of language models, which are crucial for improving model performance and adapting them to specific tasks.

[0070] Integrated retrieval model: DSPy is able to integrate retrieval models, which is very useful for tasks that require a lot of background knowledge or data, such as question answering systems and information retrieval.

[0071] Llama 3.1 is a large-scale language model open-sourced by Meta (Facebook's predecessor company). It features pre-trained and fine-tuned language models with 8B and 70B parameters, respectively. It builds on the latest advances in NLP technology and claims to have high accuracy in understanding and responding to complex queries. Key features of the Llama 3.1 model include:

[0072] Pre-trained and fine-tuned on instructions: The Llama 3.1 model is pre-trained and fine-tuned on instructions, enabling it to understand and respond to complex natural language queries.

[0073] High Accuracy: Llama 3.1 demonstrates high accuracy when processing complex queries, thanks to its large number of parameters and advanced training techniques.

[0074] Open source available: As an open source model, Llama 3.1 can be freely used and modified by researchers and developers to adapt to various NLP tasks.

[0075] Combining DSPy technology with Llama 3.1 models enables powerful applications in natural language processing, such as building efficient question-answering systems, information retrieval systems, and text classifiers. This technology combination provides a powerful toolset for processing and understanding large amounts of natural language data and generating useful insights and responses.

[0076] Furthermore, a clustering algorithm is applied to perform cluster analysis on the generated initial tags, and the initial tags are grouped based on the similarity or correlation between the tags, including:

[0077] Determine the similarity between any two initial labels;

[0078] For each initial tag, determine the n other initial tags with the highest similarity to the initial tag; accumulate the similarities between the n other initial tags and the initial tag to obtain the score corresponding to the initial tag;

[0079] Filter out the m initial labels with the highest scores as the target initial labels;

[0080] For each target initial label, the target initial label and n other initial labels with the highest similarity to the target initial label are determined to form a group of labels to achieve initial label grouping.

[0081] In this step, we use natural language processing techniques to calculate the similarity between any two initial labels. This can be achieved through various algorithms, such as cosine similarity, Levenshtein distance, or Word2Vec. Cosine similarity is a common method that determines the similarity between two vectors by calculating the ratio of the dot product of two vectors to the product of their norms. Word2Vec is a technology that converts text into vectors, which can be used to calculate the similarity between texts.

[0082] For each initial label, we need to find the n other initial labels that are most similar to it. This can be done by comparing the similarity scores of all label pairs. For example, if we use cosine similarity, we can calculate the cosine similarity between a label vector and all other label vectors and select the n labels with the highest similarity scores.

[0083] Once we have determined the n labels that are most similar to each initial label, the next step is to accumulate the similarity scores between these labels and the initial labels. This summed score will represent the "centrality" or "representativeness" of each initial label. The higher the score, the more representative the label is in the cluster.

[0084] After accumulating the scores, we select the m initial labels with the highest scores as the target initial labels. These labels will be used in the subsequent clustering process because they are the most representative in their respective clusters.

[0085] Finally, for each target initial label, we again identify the n other initial labels with the highest similarity and group these labels together with the target initial label to achieve grouping of the initial labels. This step helps to form more compact and homogeneous clusters, where each cluster is centered around one or more representative labels.

[0086] Through these steps, we can effectively use natural language processing technology to organize and simplify the label system, providing clearer and more accurate label classification for subsequent data analysis and machine learning tasks.

[0087] Specifically, determining the similarity between any two initial labels includes: determining the similarity between the word vectors corresponding to the two initial labels as a first similarity value; determining the similarity between the data corresponding to the two initial labels as a second similarity value; and determining that the weighted sum of the first similarity value and the second similarity value is the similarity between the two initial labels.

[0088] Determining the similarity between any two initial labels is a key step in natural language processing, especially when using clustering algorithms to group labels. The following is a detailed description of this step:

[0089] First similarity value: similarity between word vectors

[0090] 1. Word vector generation: First, we need to convert each initial label into a word vector. This can be achieved by using pre-trained word embedding models (such as Word2Vec, GloVe, or BERT), which can map words to vectors in a high-dimensional space while preserving the semantic information of the words.

[0091] 2. Similarity Calculation: Once we have the word vectors for the two initial tags, we can use a metric such as cosine similarity to calculate the similarity between them. Cosine similarity determines the similarity between two vectors by calculating the ratio of the dot product of the two vectors to the product of their modulos.

[0092] Second similarity value: similarity between data

[0093] 1. Data representation: In addition to word vectors, we can also calculate the similarity between two initial labels based on raw data (such as text data). This may involve converting the entire text data into a vector representation, such as using models such as TF-IDF, LDA, or BERT.

[0094] 2. Similarity calculation: For the data corresponding to two initial labels, we can also use cosine similarity or other similarity measurement methods to calculate the similarity between them.

[0095] Weighted sum: combination of the first similarity value and the second similarity value

[0096] 1. Weight Assignment: When calculating the overall similarity between two initial labels, we may need to weight the first similarity value (word vector similarity) and the second similarity value (data similarity). The weight assignment can be determined based on the specific application scenario and data characteristics. For example, if word vector similarity is more important, we can assign it a higher weight.

[0097] 2. Weighted Sum Calculation: The final similarity is the weighted sum of the first and second similarity values. This can be expressed as follows: Total Similarity = w_1 First Similarity Value + w_2 Second Similarity Value, where w_1 and w_2 are the weights of the first and second similarity values, respectively.

[0098] This approach allows us to combine the semantic similarity and data similarity of tags to produce a more comprehensive similarity metric. This comprehensive similarity metric is very useful for clustering algorithms because it helps them more accurately identify and group similar tags, thereby improving the quality and practicality of clustering results.

[0099] Furthermore, determining the similarity between any two data in the original data set specifically includes: performing cosine similarity calculation on the any two data based on the gte-large model and the milvus database.

[0100] The following is an explanation of the specific steps to determine the similarity between any two data in the original data set:

[0101] Step 1: Generate word vectors using the gte-large model

[0102] First, we use the gte-large model to convert each piece of data in the original dataset into a word vector. The gte-large model can understand the text content and extract key information, mapping the text into vectors in a high-dimensional space. These vectors can capture the semantic information of the words.

[0103] Step 2: Store word vectors in the Milvus database

[0104] Next, we store these word vectors in the Milvus database. Milvus is an open-source vector database designed for storing and retrieving large-scale vector data. In Milvus, we can define collections and specify the data type for each field, including the dimension of the word vector.

[0105] Step 3: Create indexes to improve query efficiency

[0106] To improve query efficiency, Milvus supports creating indexes for vector fields. We can specify the index type (such as IVF_FLAT) and parameters (such as nlist) for the vector field to create an index suitable for our data.

[0107] Step 4: Calculate cosine similarity

[0108] In Milvus, cosine similarity is a commonly used similarity metric. Cosine similarity measures the similarity between two vectors by calculating the cosine of the angle between them.

[0109] Step 5: Perform similarity query

[0110] Finally, we can use Milvus to perform similarity queries. By specifying a query vector and a similarity threshold, Milvus can quickly retrieve the data vector that is most similar to the query vector.

[0111] Through these steps, we can effectively use the gte-large model and the Milvus database to determine the similarity between any two pieces of data in the original dataset, providing strong support for natural language processing tasks.

[0112] Furthermore, adjusting the label system based on the similarity includes:

[0113] Determine the two data with cosine similarity greater than 0.9 as the target data pair; verify whether the two data corresponding to the target data pair belong to the same label under the simplified label system; if not, check again based on DSPy's llama3.1 model whether the two data corresponding to the target data pair should belong to the same label; if so, modify the labels of the two data corresponding to the target data pair.

[0114] In some embodiments, if the llama3.1 model based on DSPy determines that two data corresponding to a target data pair should have the same label and determines that they do not have the same label, training data is generated based on the target data pair; the training data is used to retrain the gte-large model.

[0115] The solution provided in this application is further described below with reference to specific embodiments:

[0116] To optimize the processing of unlabeled data and improve classification results, we can systematically use the following steps to consolidate and refine the dataset, combining the features of dspy's llama3.1 model and the gte-large model:

[0117] 1. Preliminary annotation and label generation

[0118] Overview of steps:

[0119] Using DSPy's llama3.1 model: We perform a preliminary analysis of the unlabeled dataset and generate an initial label for each sentence. These labels allow for duplication to capture the diversity and potential ambiguity in the data.

[0120] Label diversity: Ensure label diversity to cover a wide range of business topics and categories in the data, providing a rich information foundation for subsequent processing.

[0121] 2. Label clustering and merging

[0122] Overview of steps:

[0123] Apply clustering algorithm: Perform cluster analysis on the initially generated tags and group them based on the similarity or correlation between the tags.

[0124] Select representative labels: Select the most representative label from each cluster to replace all labels in the cluster, thereby simplifying the label system and reducing redundancy.

[0125] Alleviate the small sample problem: By clustering and merging, the sample size of each category is increased, which helps to alleviate the classification difficulties caused by sample scarcity.

[0126] Detailed description of joint clustering algorithm:

[0127] (1) Using M3E as the word vector, find the similarity L between any two label vectors;

[0128] (2) Using the similarity of the words in the sentence, find the similarity K between any two label vectors;

[0129] (3) Vector similarity L + vector similarity K = total similarity H between any two labels;

[0130] (4) Calculate any label A, find the 8 labels with the closest total similarity H, add them together, and finally get a total score sorce;

[0131] (5) Sort the total scores from high to low and keep the top 50. See the following table:

[0132] Table 1: Initial label grouping table

[0133]

[0134]

[0135] (6) For the first 50, in each label group, select label A as the group label, and all labels in the same group are changed to label A;

[0136] (7) In the above example, “complaints and dissatisfaction with the service” replaces the other tags in the previous text;

[0137] (8) The above work achieves label clustering and deduplication;

[0138] (9) Obtain the processed data set (S)

[0139] 3. Data Distillation and Similarity Judgment

[0140] Step by step:

[0141] Similarity calculation: Use the untrained gte-large model and the milvus database to calculate the cosine similarity of each pair of sentences in the data set (S), paying special attention to those sentence pairs with a cosine similarity greater than 0.9.

[0142] Category consistency check:

[0143] 1. For high similarity sentence pairs (A and B), first check whether they have been classified into the same category.

[0144] 2. If they are not of the same type, use DSPy's llama3.1 model to evaluate whether they should belong to the same category.

[0145] 2.1 If they are confirmed to be of the same type, sentence B will be classified into the category of sentence A.

[0146] 2.2 If they are not of the same type, construct a triple consisting of sentence A, a sentence of the same type as A, and sentence B as the training sample G.

[0147] 3. Iterative Distillation: The organized training samples G are fed into the gte-large model for comparative learning. The similarity calculation and category consistency check described above are repeated at least twice to further refine the dataset. This process allows sentences with similar semantics to be clustered more closely, while sentences with significant semantic differences retain sufficient distinction.

[0148] 4. Producing optimized datasets

[0149] The final result: After the above steps, the dataset not only significantly improves classification performance, but also makes the logical relationships and semantic hierarchy within the data clearer. The output dataset will become a high-quality resource with accurate classification and reasonable structure, providing solid support for subsequent natural language processing tasks.

[0150] Through the above structured processing flow, we can effectively leverage the powerful capabilities of dspy's llama3.1 and gte-large models to extract high-quality, high-value classification datasets from unlabeled raw data.

[0151] Exemplary devices

[0152] The device embodiments of this application can be used to execute the method embodiments of this application. For details not disclosed in the device embodiments of this application, please refer to the method embodiments of this application.

[0153] Figure 3 FIG. 1 is a block diagram of an automatic labeling device based on a large model and clustering algorithm provided by an embodiment of the present application. Figure 3 As shown, the device includes:

[0154] A preliminary analysis module 31 is used to perform preliminary analysis on the original data set using natural language processing technology and generate an initial label for each data item in the original data set;

[0155] A clustering module 32 is configured to apply a clustering algorithm to perform cluster analysis on the generated initial tags, and group the initial tags based on similarity or correlation between the tags to obtain multiple initial tag groups;

[0156] A simplification module 33 is configured to select the most representative label from each initial label group to replace all labels in the initial label group, so as to simplify the label system and reduce redundancy;

[0157] The judgment module 34 is used to judge the similarity between any two data in the original data set; the adjustment module 35 is used to adjust the label system based on the similarity.

[0158] Exemplary electronic devices

[0159] Below, reference Figure 4 To describe the electronic device according to the embodiment of the present application. Figure 4 A block diagram of an electronic device according to an embodiment of the present application is illustrated.

[0160] like Figure 4 As shown, electronic device 400 includes one or more processors 410 and memory 420 .

[0161] The processor 410 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 400 to perform desired functions.

[0162] The memory 420 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. The non-volatile memory may include, for example, read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 410 may execute the program instructions to implement the automatic labeling method based on the large model and clustering algorithm of each embodiment of the present application described above and / or other desired functions. Various contents such as category correspondences may also be stored in the computer-readable storage medium.

[0163] In one example, the electronic device 400 may further include an input device 430 and an output device 440 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0164] In addition, the input device 430 may also include, for example, a keyboard, a mouse, an interface, etc. The output device 440 may output various information to the outside, including analysis results, etc. The output device 440 may include, for example, a display, a speaker, a printer, a communication network and its connected remote output device, etc.

[0165] Of course, to simplify, Figure 4 Only some of the components in the electronic device related to the present application are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, the electronic device may further include any other appropriate components according to specific application scenarios.

[0166] Exemplary computer program products and computer-readable storage media

[0167] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the automatic labeling method based on a large model and a clustering algorithm according to various embodiments of the present application described in the above "Exemplary Method" section of this specification.

[0168] The computer program product may be written in any combination of one or more programming languages ​​to implement the program code for performing the operations of the embodiments of the present application, including object-oriented programming languages ​​such as Java, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0169] In addition, an embodiment of the present application may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, causes the processor to execute the steps of the automatic labeling method based on a large model and a clustering algorithm according to various embodiments of the present application described in the above "Exemplary Method" section of this specification.

[0170] The computer-readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can, for example, include but is not limited to a system, device or component of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0171] The above description has been provided for the purpose of illustration and description. Furthermore, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although a number of example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. An automatic labeling method based on a large model and clustering algorithm, characterized in that: include: Use natural language processing technology to perform preliminary analysis on the original data set and generate an initial label for each data in the original data set; Applying a clustering algorithm to perform cluster analysis on the generated initial labels, grouping the initial labels based on the similarity or correlation between the labels to obtain multiple initial label groups; Select the most representative label from each initial label group to replace all labels in the initial label group to simplify the label system and reduce redundancy; Determine the similarity between any two data in the original data set; Determine the similarity between any two data in the original data set determined according to the judgment, and the two data with a cosine similarity greater than a preset threshold are the target data pair; Verify whether the two pieces of data corresponding to the target data pair belong to the same label under the simplified label system; If they do not belong to the same label, then judging again based on the first large model whether the two pieces of data corresponding to the target data pair should belong to the same label; If the first large model determines that the two data corresponding to the target data pair should belong to the same label, modify the labels of the two data corresponding to the target data pair to make them consistent; Furthermore, generating training data based on the second judgment result of the first large model or the result of the label modification; The generated training data is used to retrain a model for determining the similarity between any two data in the original data set.

2. The automatic labeling method based on a large model and a clustering algorithm according to claim 1, characterized in that: The natural language processing technology includes: technology corresponding to the llama3.1 model of DSPy technology.

3. The automatic labeling method based on a large model and a clustering algorithm according to claim 1, characterized in that: Apply a clustering algorithm to perform cluster analysis on the generated initial tags, and group the initial tags based on the similarity or correlation between the tags, including: Determine the similarity between any two initial labels; For each initial tag, determine the n other initial tags with the highest similarity to the initial tag; accumulate the similarities between the n other initial tags and the initial tag to obtain the score corresponding to the initial tag; Filter out the m initial labels with the highest scores as the target initial labels; For each target initial label, the target initial label and n other initial labels with the highest similarity to the target initial label are determined to form a group of labels to achieve initial label grouping.

4. The automatic labeling method based on a large model and a clustering algorithm according to claim 3 is characterized in that: Determine the similarity between any two initial labels, including: Determine the similarity between the word vectors corresponding to the two initial labels as the first similarity value; Determine the similarity between the data corresponding to the two initial labels as a second similarity value; It is determined that a weighted sum of the first similarity value and the second similarity value is the similarity between the two initial labels.

5. The automatic labeling method based on a large model and a clustering algorithm according to claim 4 is characterized in that: Determining the similarity between any two pieces of data in the original data set includes: Calculate the cosine similarity of any two data based on the gte-large model and milvus database.

6. An automatic labeling device based on a large model and clustering algorithm, characterized in that: include: The preliminary analysis module is used to perform preliminary analysis on the original data set using natural language processing technology and generate an initial label for each data in the original data set; A clustering module is used to apply a clustering algorithm to perform cluster analysis on the generated initial labels, grouping the initial labels based on the similarity or correlation between the labels to obtain multiple initial label groups; A simplification module is used to select the most representative label from each initial label group to replace all labels in the initial label group, so as to simplify the label system and reduce redundancy; The judgment module is used to judge the similarity between any two data in the original data set; an adjustment module, configured to determine the similarity between any two data in the original data set determined according to the determination, wherein the two data with a cosine similarity greater than a preset threshold are the target data pair; verify whether the two data corresponding to the target data pair belong to the same label under the simplified label system; if they do not belong to the same label, determine again based on the first large model whether the two data corresponding to the target data pair should belong to the same label; if the first large model determines that the two data corresponding to the target data pair should belong to the same label, modify the labels of the two data corresponding to the target data pair to make them consistent; Furthermore, generating training data based on the second judgment result of the first large model or the result of the label modification; The generated training data is used to retrain a model for determining the similarity between any two data in the original data set.

7. An electronic device, characterized in that: include: A processor, and a memory for storing a program executable by the processor; The processor is configured to implement the automatic labeling method based on a large model and a clustering algorithm according to any one of claims 1 to 5 by running the program in the memory.

Images