Automatic labeling method, device and equipment based on large model and clustering algorithm

Abstract

The invention relates to the technical field related to data processing, in particular to an automatic labeling method, device and equipment based on a large model and a clustering algorithm. The method comprises the following steps: performing preliminary analysis on an original data set by using a natural language processing technology, and generating an initial label for each piece of data in the original data set; performing clustering analysis on the generated initial labels by applying a clustering algorithm, and grouping the initial labels based on similarity or relevance among the labels to obtain a plurality of initial label groups; selecting the most representative label from each initial label group to replace all labels in the initial label group so as to simplify a label system and reduce redundancy; judging the similarity between any two pieces of data in the original data set; and adjusting the label system based on the similarity.

Description

Technical Field

[0001] The present application relates to the technical field related to data processing, and specifically to an automatic labeling method, device and equipment based on a large model and a clustering algorithm. Background Art

[0002] With the rapid development of technology, the amount of data has shown explosive growth, and the demand for data annotation has also increased accordingly.

[0003] However, most of the current data annotation relies on manual annotation, which has obvious efficiency problems. Manual annotation is not only time-consuming and labor-intensive, but also limited by the speed and accuracy of manual work, making it difficult to meet the needs of large-scale data sets. In addition, manual annotation is costly and prone to fatigue and errors when processing large amounts of data, affecting the quality of annotation. Summary of the invention

[0004] In view of this, the embodiments of the present application are directed to providing an automatic labeling method, apparatus and device based on a large model and a clustering algorithm.

[0005] The present application provides an automatic annotation method based on a large model and a clustering algorithm, comprising:

[0006] Use natural language processing technology to perform preliminary analysis on the original data set and generate an initial label for each piece of data in the original data set;

[0007] Applying a clustering algorithm to perform cluster analysis on the generated initial labels, grouping the initial labels based on the similarity or correlation between the labels, and obtaining multiple initial label groups;

[0008] Select the most representative label from each initial label group to replace all labels in the initial label group to simplify the label system and reduce redundancy;

[0009] Determine the similarity between any two pieces of data in the original data set;

[0010] The labeling system is adjusted based on the similarity.

[0011] In some embodiments, the natural language processing technology includes: technology corresponding to the llama3.1 model of DSPy technology.

[0012] In some embodiments, a clustering algorithm is applied to perform cluster analysis on the generated initial tags, and the initial tags are grouped based on the similarity or correlation between the tags, including:

[0013] Determine the similarity between any two initial labels;

[0014] For each initial tag, determine the n other initial tags with the highest similarity to the initial tag; accumulate the similarities between the n other initial tags and the initial tag to obtain a score corresponding to the initial tag;

[0015] Filter out the m initial labels with the highest scores as the target initial labels;

[0016] For each target initial label, the target initial label and n other initial labels with the highest similarity to the target initial label are determined to form a group of labels to achieve initial label grouping.

[0017] In some embodiments, determining the similarity between any two initial tags includes:

[0018] Determine the similarity between the word vectors corresponding to the two initial labels as the first similarity value;

[0019] Determine the similarity between the data corresponding to the two initial labels as a second similarity value;

[0020] It is determined that a weighted sum of the first similarity value and the second similarity value is the similarity between the two initial labels.

[0021] In some embodiments, determining the similarity between any two pieces of data in the original data set;

[0022] Calculate the cosine similarity of any two data based on the gte-large model and milvus database.

[0023] In some embodiments, adjusting the label system based on the similarity includes:

[0024] Determine the two data with cosine similarity greater than 0.9 as the target data pair;

[0025] Verify whether the two pieces of data corresponding to the target data pair belong to the same label under the simplified label system;

[0026] If not, the llama3.1 model based on DSPy again targets the data pair to determine whether the two corresponding data should belong to the same label;

[0027] If so, modify the labels of the two data corresponding to the target data pair.

[0028] In some embodiments, if the llama3.1 model based on DSPy determines whether the two data corresponding to the target data pair should belong to the same label again, and it is determined that they do not belong to the same label, then training data is generated based on the target data pair;

[0029] The training data is used to retrain the gte-large model.

[0030] In some embodiments, after modifying the labels of the two pieces of data corresponding to the target data pair, the method further includes:

[0031] Based on the target data pair, training data is generated; the training data is used to retrain the gte-large model.

[0032] The present application also provides an automatic labeling device based on a large model and a clustering algorithm, comprising:

[0033] A preliminary analysis module is used to perform preliminary analysis on the original data set using natural language processing technology and generate an initial label for each piece of data in the original data set;

[0034] A clustering module is used to apply a clustering algorithm to perform cluster analysis on the generated initial labels, and to group the initial labels based on the similarity or correlation between the labels to obtain multiple initial label groups;

[0035] A simplification module, used to select the most representative label from each initial label group to replace all labels in the initial label group, so as to simplify the label system and reduce redundancy;

[0036] The judgment module is used to judge the similarity between any two data in the original data set;

[0037] An adjustment module is used to adjust the label system based on the similarity.

[0038] The present application also provides an electronic device, comprising:

[0039] A processor, and a memory for storing a program executable by the processor;

[0040] The processor is used to implement the automatic labeling method of the above-mentioned large model and clustering algorithm by running the program in the memory.

[0041] The present application provides an automatic annotation method based on a large model and a clustering algorithm, including: using natural language processing technology to perform a preliminary analysis on the original data set, generating an initial label for each data in the original data set; applying a clustering algorithm to perform cluster analysis on the generated initial labels, grouping the initial labels based on the similarity or correlation between the labels, and obtaining multiple initial label groups; selecting the most representative label from each initial label group to replace all labels in the initial label group to simplify the label system and reduce redundancy; judging the similarity between any two data in the original data set; and adjusting the label system based on the similarity. In this way, the automatic annotation method performs a preliminary analysis on the original data set using natural language processing technology and generates initial labels, which significantly reduces the workload of manual annotation. This method can quickly process a large amount of data and improve the efficiency of data annotation. The application of clustering algorithms helps to group similar labels, further simplify the label system and reduce redundancy, thereby improving the accuracy and consistency of annotation. The traditional manual annotation method is costly, while the automatic annotation method can reduce the dependence on manual labor, thereby reducing the overall annotation cost. By preprocessing the data with clustering algorithms, patterns and structures in the data can be discovered, allowing subsequent machine learning algorithms to process and analyze the data more effectively. This method can adapt to different data types and application scenarios and has good flexibility. Whether it is text, images or other types of data, this automatic annotation method can be applied to improve efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0043] Figure 1 It is a flowchart of an automatic labeling method based on a large model and a clustering algorithm provided in one embodiment of the present application.

[0044] Figure 2 It is a flowchart of an automatic labeling method based on a large model and a clustering algorithm provided in another embodiment of the present application.

[0045] Figure 3 It is a structural diagram of an automatic labeling device based on a large model and a clustering algorithm provided by an embodiment of the present application.

[0046] Figure 4 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] In order to effectively solve the problems faced in the labeling process, such as large sample size, low answer matching, and cognitive differences between manual labeling and large models, we need to integrate efficient and intelligent technical means and strategy optimization to build a labeling system that is both accurate and automated.

[0049] First, we make full use of the latest advances in natural language processing technology, DSPy technology, especially in transfer learning and few-sample learning technology. These technologies enable large models to effectively learn and generalize to new problems under the condition of limited samples, significantly reducing the reliance on large-scale manually labeled data. In this way, we can not only improve the labeling efficiency, but also make up for the shortcomings of large models in understanding complex business scenarios to a certain extent.

[0050] Secondly, we first use the big model to do preliminary labeling, and then use label clustering technology to classify similar questions under the same label. In this process, we pay special attention to the business cognition advantages of manual labeling, and through repeated sorting and adjustment of similar questions, we ensure that those questions with different wordings but the same meaning can be correctly classified.

[0051] This comprehensive approach not only takes advantage of the large model in automatic labeling, but also absorbs the idea of ​​accurate classification based on similarity measurement in traditional methods. Through the organic combination of the two, we have achieved accurate and automatic labeling and answer generation, providing support for online question-answering systems.

[0052] After introducing the basic principles of the present application, various non-limiting embodiments of the present application will be described in detail with reference to the accompanying drawings.

[0053] Exemplary Methods

[0054] Reference Figure 1 and Figure 2 , the method provided in this application includes the following contents.

[0055] Step S110, using natural language processing technology to perform a preliminary analysis on the original data set and generate an initial label for each piece of data in the original data set;

[0056] In this step, natural language processing (NLP) technology is applied to analyze the original data set. NLP technology includes but is not limited to text segmentation, part-of-speech tagging, named entity recognition, etc. to understand the text content and extract key information. The goal of this step is to generate an initial label for each piece of data in the data set. This label is a preliminary classification of the data content. For example, if the data set contains user comments, NLP technology will identify the sentiment of the comments and assign positive or negative initial labels accordingly.

[0057] Step S120, applying a clustering algorithm to perform cluster analysis on the generated initial tags, grouping the initial tags based on the similarity or correlation between the tags, and obtaining a plurality of initial tag groups;

[0058] After generating the initial labels, the next step is to analyze them using a clustering algorithm. Clustering algorithms, such as K-means, hierarchical clustering, or DBSCAN, will group labels based on their similarity or association. The goal of this step is to group similar labels together to identify underlying patterns and structures in the data. For example, if the initial labels include "satisfied," "happy," and "pleased," a clustering algorithm will group these labels into the "positive emotion" group.

[0059] Step S130, selecting the most representative tag from each initial tag group to replace all tags in the initial tag group, so as to simplify the tag system and reduce redundancy;

[0060] After the cluster analysis is completed, the most representative label will be selected from each initial label group. This representative label will be used to replace all labels in the group to simplify the entire label system and reduce redundancy. Methods for selecting representative labels include calculating the center point of the labels within the group, frequency analysis, or measures based on semantic similarity. For example, if a group contains "satisfied", "happy", and "pleased", "satisfied" is selected as the representative label because it appears most frequently in the group or best represents the common characteristics of the labels in the group.

[0061] Step S140, determining the similarity between any two pieces of data in the original data set;

[0062] After the label system is simplified, the next step is to determine the similarity between any two data in the original dataset. This can be achieved by calculating the distance between data vectors (such as Euclidean distance or cosine similarity). This step is crucial for identifying and processing similar data points, which helps to further optimize the accuracy and consistency of labels. For example, if two reviews are very similar in semantics, they will be assigned the same label.

[0063] Step S150: adjusting the label system based on the similarity.

[0064] The final step is to adjust the labeling system based on the similarity between the data. If two pieces of data are found to be very similar but assigned different labels, the system will adjust these labels to ensure consistency. This involves re-evaluating and adjusting representative labels, or introducing new labels when necessary to better reflect the actual content of the data. This step ensures that the labeling system can flexibly adapt to the actual characteristics of the data and improve the accuracy and reliability of the annotation.

[0065] Through these steps, the automatic labeling method not only improves the efficiency of data processing, but also enhances the accuracy and consistency of labeling, providing a solid foundation for subsequent data analysis and machine learning tasks.

[0066] In some embodiments, the natural language processing technology includes: technology corresponding to the llama3.1 model of DSPy technology.

[0067] DSPy is a framework that uses language models and retrieval models to solve high-level tasks. It allows for rapid iteration to build modular AI systems and provides algorithms to optimize their cues and weights. The core advantages of the DSPy framework are:

[0068] Modular AI system building: DSPy allows builders to quickly iterate and build modular AI systems, which makes the system more flexible and scalable.

[0069] Optimizing hints and weights: DSPy provides algorithms to optimize hints and weights of language models, which are crucial to improving model performance and adapting them to specific tasks.

[0070] Integrated retrieval model: DSPy is able to integrate retrieval models, which is very useful for tasks that require a lot of background knowledge or data, such as question answering systems and information retrieval.

[0071] Llama3.1 is a large language model open sourced by Meta (Facebook's predecessor company), with pre-trained and instruction-fine-tuned language models, with 8B and 70B parameters. It is built on the latest advances in NLP technology and claims to have high accuracy in understanding and responding to complex queries. The key features of the Llama3.1 model include:

[0072] Pre-trained and fine-tuned: The Llama 3.1 model is pre-trained and fine-tuned, enabling it to understand and respond to complex natural language queries.

[0073] High Accuracy: Llama3.1 shows high accuracy when processing complex queries, thanks to its large number of parameters and advanced training techniques.

[0074] Open source available: As an open source model, Llama 3.1 can be freely used and modified by researchers and developers to adapt to various NLP tasks.

[0075] Combining DSPy technology and Llama3.1 models enables powerful applications in the field of natural language processing, such as building efficient question-answering systems, information retrieval systems, and text classifiers. This combination of technologies provides a powerful toolset for processing and understanding large amounts of natural language data and generating useful insights and responses.

[0076] Furthermore, a clustering algorithm is applied to perform cluster analysis on the generated initial tags, and the initial tags are grouped based on the similarity or correlation between the tags, including:

[0077] Determine the similarity between any two initial labels;

[0078] For each initial tag, determine the n other initial tags with the highest similarity to the initial tag; accumulate the similarities between the n other initial tags and the initial tag to obtain a score corresponding to the initial tag;

[0079] Filter out the m initial labels with the highest scores as the target initial labels;

[0080] For each target initial label, the target initial label and n other initial labels with the highest similarity to the target initial label are determined to form a group of labels to achieve initial label grouping.

[0081] In this step, we use natural language processing techniques to calculate the similarity between any two initial labels. This can be achieved through a variety of algorithms, such as cosine similarity, LevenshteinDistance, or Word2Vec. Cosine similarity is a common method that determines the similarity between vectors by calculating the ratio of the dot product of two vectors and the product of their modulos. Word2Vec is a technique that converts text into vectors, and these vectors can be used to calculate the similarity between texts.

[0082] For each initial label, we need to find the n other initial labels that are most similar to it. This can be done by comparing the similarity scores of all label pairs. For example, if we use cosine similarity, we can calculate the cosine similarity between a label vector and all other label vectors and select the n labels with the highest similarity scores.

[0083] Once we have determined the n labels with the highest similarity to each initial label, the next step is to accumulate the similarity scores between these labels and the initial labels. This summed score will represent the "centrality" or "representativeness" of each initial label, with the higher the score, the more representative the label is in the cluster.

[0084] After accumulating the scores, we select the m initial labels with the highest scores as the target initial labels. These labels will be used in the subsequent clustering process because they are the most representative in their respective clusters.

[0085] Finally, for each target initial label, we again determine the n other initial labels with the highest similarity, and group these labels together with the target initial label to achieve grouping of initial labels. This step helps to form more compact and homogeneous clusters, each of which is centered around one or more representative labels.

[0086] Through these steps, we can effectively use natural language processing technology to organize and simplify the label system, providing clearer and more accurate label classification for subsequent data analysis and machine learning tasks.

[0087] Specifically, determining the similarity between any two initial labels includes: determining the similarity between the word vectors corresponding to the two initial labels as a first similarity value; determining the similarity between the data corresponding to the two initial labels as a second similarity value; and determining that the weighted sum of the first similarity value and the second similarity value is the similarity between the two initial labels.

[0088] Determining the similarity between any two initial labels is a critical step in natural language processing, especially when using clustering algorithms to group labels. Here is a detailed description of this step:

[0089] First similarity value: similarity between word vectors

[0090] 1. Word vector generation: First, we need to convert each initial label into a word vector. This can be achieved through pre-trained word embedding models (such as Word2Vec, GloVe or BERT), which can map words to vectors in high-dimensional space while retaining the semantic information of the words.

[0091] 2. Similarity calculation: Once we have the word vectors of the two initial tags, we can use a metric such as cosine similarity to calculate the similarity between them. Cosine similarity determines the similarity between vectors by calculating the ratio of the dot product of two vectors and the product of their modulos.

[0092] Second similarity value: similarity between data

[0093] 1. Data representation: In addition to word vectors, we can also calculate the similarity between two initial tags based on raw data (such as text data). This may involve converting the entire text data into a vector representation, such as using models such as TF-IDF, LDA, or BERT.

[0094] 2. Similarity calculation: For the data corresponding to two initial labels, we can also use cosine similarity or other similarity measurement methods to calculate the similarity between them.

[0095] Weighted sum: combination of the first similarity value and the second similarity value

[0096] 1. Weight distribution: When calculating the total similarity between two initial tags, we may need to weight the first similarity value (word vector similarity) and the second similarity value (data similarity). The weight distribution can be determined according to the specific application scenario and data characteristics. For example, if word vector similarity is more important, we can assign it a higher weight.

[0097] 2. Weighted sum calculation: The final similarity is the weighted sum of the first similarity value and the second similarity value. This can be expressed by the following formula: Total similarity = w_1 first similarity value + w_2 second similarity value, where w_1 and w_2 are the weights of the first similarity value and the second similarity value respectively.

[0098] In this way, we can comprehensively consider the semantic similarity and data similarity of tags to obtain a more comprehensive similarity measure. This comprehensive similarity measure is very useful for clustering algorithms because it can help the algorithm more accurately identify and group similar tags, thereby improving the quality and practicality of clustering results.

[0099] Furthermore, determining the similarity between any two data in the original data set specifically includes: calculating the cosine similarity of the any two data based on the gte-large model and the milvus database.

[0100] The following is an explanation of the specific steps to determine the similarity between any two data in the original data set:

[0101] Step 1: Generate word vectors using the gte-large model

[0102] First, we use the gte-large model to convert each piece of data in the original dataset into a word vector. The gte-large model can understand the text content and extract key information, mapping the text to vectors in a high-dimensional space that can capture the semantic information of the vocabulary.

[0103] Step 2: Store word vectors in the Milvus database

[0104] Next, we store these word vectors in the Milvus database. Milvus is an open source vector database that is specifically designed for storing and retrieving large-scale vector data. In Milvus, we can define a collection and specify the data type for each field, including the dimension of the word vector.

[0105] Step 3: Create indexes to improve query efficiency

[0106] In order to improve query efficiency, Milvus supports creating indexes for vector fields. We can specify the index type (such as IVF_FLAT) and parameters (such as nlist) for the vector field to create an index suitable for our data.

[0107] Step 4: Calculate cosine similarity

[0108] In Milvus, cosine similarity is a commonly used similarity measure. Cosine similarity measures the similarity between two vectors by calculating the cosine value of the angle between them.

[0109] Step 5: Perform a similarity query

[0110] Finally, we can use Milvus to perform similarity query. By specifying the query vector and similarity threshold, Milvus can quickly retrieve the data vector with the highest similarity to the query vector.

[0111] Through these steps, we can effectively use the gte-large model and Milvus database to determine the similarity between any two data in the original dataset, providing strong support for natural language processing tasks.

[0112] Furthermore, the adjusting the label system based on the similarity includes:

[0113] Determine the two data with cosine similarity greater than 0.9 as the target data pair; verify whether the two data corresponding to the target data pair belong to the same label under the simplified label system; if not, check again based on DSPy's llama3.1 model whether the two data corresponding to the target data pair should belong to the same label; if so, modify the labels of the two data corresponding to the target data pair.

[0114] In some embodiments, if the llama3.1 model based on DSPy determines that the two data corresponding to the target data pair should belong to the same label again, and it is determined that they do not belong to the same label, then training data is generated based on the target data pair; the training data is used to retrain the gte-large model. Based on the target data pair, training data is generated; the training data is used to retrain the gte-large model.

[0115] The solution provided by this application is further described below in conjunction with specific embodiments:

[0116] In order to optimize the processing flow of unlabeled data and improve the classification effect, we can systematically adopt the following steps to integrate and refine the dataset, combining the characteristics of dspy's llama3.1 model and gte-large model:

[0117] 1. Preliminary annotation and label generation

[0118] Overview of steps:

[0119] Using DSPy's llama3.1 model: Perform a preliminary analysis of the unlabeled dataset and generate an initial label for each sentence. These labels allow for repetition to capture the diversity and potential ambiguity in the data.

[0120] Label diversity: Ensure the diversity of labels to cover a wide range of business topics and categories in the data, providing a rich information foundation for subsequent processing.

[0121] 2. Label clustering and merging

[0122] Overview of steps:

[0123] Apply clustering algorithm: Perform cluster analysis on the initially generated tags and group them based on the similarity or correlation between the tags.

[0124] Select representative labels: Select the most representative label from each cluster to replace all labels in the cluster, thereby simplifying the label system and reducing redundancy.

[0125] Alleviate the small sample problem: By clustering and merging, the sample size of each category is increased, which helps to alleviate the classification difficulties caused by scarce samples.

[0126] Detailed description of the joint clustering algorithm:

[0127] (1) Using M3E as the word vector, find the similarity L between any two label vectors;

[0128] (2) Using the similarity of the words in the sentence, find the similarity K between any two label vectors;

[0129] (3) Vector similarity L + vector similarity K = total similarity H between any two labels;

[0130] (4) Calculate any label A, find the 8 labels with the closest total similarity H, add them together, and finally get a total score;

[0131] (5) Sort the total scores from high to low and keep the top 50. See the following table:

[0132] Table 1: Initial label grouping table

[0133]

[0134]

[0135] (6) For the first 50, in each label group, select label A as the group label, and all labels in the same group are changed to label A;

[0136] (7) In the above example, "complaints and dissatisfaction with the service" replaces the other tags in the previous Text;

[0137] (8) The above work realizes label clustering and deduplication;

[0138] (9) Obtain the processed data set (S)

[0139] 3. Data distillation and similarity determination

[0140] Steps to refine:

[0141] Similarity calculation: Use the untrained gte-large model and the milvus database to calculate the cosine similarity of each pair of sentences in the data set (S), paying special attention to those sentence pairs with a cosine similarity greater than 0.9.

[0142] Category consistency check:

[0143] 1. For high similarity sentence pairs (A and B), first check whether they have been classified into the same category.

[0144] 2. If they are not of the same type, use DSPy's llama3.1 model to evaluate again whether they should belong to the same category.

[0145] 2.1 If they are confirmed to be of the same type, sentence B will be classified into the category of A.

[0146] 2.2 If they are not of the same type, construct a triple consisting of sentence A, a sentence of the same type as A, and sentence B as the training sample G.

[0147] 3. Iterative distillation: Put the sorted training samples G into the gte-large model for comparative learning training, and repeat the above similarity calculation and category consistency verification process at least twice to further refine the data set. Through this process, sentences with similar semantics are more closely clustered, while sentences with obvious semantic differences maintain sufficient differentiation.

[0148] 4. Produce optimized data sets

[0149] Final result: After the above steps, the data set not only has a significantly improved classification effect, but also has a clearer logical relationship and semantic hierarchy between internal data. The output data set will become a high-quality resource with accurate classification and reasonable structure, providing solid support for subsequent natural language processing tasks.

[0150] Through the above structured processing flow, we can effectively utilize the powerful capabilities of dspy's llama3.1 and gte-large models to extract high-quality and high-value classification datasets from unlabeled raw data.

[0151] Exemplary Devices

[0152] The device embodiments of the present application can be used to execute the method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.

[0153] Figure 3 FIG. 1 is a block diagram of an automatic labeling device based on a large model and a clustering algorithm provided by an embodiment of the present application. Figure 3 As shown, the device comprises:

[0154] A preliminary analysis module 31 is used to perform preliminary analysis on the original data set using natural language processing technology and generate an initial label for each piece of data in the original data set;

[0155] A clustering module 32 is used to apply a clustering algorithm to perform cluster analysis on the generated initial tags, and group the initial tags based on the similarity or correlation between the tags to obtain multiple initial tag groups;

[0156] A simplification module 33, used to select the most representative label from each initial label group to replace all labels in the initial label group, so as to simplify the label system and reduce redundancy;

[0157] The judging module 34 is used to judge the similarity between any two data in the original data set; the adjusting module 35 is used to adjust the label system based on the similarity.

[0158] Exemplary Electronic Devices

[0159] Below, reference Figure 4 To describe an electronic device according to an embodiment of the present application. Figure 4 A block diagram of an electronic device according to an embodiment of the present application is illustrated.

[0160] like Figure 4 As shown, electronic device 400 includes one or more processors 410 and memory 420 .

[0161] The processor 410 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 400 to perform desired functions.

[0162] The memory 420 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 410 may run the program instructions to implement the automatic annotation method based on the large model and clustering algorithm of each embodiment of the present application described above and / or other desired functions. Various contents such as category correspondences may also be stored in the computer-readable storage medium.

[0163] In one example, the electronic device 400 may further include: an input device 430 and an output device 440 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0164] In addition, the input device 430 may also include, for example, a keyboard, a mouse, an interface, etc. The output device 440 may output various information to the outside, including analysis results, etc. The output device 440 may include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, etc.

[0165] Of course, to simplify, Figure 4 Only some of the components in the electronic device related to the present application are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, the electronic device may further include any other appropriate components according to specific application conditions.

[0166] Exemplary computer program products and computer-readable storage media

[0167] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the automatic labeling method based on a large model and a clustering algorithm according to various embodiments of the present application described in the above "Exemplary Method" section of this specification.

[0168] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0169] In addition, an embodiment of the present application may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, causes the processor to execute the steps of the automatic labeling method based on a large model and a clustering algorithm according to various embodiments of the present application described in the above “Exemplary Method” section of this specification.

[0170] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0171] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. An automatic labeling method based on a large model and a clustering algorithm, characterized in that: include: Use natural language processing technology to perform preliminary analysis on the original data set and generate an initial label for each piece of data in the original data set; Applying a clustering algorithm to perform cluster analysis on the generated initial labels, grouping the initial labels based on the similarity or correlation between the labels, and obtaining multiple initial label groups; Select the most representative label from each initial label group to replace all labels in the initial label group to simplify the label system and reduce redundancy; Determine the similarity between any two pieces of data in the original data set; The labeling system is adjusted based on the similarity.

2. The automatic labeling method based on a large model and a clustering algorithm according to claim 1, characterized in that: The natural language processing technology includes: technology corresponding to the llama3.1 model of DSPy technology.

3. The automatic labeling method based on a large model and a clustering algorithm according to claim 1, characterized in that: Apply a clustering algorithm to perform cluster analysis on the generated initial tags, and group the initial tags based on the similarity or correlation between the tags, including: Determine the similarity between any two initial labels; For each initial tag, determine the n other initial tags with the highest similarity to the initial tag; accumulate the similarities between the n other initial tags and the initial tag to obtain a score corresponding to the initial tag; Filter out the m initial labels with the highest scores as the target initial labels; For each target initial label, the target initial label and n other initial labels with the highest similarity to the target initial label are determined to form a group of labels to achieve initial label grouping.

4. The automatic labeling method based on a large model and a clustering algorithm according to claim 3 is characterized in that: Determine the similarity between any two initial labels, including: Determine the similarity between the word vectors corresponding to the two initial labels as the first similarity value; Determine the similarity between the data corresponding to the two initial labels as a second similarity value; It is determined that a weighted sum of the first similarity value and the second similarity value is the similarity between the two initial labels.

5. The automatic labeling method based on a large model and a clustering algorithm according to claim 4, characterized in that: The determining the similarity between any two pieces of data in the original data set includes: Calculate the cosine similarity of any two data based on the gte-large model and milvus database.

6. The automatic labeling method based on a large model and a clustering algorithm according to claim 5, characterized in that: The adjusting the label system based on the similarity includes: Determine the two data with cosine similarity greater than 0.9 as the target data pair; Verify whether the two pieces of data corresponding to the target data pair belong to the same label under the simplified label system; If not, the llama3.1 model based on DSPy again targets the data pair to determine whether the two corresponding data should belong to the same label; If so, modify the labels of the two data corresponding to the target data pair.

7. The automatic labeling method based on a large model and a clustering algorithm according to claim 6, characterized in that: If the llama3.1 model based on DSPy determines whether the two data corresponding to the target data pair should belong to the same label again, and it is determined that they do not belong to the same label, then training data is generated based on the target data pair; The training data is used to retrain the gte-large model.

8. The automatic labeling method based on a large model and a clustering algorithm according to claim 6, characterized in that: After modifying the labels of the two data corresponding to the target data pair, it also includes: Based on the target data pair, training data is generated; the training data is used to retrain the gte-large model.

9. An automatic labeling device based on a large model and a clustering algorithm, characterized in that: include: A preliminary analysis module is used to perform preliminary analysis on the original data set using natural language processing technology and generate an initial label for each piece of data in the original data set; A clustering module is used to apply a clustering algorithm to perform cluster analysis on the generated initial labels, and to group the initial labels based on the similarity or correlation between the labels to obtain multiple initial label groups; A simplification module, used to select the most representative label from each initial label group to replace all labels in the initial label group, so as to simplify the label system and reduce redundancy; The judgment module is used to judge the similarity between any two data in the original data set; An adjustment module is used to adjust the label system based on the similarity.

10. An electronic device, characterized in that: include: A processor, and a memory for storing a program executable by the processor; The processor is used to implement the automatic labeling method based on a large model and a clustering algorithm as described in any one of claims 1 to 8 by running the program in the memory.

Images