Intelligent labeling method and device for text data and medium

Abstract

The invention discloses an intelligent annotation method and device for text data and a medium, and belongs to the technical field of big data annotation and processing. The method comprises the steps that to-be-labeled data is obtained and preprocessed, and the to-be-labeled data at least comprises text data; for the preprocessed to-be-labeled data, extracting non-duplicated words to construct a word list; calculating a word frequency TF and an inverse document frequency IDF corresponding to the to-be-labeled data based on the word list, so as to construct a TF-IDF vector of the to-be-labeled data through the word frequency TF and the inverse document frequency IDF; and inputting the TF-IDF vector into a pre-trained intelligent labeling model, and obtaining a labeling result of the to-be-labeled data through an output result of the intelligent labeling model. By means of the method, the data labeling efficiency and accuracy are improved while the labeling requirements of various services for high-quality and large-scale data are met.

Description

Technical Field

[0001] The present application relates to the technical field of big data annotation and processing, and in particular to a method, device and medium for intelligent annotation of text data. Background Art

[0002] In the current era of big data and internet information, information is exploding daily, its form and content constantly changing. Text data, in particular, is enormous in volume, possessing immense potential and practical value across business, politics, and society. Typical text data types include policy documents, internet social media data, news media data, e-commerce platform data, and academic literature. In the business world, text data can help businesses understand customer needs and preferences, and capture market trends and consumer needs. In the political realm, text data can help governments understand public opinion and sentiment, enabling timely policy adjustments and improvements.

[0003] Government departments at all levels will continue to issue various policy documents covering a wide range of areas, including the economy, society, environment, and education. These documents are all unstructured information, characterized by large amounts of data, disorganized data, and diverse data types. Policy data is highly valuable. By obtaining timely, comprehensive, and accurate policy information, businesses can stay abreast of policy trends, grasp policy directions, apply for relevant rewards and subsidies or qualifications, and adjust operational decisions in a timely manner to enhance the effectiveness of digital transformation. Furthermore, in the past two years, big model technology has flourished, with major manufacturers launching big model product tools. The demand for policy data annotation by big model manufacturers continues to rise. Both training new big models and optimizing and updating existing models require large amounts of high-quality annotated data.

[0004] On the one hand, some local governments have built policy public service platforms, which often select important policies and then manually select them based on tags set by policy-making departments during back-end maintenance. This is a relatively traditional approach, and the annotation content varies widely, with different standards in each location, making it difficult to use directly. On the other hand, the requirements for data annotation personnel are high. This poses a significant challenge to the professionalism and understanding of annotation personnel. To ensure the annotation quality of large-scale datasets, personnel with professional backgrounds are often required to annotate the datasets. In addition, manual data annotation is inefficient. To ensure the quality of data annotation, it is necessary to establish corresponding annotation processes and methods for annotation quality verification. At the same time, to ensure the implementation and execution of these processes and methods, appropriate reward mechanisms must be added and integrated with personnel management. This also results in high labor costs for current text data annotation solutions, as well as low annotation efficiency and accuracy. Summary of the Invention

[0005] The present application provides a method, device and medium for intelligent annotation of text data, which are used to solve at least one of the above technical problems.

[0006] This application adopts the following technical solutions:

[0007] In the first aspect, the present application provides a method for intelligent annotation of text data, the method comprising: obtaining data to be annotated and preprocessing the data to be annotated, the data to be annotated comprising at least text data; extracting non-repeated words from the preprocessed data to be annotated to construct a vocabulary; calculating the term frequency TF and inverse document frequency IDF corresponding to the data to be annotated based on the vocabulary, so as to construct a TF-IDF vector of the data to be annotated through the term frequency TF and inverse document frequency IDF; inputting the TF-IDF vector into a pre-trained intelligent annotation model, and obtaining the annotation result of the data to be annotated through the output result of the intelligent annotation model.

[0008] In one possible implementation of the present application, the data to be labeled is preprocessed, including: removing noise data and duplicate data in the data to be labeled, the noise data including at least one or more of spaces, tabs, line breaks and garbled characters; removing stop words in the data to be labeled, and performing word segmentation on the data to be labeled to obtain a vocabulary.

[0009] In a possible implementation of the present application, the term frequency TF and the inverse document frequency IDF corresponding to the data to be labeled are calculated based on the vocabulary, including: for any word in the vocabulary, calculating the frequency of occurrence of any word in the data to be labeled; dividing the frequency by the total number of words in the data to be labeled to obtain the term frequency TF; for any word in the vocabulary, determining the number of documents in the data to be labeled that contain the any word; obtaining the total number of documents corresponding to the data to be labeled, and dividing the total number of documents by the number of documents containing the any word to obtain a calculation result; taking the logarithm of the calculation result to obtain the inverse document frequency IDF.

[0010] In one possible implementation of the present application, the training process of the intelligent labeling model includes: obtaining training data, and determining the labeling categories corresponding to the training data and the category labels corresponding to each labeling category, the labeling categories at least including classification labels and sequence labels, the category labels of the classification labels at least including industry classification and file classification, and the category labels of the sequence labels at least including date, region and institution; calculating the TF-IDF vector corresponding to the training data; inputting the TF-IDF vector and the labeling categories into a support vector machine SVM model, training the SVM model, and obtaining the intelligent labeling model.

[0011] In one possible implementation of the present application, the SVM model is trained, including: setting a penalty parameter for controlling the degree of penalty for misclassified samples, the value of the penalty parameter being 1; setting a linear kernel function for mapping the TF-IDF vector to a high-dimensional space; according to the penalty parameter and the linear kernel function, based on the TF-IDF vector and the category label corresponding to the labeled category, finding the optimal classification hyperplane through a sequential minimum optimization algorithm; during the search process, adjusting the coefficients of the SVM model to minimize the classification error until the SVM model meets the classification performance requirements, thereby obtaining the intelligent labeling model, and the classification performance requirements at least include minimizing the classification error within a preset range.

[0012] In one possible implementation of the present application, after the SVM model is trained, the method further includes: using verification data and / or test data to perform performance evaluation on the trained intelligent labeling model, including calculating the performance score of the model based on pre-selected evaluation indicators, the evaluation indicators including at least one or more of accuracy, recall rate and F1 value; when the performance score is lower than a preset score threshold, adjusting the penalty parameter and / or linear kernel function of the SVM model.

[0013] In one possible implementation of the present application, obtaining training data includes: collecting a number of text data and manually annotating the text data; the manual annotation is implemented through a web interface, and the web interface at least includes an annotation category and a category label corresponding to the annotation category; during the manual annotation process, the text data are allocated in sequence according to the time priority principle.

[0014] In a possible implementation of the present application, the method further includes: performing data segmentation on the training data according to a preset ratio to obtain the verification data and / or test data, and the preset ratio is 7:3 and / or 7:1:2.

[0015] In the second aspect, the present application also provides an intelligent tagging device for text data, the device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, so that the at least one processor can execute: obtaining data to be labeled, and preprocessing the data to be labeled, the data to be labeled at least including text data; extracting non-repeating words from the preprocessed data to be labeled to construct a vocabulary; calculating the term frequency TF and inverse document frequency IDF corresponding to the data to be labeled based on the vocabulary, so as to construct a TF-IDF vector of the data to be labeled through the term frequency TF and inverse document frequency IDF; inputting the TF-IDF vector into a pre-trained intelligent tagging model, and obtaining the tagging result of the data to be labeled through the output result of the intelligent tagging model.

[0016] In a third aspect, the present application also provides a non-volatile computer storage medium having computer executable instructions stored thereon, wherein the computer executable instructions are configured to execute: obtaining data to be labeled and preprocessing the data to be labeled, wherein the data to be labeled at least includes text data; extracting non-repeating words from the preprocessed data to be labeled to construct a vocabulary; calculating the term frequency TF and inverse document frequency IDF corresponding to the data to be labeled based on the vocabulary, so as to construct a TF-IDF vector of the data to be labeled through the term frequency TF and inverse document frequency IDF; inputting the TF-IDF vector into a pre-trained intelligent labeling model, and obtaining the labeling result of the data to be labeled through the output result of the intelligent labeling model.

[0017] The present application provides a method, device, and medium for intelligent annotation of text data, which have the following beneficial effects:

[0018] This application pre-processes the data to be annotated, constructs a corresponding vocabulary, and then constructs a TF-IDF vector for the annotated data based on the vocabulary. The TF-IDF vector is used as the input of the intelligent annotation model, and the annotation results of the data to be annotated are output through the pre-trained intelligent annotation model. It realizes the intelligent structuring and labeling of large-scale text data such as policy data, and can separately annotate the different regions, industries and other elements involved in it. While supporting the requirements of various businesses for high-quality large-scale data, it improves the efficiency and accuracy of annotation and reduces labor costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in this application or the prior art, the following briefly introduces the drawings required for the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments described in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0020] Figure 1 A flowchart of an intelligent annotation method for text data provided in this application;

[0021] Figure 2 A schematic diagram of the structure of an intelligent annotation device for text data provided in this application. DETAILED DESCRIPTION

[0022] In order to enable those skilled in the art to better understand the technical solutions of this application, the technical solutions of this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0023] The method in this application is described in detail below with reference to the accompanying drawings.

[0024] Figure 1 This application provides a flowchart of an intelligent annotation method for text data, such as Figure 1 As shown, the intelligent annotation method in this application includes at least the following execution steps:

[0025] Step 101: Obtain the data to be labeled and preprocess the data to be labeled.

[0026] The data to be annotated in this application is preferably large text data such as policy documents. After obtaining these data to be annotated, they need to be preprocessed first. In one possible implementation method, the preprocessing process includes denoising, deduplication and word segmentation. The noise data and repeated data such as spaces, tabs, line breaks and garbled characters in the annotated data are removed, and then the stop words in the text are removed. After that, the obtained data is segmented to obtain a word list that does not contain repeated words. It should be noted that the word segmentation process in this process can be implemented by existing word segmentation technology or algorithms, which will not be elaborated in this application.

[0027] Step 102: extract non-repeated words from the pre-processed data to be annotated and construct a vocabulary.

[0028] After the aforementioned processing, the data to be annotated is actually a group of words. Since there are repeated words in these words, it will affect the subsequent annotation results. Therefore, the non-repeated words are extracted to construct a vocabulary, and a vocabulary containing only non-repeated words is obtained.

[0029] Step 103: Calculate the term frequency TF and the inverse document frequency IDF corresponding to the data to be annotated based on the vocabulary, so as to construct a TF-IDF vector of the data to be annotated through the term frequency TF and the inverse document frequency IDF.

[0030] In one possible implementation of the present application, for any word in the vocabulary, the frequency of occurrence of any word in the data to be labeled is calculated, and the frequency is divided by the total number of words in the data to be labeled to obtain the word frequency TF; then the number of documents containing any word in the data to be labeled is determined, the total number of documents corresponding to the data to be labeled is obtained, and the total number of documents is divided by the number of documents containing any word, and then the logarithm of the calculated result is obtained to obtain the inverse document frequency IDF; finally, the word frequency TF and the inverse document frequency IDF are used to construct the TF-IDF vector corresponding to the data to be labeled. The construction process of the TF-IDF vector here is the same or similar to the TF-IDF vector construction process involved in the intelligent labeling model training process below. For details, please refer to the relevant description below, and this application will not elaborate on it here.

[0031] Step 104: Input the TF-IDF vector into a pre-trained intelligent labeling model, and obtain the labeling result of the data to be labeled through the output result of the intelligent labeling model.

[0032] The TF-IDF vector obtained in the above process is input into the pre-trained intelligent labeling model. The model outputs the labeling results corresponding to the data to be labeled, completing the intelligent labeling process of the data.

[0033] In one possible implementation of the present application, the training process of the aforementioned intelligent labeling model is as follows:

[0034] First, data collection and preprocessing are performed. For example, using policy documents as text data, these documents are derived from public policy information from government departments at all levels and are generally published through websites. The main information collected includes the policy title, issuing authority, document number, publication date, text, and attachment links. After collecting the file data, noise data is removed from the text, such as extra spaces, tabs, line breaks, and other formatting issues, as well as meaningless special symbols and garbled text. Duplicate data records are then processed. If identical text exists, a copy is retained or combined for statistical analysis, depending on the specific situation, to avoid unnecessary impacts of data redundancy on subsequent model training. Finally, simple rules are used to preliminarily divide the obtained text data into policy categories, facilitating more detailed labeling and model training.

[0035] Further, after processing the collected text data, clearly label the categories, including classification labeling: identifying various classifications, such as industrial classifications, document categories, etc.; sequence labeling: identifying various dates, region names, institution names, etc.; and formulating labeling specifications: defining the connotations, extensions, and specific standards of each labeled category.

[0036] Secondly, perform vectorization processing on the collected text data. This process includes:

[0037] 1) Construct a vocabulary: Collect the words in all text data, remove stop words (such as common words like "of", "is", "in", etc. that are not very helpful for semantic discrimination), and form a vocabulary containing all non-repeating words.

[0038] 2) Calculate the term frequency (TF): Count the number of occurrences of each word in each text, and then divide this frequency by the total number of words in the text to obtain the relative term frequency of the word in this text.

[0039] 3) Calculate the inverse document frequency (IDF): First, count the number of documents containing a certain word, then divide the total number of documents by the number of documents containing the word, and then take the logarithm, that is, take the log function, to obtain the inverse document frequency.

[0040] 4) Construct a TF-IDF vector: Multiply the term frequency (TF) of each word by the inverse document frequency (IDF) to obtain the TF-IDF value of the word in the text, and construct the TF-IDF vector of the text in this way, which can not only reflect the frequency of the word in the text but also reflect its rarity in the entire document set, highlighting the key features of the text.

[0041] Then, construct a training data set. Select a part of the pre-classified data and store this part of the data in a data table in the database. Divide the text data after vectorization processing into a training set, a validation set, and a test set according to a ratio. Select 7:1:2 (70% for training, 10% for validation, and 20% for testing), so that each part of the data can reasonably cover text samples of different types and features and maintain the consistency of the data distribution. Develop a WEB program application to display the data to be labeled and trained and support multi-person online labeling. In the scenario of multi-person labeling, in order to handle the conflicts of labeled data, it is assigned to the corresponding person according to the time priority principle of the labeled data; at the same time, the labeling page includes the original data, target label 1, target label 2, target label 3, target label N, etc. Each label is set in the way of dropdown + customization. The content of the dropdown box follows the labeling specifications to ensure the consistency of information as much as possible. After the user labels a piece of data and selects to submit, they can choose the next data for labeling; they can also return to the previous data for modification operations.

[0042] Finally, the support vector machine (SVM) model is combined with the TF-IDF algorithm to train the model, which includes the following steps:

[0043] 1) Parameter Initialization: Set the penalty parameter C to control the degree of penalty for misclassified samples. A larger C value results in a heavier penalty for misclassification, and the model tends to classify all training samples as correctly as possible, but this may lead to overfitting. A smaller C value results in a higher tolerance for misclassification, which may lead to underfitting. This application prefers to start model training and tuning with C = 1.

[0044] 2) Select a linear kernel function to map the original text data, preferably the vector data after TF-IDF vector conversion, into a high-dimensional space to better find the dividing boundaries between different categories.

[0045] 3) The SVM model is trained using the training set data, namely the TF-IDF vector representation of the text data and the corresponding annotated category labels. The training set samples are sequentially input into the SVM model. The model uses the Sequential Minimization Optimization (SMO) algorithm, based on the set kernel function and parameters, the TF-IDF vector features of the training samples, and the annotated category labels, to find the optimal classification hyperplane. This algorithm separates samples of different annotated categories as accurately as possible. This process continuously adjusts the coefficients corresponding to the support vectors to minimize classification error. This is achieved using the hinge loss function. The training process requires multiple rounds of iteration until the model's performance indicators on the training set reach a relatively stable state.

[0046] 4) Use the validation set to evaluate and adjust parameters: Use the validation set data to evaluate the performance of the trained intelligent labeling model. Use pre-selected evaluation metrics, such as accuracy, precision, recall, and F1 value, to check the model's performance. If the performance is poor, adjust the penalty parameter C.

[0047] 5) Test set testing: Use a trained and fine-tuned intelligent labeling model to predict labels for the test set data. Specifically, the test set text undergoes the same TF-IDF vectorization process, converts it into vector form, and inputs it into the model. The model outputs the corresponding label category prediction result. The model's performance score is then calculated based on the classification task's corresponding evaluation metrics, such as accuracy and F1 value, to intuitively understand the model's labeling accuracy, effectiveness, and labeling capabilities on unknown data.

[0048] Finally, the trained and performance-compliant intelligent tagging model can be applied to actual text data tagging tasks. The new text to be labeled can be vectorized using the same steps, including building a vocabulary and calculating the TF-IDF value. The obtained TF-IDF vector is then input into the intelligent tagging model. The model will output the corresponding tagging results, achieving the purpose of intelligent tagging and completing the intelligent tagging process of text data.

[0049] The above-mentioned intelligent labeling solution in this application uses machine learning algorithms to achieve intelligent structuring and labeling of large-scale text data such as policy data, and separately label the different regions, industries and other elements involved. While supporting the requirements of various businesses for high-quality large-scale data, it improves the efficiency and accuracy of text data labeling and reduces labor costs.

[0050] Based on the same inventive concept, the present application also provides an intelligent annotation device for text data, the structure of which is as follows: Figure 2 shown.

[0051] Figure 2 This is a schematic diagram of the structure of an intelligent annotation device for text data provided by this application. Figure 2 As shown, the intelligent annotation device 200 for text data in the present application specifically includes: at least one processor 201; and a memory 203 that is communicatively connected to the at least one processor 201 (connected via a bus 202); wherein the memory 203 stores instructions that can be executed by the at least one processor 201, so that the at least one processor 201 can execute a method for intelligent annotation of text data as described in the above embodiment.

[0052] In one possible implementation of the present application, the aforementioned processor 201 is used to execute, obtain data to be labeled, and preprocess the data to be labeled, wherein the data to be labeled at least includes text data; for the preprocessed data to be labeled, extract non-repeated words to construct a vocabulary; calculate the term frequency TF and inverse document frequency IDF corresponding to the data to be labeled based on the vocabulary, so as to construct the TF-IDF vector of the data to be labeled through the term frequency TF and inverse document frequency IDF; input the TF-IDF vector into a pre-trained intelligent labeling model, and obtain the labeling result of the data to be labeled through the output result of the intelligent labeling model.

[0053] In addition, the present application also provides a non-volatile computer storage medium having computer-executable instructions stored thereon, wherein the computer-executable instructions are configured to execute: obtaining data to be labeled and preprocessing the data to be labeled, wherein the data to be labeled at least includes text data; extracting non-repeating words from the preprocessed data to be labeled to construct a vocabulary; calculating the term frequency TF and inverse document frequency IDF corresponding to the data to be labeled based on the vocabulary, so as to construct a TF-IDF vector of the data to be labeled through the term frequency TF and inverse document frequency IDF; inputting the TF-IDF vector into a pre-trained intelligent labeling model, and obtaining the labeling result of the data to be labeled through the output result of the intelligent labeling model.

[0054] The various embodiments in this application are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the device embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the description of the method embodiments.

[0055] The device and method provided in this application correspond one to one, so the device also has similar beneficial technical effects as its corresponding method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the device will not be repeated here.

[0056] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.

[0057] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0058] The above are merely embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included in the protection scope of the present application.

Claims

1. A method for intelligent annotation of text data, characterized in that: The method comprises: Acquire data to be annotated and preprocess the data to be annotated, wherein the data to be annotated at least includes text data; Extracting non-repeated words from the pre-processed data to be annotated and constructing a vocabulary; Calculate the term frequency TF and the inverse document frequency IDF corresponding to the data to be annotated based on the vocabulary, so as to construct a TF-IDF vector of the data to be annotated through the term frequency TF and the inverse document frequency IDF; The TF-IDF vector is input into a pre-trained intelligent labeling model, and the labeling result of the data to be labeled is obtained through the output result of the intelligent labeling model.

2. The intelligent annotation method for text data according to claim 1, characterized in that: Preprocessing the data to be labeled includes: Removing noise data and duplicate data from the data to be annotated, wherein the noise data includes at least one or more of spaces, tabs, line breaks, and garbled characters; Stop words in the data to be annotated are removed, and word segmentation processing is performed on the data to be annotated to obtain a word list.

3. The intelligent annotation method for text data according to claim 2, characterized in that: Calculating the term frequency TF and inverse document frequency IDF corresponding to the data to be annotated based on the vocabulary includes: For any word in the vocabulary, calculate the frequency of occurrence of the word in the data to be annotated; The word frequency TF is obtained by dividing the frequency by the total number of words in the data to be labeled; For any word in the vocabulary, determining the number of documents in the data to be annotated that contain the word; Obtaining the total number of documents corresponding to the data to be annotated, and dividing the total number of documents by the number of documents containing any of the words to obtain a calculation result; The inverse document frequency (IDF) is obtained by taking the logarithm of the calculation result.

4. The intelligent annotation method for text data according to claim 1, characterized in that: The training process of the intelligent labeling model includes: Acquire training data, and determine the annotation categories corresponding to the training data and the category labels corresponding to each annotation category, wherein the annotation categories include at least classification annotation and sequence annotation, the category labels of the classification annotation include at least industry classification and document classification, and the category labels of the sequence annotation include at least date, region, and institution; Calculate the TF-IDF vector corresponding to the training data; The TF-IDF vector and the label category are input into a support vector machine (SVM) model, and the SVM model is trained to obtain the intelligent labeling model.

5. The intelligent annotation method for text data according to claim 4, characterized in that: Training the SVM model includes: Set a penalty parameter to control the degree of penalty for misclassified samples. The value of the penalty parameter is 1. Setting a linear kernel function for mapping the TF-IDF vector to a high-dimensional space; According to the penalty parameter and the linear kernel function, based on the TF-IDF vector and the category label corresponding to the annotated category, an optimal classification hyperplane is found through a sequential minimum optimization algorithm; During the search process, the coefficients of the SVM model are adjusted to minimize the classification error until the SVM model meets the classification performance requirements, thereby obtaining the intelligent labeling model. The classification performance requirements at least include minimizing the classification error within a preset range.

6. The intelligent annotation method for text data according to claim 5, characterized in that: After the SVM model is trained, the method further includes: Performing a performance evaluation on the trained intelligent labeling model using validation data and / or test data, including calculating a performance score of the model based on pre-selected evaluation metrics, where the evaluation metrics include at least one or more of accuracy, recall, and F1 value; When the performance score is lower than a preset score threshold, the penalty parameter and / or the linear kernel function of the SVM model is adjusted.

7. The intelligent annotation method for text data according to claim 4, characterized in that: Get training data, including: Collecting a number of text data and manually annotating the text data; The manual annotation is implemented through a web interface, and the web interface at least includes an annotation category and a category label corresponding to the annotation category; During the manual annotation process, the plurality of text data are allocated in sequence according to a time priority principle.

8. The intelligent annotation method for text data according to claim 6, characterized in that: The method further comprises: The training data is segmented according to a preset ratio to obtain the verification data and / or test data, and the preset ratio is 7:3 and / or 7:1:

2.

9. An intelligent annotation device for text data, characterized in that: The device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, so as to enable the at least one processor to perform: Acquire data to be annotated and preprocess the data to be annotated, wherein the data to be annotated at least includes text data; Extracting non-repeated words from the pre-processed data to be annotated and constructing a vocabulary; Calculate the term frequency TF and the inverse document frequency IDF corresponding to the data to be annotated based on the vocabulary, so as to construct a TF-IDF vector of the data to be annotated through the term frequency TF and the inverse document frequency IDF; The TF-IDF vector is input into a pre-trained intelligent labeling model, and the labeling result of the data to be labeled is obtained through the output result of the intelligent labeling model.

10. A non-volatile computer storage medium, characterized in that Computer-executable instructions are stored thereon, the computer-executable instructions being configured to perform: Acquire data to be annotated and preprocess the data to be annotated, wherein the data to be annotated at least includes text data; Extracting non-repeated words from the pre-processed data to be annotated and constructing a vocabulary; Calculate the term frequency TF and the inverse document frequency IDF corresponding to the data to be annotated based on the vocabulary, so as to construct a TF-IDF vector of the data to be annotated through the term frequency TF and the inverse document frequency IDF; The TF-IDF vector is input into a pre-trained intelligent labeling model, and the labeling result of the data to be labeled is obtained through the output result of the intelligent labeling model.

Images