Text auxiliary analysis and evaluation method

Abstract

The invention relates to a text auxiliary analysis and evaluation method. The method comprises the following steps: S1, data preparation: preprocessing a text, including stop word removal, punctuation mark processing, special character processing and case and small case conversion; s2, establishing a bag-of-words model, converting a text into a word frequency vector, and weighting by using TF-IDF; s3, word embedding: converting each word into a high-dimensional vector by respectively using Word2Vec and GloVe, and then performing result comparison; s4, performing feature selection, and performing screening according to importance or correlation of features to reduce noise and overfitting; s5, selecting a learning model, and importing the features for training; and S6, predicting new text data by using the trained model. According to the method, preprocessing, feature extraction, model training and evaluation are carried out on text data through a series of steps, and finally a model capable of being used for text auxiliary analysis and evaluation is obtained.

Description

Technical Field

[0001] This invention relates to the field of text processing, and in particular to a text-assisted analysis and evaluation method. Background Technology

[0002] Currently, the analysis and evaluation of science and technology policies mainly rely on human resources to collect and analyze data. After various analyses and discussions by experts and government management, an analysis and evaluation report is formed. Due to the vast and complex nature of the science and technology policy system, this work is time-consuming, labor-intensive, highly subjective, and lacks timeliness, making it difficult to identify problems and remedy deficiencies in a timely manner. Summary of the Invention

[0003] In view of this, the present invention aims to propose a text-assisted analysis and evaluation method to solve the problems in the prior art.

[0004] To achieve the above objectives, the technical solution of the present invention is implemented as follows: This invention discloses a text-assisted analysis and evaluation method, which includes the following steps: S1, Data Preparation: Preprocess the text, including removing stop words, handling punctuation marks, handling special characters, and converting between uppercase and lowercase. S2, Build a bag-of-words model, convert the text into word frequency vectors and then use TF-IDF for weighting; S3, word embedding: Each word is converted into a high-dimensional vector using tools such as Word2Vec and GloVe, and then the results are compared. S4, Feature selection: Filter features based on their importance or relevance to reduce noise and overfitting; S5, Select the learning model and import the features for training; S6 uses the trained model to predict new text data.

[0005] Furthermore, in S5, the learning model is a Naive Bayes algorithm, and its training steps are as follows: S511, import the vector features from S4; S512, Construct the TF-IDF vector space for the test set; S513, Import the bag of words from the training set; S514, use TFIDFVectorizer to initialize the vector space model; S515, creates a persistent bag of words.

[0006] Furthermore, in S5, the learning model is a KNN model, and its training steps are as follows: S521, calculate the distance between the test sample and all samples in the training set; S522 uses a voting mechanism to select the class with the most frequent neighbor among the k neighbors as the predicted value of the predicted sample.

[0007] Furthermore, in S521, the sample distance is Euclidean distance or cosine distance.

[0008] Furthermore, in S5, the trained learning model is evaluated using a test set, and cross-validation is used to further evaluate the stability and generalization ability of the learning model.

[0009] Furthermore, the evaluation criteria include accuracy, recall, and F1 score.

[0010] Furthermore, the evaluated model is optimized by adjusting features or learning model parameters based on the evaluation results to improve model performance.

[0011] Furthermore, in S4, the feature selection includes statement extraction of policy topics, text summary generation, entity recognition, and relation extraction.

[0012] Compared with the prior art, the present invention has the following advantages: In this invention, a series of steps are used to preprocess text data, extract features, train the model, and evaluate it, ultimately resulting in a model that can be used for text-assisted analysis and evaluation. This model is flexible, effectively improving the accuracy and efficiency of text analysis. It also saves manpower in science and technology policy analysis and evaluation, avoiding the drawbacks of strong subjectivity in manual analysis. Attached Figure Description

[0013] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings: Figure 1 This is a schematic diagram of the overall process of the present invention. Detailed Implementation

[0014] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0015] In the description of this invention, it should be noted that the terms "upper," "lower," "inner," and "back," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0016] In addition, in the description of the present invention, unless otherwise clearly defined, the terms "installation", "connection", "linkage", and "connector" shall be understood in a broad sense. For example, it may be a fixed connection, a detachable connection, or an integral connection; it may be a mechanical connection or an electrical connection; it may be a direct connection or an indirect connection through an intermediate medium, and it may be the communication inside two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood in combination with specific situations.

[0017] The following will refer to the appended Figure 1 Embodiments will be used to illustrate the present invention in detail.

[0018] Overall, the present invention discloses a method for text-assisted analysis and evaluation, which includes the following steps: S1, Data preparation: Preprocess the text, including removing stop words, punctuation processing, special character processing, and case conversion; further, remove stop words (such as common words like "de", "le" that contribute little to the meaning of the text), punctuation processing (such as removing punctuation or performing normalization), special character processing (such as removing or replacing non-alphabetic characters), and case conversion (usually converting to lowercase for consistency).

[0019] S2, Establish a bag-of-words model, convert the text into a term frequency vector, that is, count the number of times each word appears in the text, regard the text as an unordered set of words, and use the number of occurrences of each word as a feature, and then use TF-IDF for weighting; TF-IDF: Term Frequency-Inverse Document Frequency, which combines term frequency (TF) and inverse document frequency (IDF) to evaluate the importance and discrimination of a word in a document.

[0020] S3, Word embedding, use methods such as Word2Vec and GloVe to convert each word into a high-dimensional vector respectively, and then compare the results; compare the results of different word embedding methods and select the method most suitable for the current task. It should be noted that Word2Vec: A deep learning model that predicts words based on context, which can map words to a high-dimensional vector space and capture the semantic relationships between words; GloVe: A word embedding method based on global word frequency statistics, which can also map words to a high-dimensional vector space.

[0021] S4, Feature selection, screen according to the importance or relevance of features to reduce noise and overfitting; use statistical methods (such as chi-square test, mutual information, etc.) or machine learning algorithms (such as Lasso regression, random forest, etc.) to evaluate the importance or relevance of features, and remove unimportant features. Feature selection may include statement extraction of policy themes, text summary generation, entity recognition, relationship extraction, etc.

[0022] S5, Select the learning model, import the preprocessed and feature-selected text data for training; to extract key information useful for text analysis.

[0023] S6. Use the trained model to predict new text data. Process the new text data according to the same preprocessing procedure and then input it into the trained model. The model outputs the prediction result, which can be a classification label, regression value, etc.

[0024] As a preferred embodiment, in S5, the learning model is a Naive Bayes algorithm, and its training steps are as follows: S511, import the vector features from S5; prior to this step, the data preprocessing and feature selection work from S1 to S4 should have been completed.

[0025] S512, construct the TF-IDF vector space of the test set; set up a preprocessed training set and a test set (although the test set is not directly used to train the model during the training phase, it is used for subsequent work).

[0026] Training set: This provides a set of labeled data, allowing the model to learn the probability distribution of each class and the conditional probability distribution of each feature under a given class. A classification model is built using the training set to predict the class of new data.

[0027] S513, Import the bag of words from the training set; S514, use TFIDFVectorizer to initialize the vector space model; S515, creates a persistent bag of words.

[0028] As another preferred implementation, in S5, the learning model is the KNN model. The KNN model (K-Nearest Neighbors) is a simple yet powerful machine learning algorithm based on the fundamental assumption that similar samples are close to each other in the feature space. KNN predicts the category or attribute of a sample by finding its K nearest neighbors.

[0029] The core idea of ​​the KNN algorithm is to predict the class or attribute of a sample by finding its K nearest neighbors. For classification tasks, the KNN algorithm predicts the class of a new sample using majority voting; for regression tasks, it calculates the average of the target values ​​of the K nearest neighbors as the target value of the new sample.

[0030] In this embodiment, the training steps are as follows: S521, Calculate the distance between the test sample and all samples in the training set; in this step, for each test sample, the KNN algorithm calculates the distance between it and every sample in the training set, which is either Euclidean distance or cosine distance.

[0031] S522 uses a voting mechanism to select the class with the most frequent neighbor among the k neighbors as the predicted class for the test sample. After calculating all distances, the KNN algorithm finds the k nearest training samples (i.e., neighbors) to the test sample. Then, it counts the number of each class among these neighbors and selects the class with the most frequent neighbor as the predicted class for the test sample.

[0032] In S5, after training, the learning model is evaluated using a test set, and cross-validation is used to further evaluate the stability and generalization ability of the learning model.

[0033] Cross-validation: To more accurately evaluate the stability and generalization ability of a model, cross-validation can be used. Cross-validation divides the dataset into multiple subsets, and then uses one subset as the test set and the remaining subsets as the training set for multiple training and evaluation cycles.

[0034] Commonly used cross-validation methods include k-fold cross-validation, in which the dataset is divided into k equal parts. Each time, k-1 parts are used for training, and the remaining part is used for testing. This process is repeated k times, with a different part selected as the test set each time.

[0035] Test set: Contains data from the same dataset that the model has not seen before. By feeding the test set into the trained model, the model's predictions for these data can be obtained and compared with the actual labels. Evaluation metrics include accuracy, recall, and F1 score.

[0036] Accuracy: The proportion of samples that are correctly predicted out of the total number of samples.

[0037] Recall: The proportion of samples that are actually positive that are correctly predicted as positive.

[0038] F1 score: The harmonic mean of precision and recall, used to comprehensively measure the performance of a model. The higher the F1 score, the better the model's performance.

[0039] The evaluated model is then optimized by adjusting features or learning model parameters based on the evaluation results to improve model performance.

[0040] Feature adjustment: You can try adding new features, deleting unimportant features, or performing feature scaling, etc.

[0041] Model parameter tuning: You can adjust the hyperparameters of the learning model (such as the k value in KNN, the depth in decision trees, etc.) to find the optimal model configuration.

[0042] In summary, this method preprocesses text data, extracts features, trains and evaluates the model through a series of steps, and finally obtains a model that can be used for text-assisted analysis and evaluation.

[0043] This model is flexible and can effectively improve the accuracy and efficiency of text analysis. It can save manpower when conducting science and technology policy analysis and evaluation, and avoid the disadvantages of strong subjectivity in manual analysis.

[0044] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A text-assisted analysis and evaluation method, characterized in that: The method includes the following steps: S1, Data preparation, preprocessing the text, including removing stop words, handling punctuation marks, handling special characters, and converting case. S2, Build a bag-of-words model, convert the text into word frequency vectors and then use TF-IDF for weighting; S3, word embedding: Each word is converted into a high-dimensional vector using tools such as Word2Vec and GloVe, and then the results are compared. S4, Feature selection: Filter features based on their importance or relevance to reduce noise and overfitting; S5, Select the learning model and import the features for training; S6 uses the trained model to predict new text data.

2. The text-assisted analysis and evaluation method according to claim 1, characterized in that: In S5, the learning model is the Naive Bayes algorithm, and its training steps are as follows: S511, import the vector features from S4; S512, Construct the TF-IDF vector space for the test set; S513, Import the bag of words from the training set; S514, use TFIDFVectorizer to initialize the vector space model; S515, creates a persistent bag of words.

3. The text-assisted analysis and evaluation method according to claim 1, characterized in that: In S5, the learning model is a KNN model, and its training steps are as follows: S521, calculate the distance between the test sample and all samples in the training set; S522 uses a voting mechanism to select the class with the most frequent neighbor among the k neighbors as the predicted value of the predicted sample.

4. The text-assisted analysis and evaluation method according to claim 3, characterized in that: In S521, the sample distance is Euclidean distance or cosine distance.

5. The text-assisted analysis and evaluation method according to claim 1, characterized in that: In S5, after training, the learning model is evaluated using a test set, and cross-validation is used to further evaluate the stability and generalization ability of the learning model.

6. The text-assisted analysis and evaluation method according to claim 5, characterized in that: The evaluation criteria include accuracy, recall, and F1 score.

7. The text-assisted analysis and evaluation method according to claim 5, characterized in that: The evaluated model is then optimized by adjusting features or learning model parameters based on the evaluation results to improve model performance.

8. The text-assisted analysis and evaluation method according to claim 1, characterized in that: In S4, the feature selection includes statement extraction of policy topics, text summary generation, entity recognition, and relation extraction.

Images