Specific text classification and grading evaluation system and method based on information element extraction technology

Abstract

The invention relates to the technical field of text classification and grading evaluation, in particular to a specific text classification and grading evaluation system and method based on an information element extraction technology. The method comprises the steps that an extraction unit performs key element field extraction on text data based on a large model fine tuning technology; the standardization unit carries out standardization processing on each extracted element field; and the evaluation unit constructs a text classification and grading evaluation system through a decision tree based on the normalized fields. According to the technology, the accuracy and the coverage rate of key element field extraction can be improved through the large model fine tuning technology and the model, the model can be helped to better understand the context in a text, so that the situations of false report and missing report are reduced, the extraction quality is improved, the fine tuning technology can enable the model to better adapt to a new data set, and the extraction efficiency is improved. Even if the data set is slightly different from the pre-training data set, good performance can be kept, and the robustness of the model facing texts of different styles or formats can be enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of specific text classification and grading evaluation, and in particular to a specific text classification and grading evaluation system and method based on information element extraction technology. Background Art

[0002] With the popularization of the Internet and social media, the growth and spread of information have accelerated. The massive amount of text information contains text contents with different connotations. Among them, positive energy text information gives people confidence and encouragement, and helps the harmonious development of society; negative energy text information distorts the facts and increases social contradictions and conflicts. Large-scale text information needs to be classified and graded in a timely manner to realize the identification and screening of text content. Therefore, a specific text classification and grading evaluation system and method based on information element extraction technology is provided. Summary of the invention

[0003] The purpose of the present invention is to provide a specific text classification and grading evaluation system and method based on information element extraction technology to solve the problem of realizing text content recognition and screening proposed in the above background technology.

[0004] To achieve the above objectives, on the one hand, the present invention aims to provide a specific text classification and grading evaluation system based on information element extraction technology, comprising:

[0005] An extraction unit, wherein the extraction unit extracts key element fields from text data based on a large model fine-tuning technology;

[0006] A standardization unit, wherein the standardization unit performs standardization processing on each extracted element field;

[0007] An evaluation unit constructs a text classification and grading evaluation system through a decision tree based on the normalized fields.

[0008] As a further improvement of the technical solution, the key element fields in the extraction unit include:

[0009] Time, place, behavior, event.

[0010] As a further improvement of the technical solution, the extraction of key element fields from text data based on the large model fine-tuning technology includes the following steps:

[0011] S3.1. Collect text data containing keywords to build a dataset; annotate keywords for each document;

[0012] S3.2. Select a pre-trained large language model and use the Hugging Face library to load the pre-trained model and its corresponding word segmenter.

[0013] S3.3. According to the requirements of the keyword extraction task, a sequence labeling layer is added on top of the model, and a sequence classification model is used to mark the keywords in the text;

[0014] S3.4. Select the cross entropy loss function to optimize the model;

[0015] S3.5, divide the data set into training set, validation set and test set;

[0016] S3.6. Fine-tune the pre-trained model using the training set data and adjust the model parameters to optimize the loss function;

[0017] S3.7. Use the fine-tuned model to perform keyword extraction on text data.

[0018] As a further improvement of the technical solution, in S3.4, the cross entropy loss function is specifically:

[0019]

[0020] Where L represents the total cross entropy loss of the entire sequence; n represents the vocabulary; m represents the number of tags for each vocabulary; i represents the index of the vocabulary; j represents the index of the tag; p ij Indicates the one-hot encoding value that the model predicts word i belongs to label j; y ij Represents the one-hot encoded value of the true label of vocabulary i on label j.

[0021] As a further improvement of the technical solution, the normalization processing of each extracted element field includes the following steps:

[0022] S5.1. Create a data model to define data elements and their relationships;

[0023] S5.2. Convert data elements into a unified standard;

[0024] S5.3. Clean the data;

[0025] S5.4, convert the normalized data into the target format;

[0026] S5.5. Apply data validation rules and constraints to identify and correct data errors;

[0027] S5.6. Load the converted data into the evaluation system and perform data verification to ensure that the loaded data meets the requirements of the evaluation system.

[0028] As a further improvement of the technical solution, the text classification and grading evaluation system is constructed based on the normalized fields through a decision tree, comprising the following steps:

[0029] S6.1. Classify specific texts and divide them into different levels according to the text type;

[0030] S6.2. Construct features based on the extracted fields and select decision trees to construct the evaluation system;

[0031] S6.3. Collect specific text data containing different levels and label each text with its level;

[0032] S6.4. Use the bag-of-words model to convert text into numerical representation and select the most valuable features based on their importance;

[0033] S6.5. Use the training set data to train the model so that it can perform classification evaluation on newly received text.

[0034] As a further improvement of the technical solution, in S6.2, selecting a decision tree to construct an evaluation system comprises the following steps:

[0035] S6.21. Select the features that are most influential in grading a particular text;

[0036] S6.22. Prepare a training dataset to ensure that each sample has a corresponding specific text level label;

[0037] S6.23, use decision tree algorithm ID3 to train the model and adjust the model parameters;

[0038] S6.24. Evaluate the importance of each feature to determine which features have the greatest impact on text classification;

[0039] S6.25. Use feature importance scores to rank features and select the top-ranked features as input features of the final model for building an evaluation system.

[0040] As a further improvement of the technical solution, the decision tree algorithm ID3 is specifically:

[0041]

[0042] Where Ga(D, A) represents information gain, D represents the data set, A represents the feature, Va(A) represents all possible values ​​of feature A, and D represents the v represents a subset of feature A whose value is v; En(D) represents the entropy of data set D; En(D v ) represents the data subset D v The entropy of .

[0043] As a further improvement of the technical solution, in S8.4, the bag-of-words model is specifically:

[0044] BoW(d, t) = Count(t, d)

[0045] Wherein, d represents the text to be evaluated; t represents the keywords related to the specific text content extracted from the training data set; BoW(d, t) represents the word frequency of keyword t in the text d to be evaluated; Count(t, d) represents the number of times keyword t appears in the text d to be evaluated.

[0046] On the other hand, the present invention provides a specific text classification and grading evaluation method based on information element extraction technology, which is used in any of the above-mentioned specific text classification and grading evaluation systems based on information element extraction technology, comprising the following steps:

[0047] S10.1. Extract key element fields from text data based on large model fine-tuning technology;

[0048] S10.2, normalize each extracted element field;

[0049] S10.3. Construct a text classification and grading evaluation system based on the normalized fields through a decision tree.

[0050] Compared with the prior art, the present invention has the following beneficial effects:

[0051] 1. In the specific text classification and grading evaluation system and method based on information element extraction technology, the model can improve the accuracy and coverage of key element field extraction through large model fine-tuning technology, which can help the model better understand the context in the text, thereby reducing false positives and negatives and improving the extraction quality. The fine-tuning technology can make the model better adapt to the new data set, even if the data set is slightly different from the pre-training data set, it can maintain good performance, which helps to enhance the robustness of the model when facing texts of different styles or formats.

[0052] 2. In the specific text classification and grading evaluation system and method based on information element extraction technology, by normalizing the fields, the consistency and comparability of all text data can be ensured, thereby improving the accuracy of specific text grading. The normalized fields reduce ambiguity in the text and help the model to more accurately identify specific text content; the normalized fields are input into the decision tree model as features, which can clearly show which features are most influential in the specific text grading. The decision tree model can evaluate the importance of each feature and help select the most influential features, thereby optimizing the feature set. Through feature importance evaluation, redundant features can be reduced and the efficiency and accuracy of the model can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a flowchart of the overall process of the present invention.

[0054] The meaning of each number in the figure is:

[0055] 1. Extraction unit; 2. Standardization unit; 3. Evaluation unit. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0057] Embodiment 1:

[0058] See also Figure 1 As shown, a specific text classification and grading evaluation system based on information element extraction technology is provided, including.

[0059] The extraction unit 1 extracts key element fields from the text data based on the large model fine-tuning technology;

[0060] In this embodiment, the key element fields include:

[0061] time, place, action, event;

[0062] Among them, the large model has learned rich language knowledge by pre-training on a large amount of unlabeled text, and can better understand and represent the text content. The large model can capture the difference in the meaning of words in different contexts, which is very important for the extraction of key element fields. The large model can quickly adapt to new task areas and only requires a small amount of labeled data for fine-tuning to achieve good performance. Compared with training the model from scratch, fine-tuning the pre-trained model requires much less computing resources and time. Extracting key element fields from text data based on the large model fine-tuning technology includes the following steps:

[0063] S3.1. Collect text data containing keywords to build a data set. This data can be obtained from public resources or created by yourself. Label each document with keywords, including time, place, behavior, and event.

[0064] S3.2. Select a pre-trained large language model and use the Hugging Face library to load the pre-trained model and its corresponding word segmenter. The Hugging Face Transformers library provides an easy-to-use interface to load pre-trained models.

[0065] S3.3. According to the requirements of the keyword extraction task, a sequence labeling layer is added on top of the language model, and a sequence classification model is used to mark the keywords in the text;

[0066] S3.4. Select the cross entropy loss function to optimize the model;

[0067] Among them, the cross entropy loss function can directly measure the difference between the probability distribution predicted by the model and the probability distribution of the true label. It can provide a strong gradient signal, which helps the model to quickly converge to the optimal solution. The cross entropy loss function can provide a clear gradient signal to help the model quickly correct wrong predictions. Even when the predicted probability is close to 1 or 0, the cross entropy loss function can provide a stable gradient signal. The cross entropy loss function is specifically:

[0068]

[0069] Where L represents the total cross entropy loss of the entire sequence; n represents the vocabulary; m represents the number of labels for each vocabulary; i represents the index of the vocabulary; j represents the index of the label; p ij Indicates the one-hot encoding value that the model predicts word i belongs to label j; y ij Represents the one-hot encoded value of the true label of vocabulary i on label j.

[0070] S3.5, divide the data set into training set, validation set and test set;

[0071] S3.6. Fine-tune the pre-trained model using the training set data and adjust the model parameters to optimize the loss function. The model parameters include learning rate, batch size, and number of iterations.

[0072] S3.7. Use the fine-tuned model to extract keywords from text data. Keywords include: time, place, behavior, and event.

[0073] The standardization unit 2 performs standardization processing on each extracted element field;

[0074] In this embodiment, normalization processing can ensure the quality and consistency of data, thereby improving the accuracy and efficiency of subsequent analysis and model training; normalization processing can simplify the data structure, make data easier to manage and analyze, reduce noise and redundant information in the data, and help reduce the risk of overfitting; normalization processing helps to reduce the deviation caused by data skew, helps to integrate data from different sources, and ensures the consistency and compatibility of data; normalization processing of each extracted element field includes the following steps:

[0075] S5.1. Create a data model to define data elements and their relationships, which is the basis of the normalization process and ensures that the data structure is clear and convenient for subsequent processing and analysis;

[0076] S5.2. Convert data elements into unified standards, including data types, formats, and units, which helps eliminate data format or coding differences and simplifies the data analysis process;

[0077] S5.3. Clean the data, including removing HTML tags, special characters, extra spaces, stop words, and processing missing values ​​and outliers. The purpose of this step is to improve data quality and ensure data accuracy and consistency;

[0078] S5.4. Convert the normalized data into the target format for use in the data integration process, including format conversion, data cleaning, and data mapping;

[0079] S5.5. Apply data validation rules and constraints to identify and correct data errors. This step ensures the legitimacy and reliability of the data and provides a reliable basis for data-driven decision-making.

[0080] S5.6. Load the converted data into the evaluation system and perform data verification to ensure that the loaded data meets the requirements of the evaluation system.

[0081] Evaluation unit 3 constructs a text classification and grading evaluation system based on the normalized fields through a decision tree;

[0082] In this embodiment, normalization helps to create more meaningful features, thereby improving the performance of the model. When building a decision tree, high-quality features help simplify the decision-making process. Normalization can reduce noise and redundant information in the data, help reduce the risk of overfitting, make the decision tree more robust, and the decision tree model has high interpretability. The normalized fields can further enhance this. Through normalization, each split point of the decision tree becomes clearer, making the model easier to understand and explain. The decision tree can be easily visualized. The normalized fields help to build a more concise and clear decision tree structure. Constructing a text classification and grading evaluation system through a decision tree based on the normalized fields includes the following steps:

[0083] S6.1. Classify specific texts and divide them into different levels according to the text type; Classify them into different levels according to the specific text categories, including low level, medium level and high level;

[0084] S6.2. Construct features based on the extracted fields. These features include keywords, word frequency, and sentiment polarity. Select a decision tree to construct an evaluation system.

[0085] Among them, selecting a decision tree to build an evaluation system includes the following steps:

[0086] S6.21. Select the most influential features for grading specific texts, such as specific keywords, sentiment scores, word frequencies, etc.

[0087] S6.22. Prepare a training dataset to ensure that each sample has a corresponding text-level label;

[0088] S6.23, use the decision tree algorithm ID3 to train the model and adjust the model parameters, such as the maximum depth h of the tree and the minimum number of samples s, to optimize the model performance;

[0089] Furthermore, the decision tree structure generated by the ID3 algorithm is intuitive and easy to understand. Each node and branch of the decision tree represents a clear decision rule, which makes the decision process of the model easy to explain. The ID3 algorithm can directly handle the nonlinear relationship between features without complex feature engineering. It can automatically select the optimal features for segmentation, which helps to reduce the workload of feature engineering. The ID3 algorithm uses information gain as the criterion for feature selection and can automatically select the features that contribute most to classification. The decision tree algorithm ID3 is specifically:

[0090]

[0091] Where Ga(D, A) represents information gain, which is a criterion for feature selection and measures the amount of information provided by the feature. D represents the data set; A represents the feature; Va(A) represents all possible values ​​of feature A; D v represents a subset of feature A whose value is v; En(D) represents the entropy of data set D, which indicates the uncertainty of data set; En(D v ) represents the data subset D v The entropy of .

[0092] S6.24. Evaluate the importance of each feature to determine which features are most influential in grading a particular text;

[0093] S6.25. Use feature importance scores to rank features and select the top-ranked features as input features of the final model for building an evaluation system.

[0094] S6.3. Collect specific text data containing different levels and label each text with its level;

[0095] S6.4. Use the bag-of-words model to convert text into numerical representation and select the most valuable features based on their importance;

[0096] Furthermore, the bag-of-words model is very intuitive and easy to implement. When building a bag-of-words model, you usually only need to count the frequency of vocabulary occurrences, and the computational cost is relatively low. You can adapt to different task requirements by adjusting the size of the vocabulary. When new text data arrives, you can easily update the bag-of-words model to adapt to the new vocabulary. The bag-of-words model is specifically:

[0097] BoW(d, t) = Count(t, d)

[0098] Wherein, d represents the text to be evaluated; t represents the keywords related to the specific text content extracted from the training data set; BoW(d, t) represents the word frequency of keyword t in the text d to be evaluated; Count(t, d) represents the number of times keyword t appears in the text d to be evaluated.

[0099] S6.5. Use the training set data to train the model so that it can perform classification evaluation on newly received text.

[0100] Embodiment 2:

[0101] The difference between Example 2 of the present invention and Example 1 is that this example introduces a specific text grading evaluation method used by a specific text classification and grading evaluation system based on information element extraction technology.

[0102] The specific text classification and grading evaluation method based on information element extraction technology is used in any of the above-mentioned specific text classification and grading evaluation systems based on information element extraction technology, comprising the following steps:

[0103] S10.1. Extract key element fields from text data based on large model fine-tuning technology;

[0104] S10.2, normalize each extracted element field;

[0105] S10.3. Construct a text grading evaluation system based on the normalized fields through a decision tree.

[0106] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments, and the above embodiments and descriptions are only preferred examples of the present invention, and are not intended to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, and these changes and improvements all fall within the scope of the present invention to be protected.

Claims

1. A specific text classification and grading evaluation system based on information element extraction technology, characterized by: include: An extraction unit (1), wherein the extraction unit (1) extracts key element fields from text data based on a large model fine-tuning technique; A standardization unit (2), wherein the standardization unit (2) performs standardization processing on each extracted element field; An evaluation unit (3), wherein the evaluation unit (3) constructs a text classification and grading evaluation system based on the normalized fields through a decision tree.

2. The specific text classification and grading evaluation system based on information element extraction technology according to claim 1 is characterized by: The key element fields in the extraction unit (1) include: Time, place, behavior, event.

3. The specific text classification and grading evaluation system based on information element extraction technology according to claim 1 is characterized by: The method of extracting key element fields from text data based on the large model fine-tuning technology includes the following steps: S3.

1. Collect text data containing keywords to build a dataset; annotate keywords for each document; S3.

2. Select a pre-trained large language model and use the Hugging Face library to load the pre-trained model and its corresponding word segmenter. S3.

3. According to the requirements of the keyword extraction task, a sequence labeling layer is added on top of the model, and a sequence classification model is used to mark the keywords in the text; S3.

4. Select the cross entropy loss function to optimize the model; S3.5, divide the data set into training set, validation set and test set; S3.

6. Fine-tune the pre-trained model using the training set data and adjust the model parameters to optimize the loss function; S3.

7. Use the fine-tuned model to perform keyword extraction on text data.

4. The specific text classification and grading evaluation system based on information element extraction technology according to claim 1 is characterized by: In S3.4, the cross entropy loss function is specifically: Where L represents the total cross entropy loss of the entire sequence; n represents the vocabulary; m represents the number of tags for each vocabulary; i represents the index of the vocabulary; j represents the index of the tag; p ij Indicates the one-hot encoding value that the model predicts word i belongs to label j; y ij Represents the one-hot encoded value of the true label of vocabulary i on label j.

5. The specific text classification and grading evaluation system based on information element extraction technology according to claim 1 is characterized by: The process of normalizing the extracted element fields comprises the following steps: S5.

1. Create a data model to define data elements and their relationships; S5.

2. Convert data elements into a unified standard; S5.

3. Clean the data; S5.4, convert the normalized data into the target format; S5.

5. Apply data validation rules and constraints to identify and correct data errors; S5.

6. Load the converted data into the evaluation system and perform data verification to ensure that the loaded data meets the requirements of the evaluation system.

6. The specific text classification and grading evaluation system based on information element extraction technology according to claim 1 is characterized by: The text classification and grading evaluation system is constructed based on the normalized fields through a decision tree, including the following steps: S6.

1. Classify specific texts and divide them into different levels according to the text type; S6.

2. Construct features based on the extracted fields and select decision trees to construct the evaluation system; S6.

3. Collect specific text data containing different levels and label each text with its level; S6.

4. Use the bag-of-words model to convert text into numerical representation and select the most valuable features based on their importance; S6.

5. Use the training set data to train the model so that it can perform classification evaluation on newly received text.

7. The specific text classification and grading evaluation system based on information element extraction technology according to claim 1 is characterized by: In S6.2, a decision tree is selected to construct an evaluation system. The following steps are involved: S6.

21. Select the features that are most influential in grading a particular text; S6.

22. Prepare a training dataset to ensure that each sample has a corresponding specific text level label; S6.23, use decision tree algorithm ID3 to train the model and adjust the model parameters; S6.

24. Evaluate the importance of each feature to determine which features have the greatest impact on text classification; S6.

25. Use feature importance scores to rank features and select the top-ranked features as input features of the final model for building an evaluation system.

8. The specific text classification and grading evaluation system based on information element extraction technology according to claim 1 is characterized by: The decision tree algorithm ID3 is specifically: Where D represents the data set; A represents the feature; Va(A) represents all possible values ​​of feature A; D v represents a subset of feature A whose value is v; En(D) represents the entropy of data set D; En(D v ) represents the data subset D v The entropy of .

9. The specific text classification and grading evaluation system based on information element extraction technology according to claim 1 is characterized by: In S8.4, the bag-of-words model is specifically: BoW(d,t)=Count(t,d) Wherein, d represents the text to be evaluated; t represents the keywords related to the specific text content extracted from the training data set; BoW(d, t) represents the word frequency of keyword t in the text d to be evaluated; Count(t, d) represents the number of times keyword t appears in the text d to be evaluated.

10. A method for classifying and grading specific texts based on information element extraction technology, used in a system for classifying and grading specific texts based on information element extraction technology as claimed in any one of claims 1 to 9, characterized in that: The steps include: S10.

1. Extract key element fields from text data based on large model fine-tuning technology; S10.2, normalize each extracted element field; S10.

3. Construct a text classification and grading evaluation system based on the normalized fields through a decision tree.

Images