Construction method, system and equipment of power news data classification model and medium

Abstract

The embodiment of the invention provides an electric power news data classification model construction method and system, equipment and a medium, and belongs to the field of data classification. The construction method comprises the steps of obtaining an electric power news text data set, and constructing a word vector matrix based on the electric power news text data set; an electric power news data classification model is constructed, and the electric power news data classification model is constructed based on a support vector machine algorithm; and inputting the word vector matrix into a pre-constructed power news data classification model to obtain a target classification result. The electric power news data classification model constructed based on the support vector machine algorithm is adopted to classify the electric power news data, when complex high-dimensional electric power news data is processed, the support vector machine algorithm can effectively find out an optimal hyperplane, different types of news data are accurately separated, and the accuracy of the classification of the electric power news data is improved. Therefore, the classification accuracy and the classification efficiency are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data classification, and in particular to a method, system, device and medium for constructing a power news data classification model. Background Art

[0002] At present, the classification and processing of power news data mainly relies on manual reading and judgment or rule-based classification methods based on keyword matching.

[0003] These methods have the following defects: First, manual reading is inefficient. A large amount of power news data needs to be read and classified manually one by one, which is time-consuming and labor-intensive. Second, the classification results are affected by personal subjectivity. Due to subjective judgment in manual classification, different people may have different classification standards and preferences, resulting in inconsistent classification results. Third, there are limitations in keyword matching. Classification methods based on simple keyword matching often fail to fully consider the semantics and contextual information of the text, resulting in limited classification results. Fourth, it is unable to cope with large-scale data. As the amount of data increases, manual classification and simple rule classification methods cannot meet the needs of quickly and accurately processing large-scale power news data.

[0004] Therefore, when classifying power news data, the current technology has defects such as low efficiency, poor accuracy, and subjectivity. There is an urgent need for a power news data classification model based on automation technology to improve processing efficiency and classification accuracy. Summary of the invention

[0005] The purpose of the embodiments of the present invention is to provide a method, system, device and medium for constructing a power news data classification model, so as to solve the problems of low efficiency and poor accuracy of power news data classification in the above-mentioned prior art.

[0006] In order to achieve the above object, an embodiment of the present invention provides a method for constructing a power news data classification model, comprising:

[0007] Obtaining an electric power news text dataset, and constructing a word vector matrix based on the electric power news text dataset;

[0008] Constructing a power news data classification model, wherein the power news data classification model is constructed based on a support vector machine algorithm;

[0009] The word vector matrix is ​​input into a pre-built electric power news data classification model to obtain the target classification result.

[0010] Optionally, the acquiring of a power news text dataset and constructing a word vector matrix based on the power news text dataset includes:

[0011] Using the pandas data processing library, load the electric power news text data, and process the missing values ​​and duplicate values ​​of the electric power news text data to obtain an initial electric power news text data set;

[0012] Perform Chinese word segmentation, remove stop words, and filter records with empty return values ​​on the initial electric power news text dataset to obtain the electric power news text dataset;

[0013] TfidfVectorizer is used to extract text features from text, and the extracted features are reduced in dimension to construct a word vector matrix.

[0014] Optionally, the construction process of the power news data classification model includes:

[0015] Obtain a historical electric power news text dataset and divide it into a training set and a validation set, wherein the training set contains feature data and labels corresponding to the feature data;

[0016] Initialize the parameters of the power news data classification model;

[0017] The training set is input into the electric power news data classification model for training to obtain the electric power news data classification model. During the training process, the electric power news data classification model will learn the relationship between the features of the electric power news data and its labels, and determine the target hyperplane that separates different categories of electric power news data.

[0018] Optionally, the power news data classification model is:

[0019]

[0020] In the formula, Indicates power news data Xt j The mapped eigenvector, K(Xt j ,z) represents the kernel function of the power news data classification model, w represents the normal vector of the hyperplane, b represents the intercept of the hyperplane, and α represents the Lagrange multiplier.

[0021] Optionally, the word vector matrix is ​​input into a pre-built electric power news data classification model to obtain a target classification result, including:

[0022] If the word vector matrix is ​​input into the electric power news data classification model and the output value is less than zero, the electric power news data corresponding to the word vector matrix is ​​classified into the first category;

[0023] If the word vector matrix is ​​input into the electric power news data classification model and the output value is greater than zero, the electric power news data corresponding to the word vector matrix is ​​classified into the second category;

[0024] If the word vector matrix is ​​input into the electric power news data classification model and the output value is equal to zero, the electric power news data corresponding to the word vector matrix is ​​classified into the first category or the second category.

[0025] On the other hand, the present invention also provides a system for constructing a power news data classification model, comprising:

[0026] An acquisition unit, used to acquire a power news text dataset and construct a word vector matrix based on the power news text dataset;

[0027] A construction unit, used for constructing a power news data classification model, wherein the power news data classification model is constructed based on a support vector machine algorithm;

[0028] The classification unit is used to input the word vector matrix into a pre-built electric power news data classification model to obtain a target classification result.

[0029] Optionally, the acquiring unit is specifically used for:

[0030] Using the pandas data processing library, load the electric power news text data, and process the missing values ​​and duplicate values ​​of the electric power news text data to obtain an initial electric power news text data set;

[0031] Perform Chinese word segmentation, remove stop words, and filter records with empty return values ​​on the initial electric power news text dataset to obtain the electric power news text dataset;

[0032] TfidfVectorizer is used to extract text features from text, and the extracted features are reduced in dimension to construct a word vector matrix.

[0033] Optionally, the construction unit is specifically used for:

[0034] Obtain a historical electric power news text dataset and divide it into a training set and a validation set, wherein the training set contains feature data and labels corresponding to the feature data;

[0035] Initialize the parameters of the power news data classification model;

[0036] The training set is input into the electric power news data classification model for training to obtain the electric power news data classification model. During the training process, the electric power news data classification model will learn the relationship between the features of the electric power news data and its labels, and determine the target hyperplane that separates different categories of electric power news data.

[0037] On the other hand, the present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned construction method when executing the program.

[0038] On the other hand, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above-mentioned construction method when executed by a processor.

[0039] Through the above technical scheme, the power news data classification model constructed based on the support vector machine algorithm is used to classify the power news data. When processing complex high-dimensional power news data, the support vector machine algorithm can effectively find the optimal hyperplane and accurately separate different categories of news data, thereby improving classification accuracy and classification efficiency.

[0040] Other features and advantages of the embodiments of the present invention will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following specific implementations, they are used to explain the embodiments of the present invention, but do not constitute a limitation on the embodiments of the present invention. In the accompanying drawings:

[0042] Figure 1 It is an implementation flow chart of a method for constructing a power news data classification model provided by an embodiment of the present invention;

[0043] Figure 2 It is a structural diagram of a system for constructing a power news data classification model provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0044] The specific implementation of the embodiment of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the embodiment of the present invention, and is not used to limit the embodiment of the present invention.

[0045] See also Figure 1 As shown, it is an implementation flow chart of a method for constructing a power news data classification model provided by an embodiment of the present invention, comprising the following execution steps:

[0046] Step 100: Obtain an electric power news text dataset, and construct a word vector matrix based on the electric power news text dataset.

[0047] In some implementations, when executing step 100, the following steps may be specifically performed:

[0048] S1000: Using the pandas data processing library, load the electric power news text data, and process the missing values ​​and duplicate values ​​of the electric power news text data to obtain an initial electric power news text data set.

[0049] Specifically, first, fill and delete missing values ​​and duplicate values. Outliers in Chinese text are generally simply understood as non-Chinese characters, which can be filtered out through regular expressions; secondly, observe the distribution of sample labels. If there is serious skewness, implement resampling and other strategies.

[0050] S1001: Perform Chinese word segmentation, remove stop words, and filter records with empty return values ​​on the initial electric power news text dataset to obtain the electric power news text dataset.

[0051] Specifically, use jieba to segment the text into Chinese words, remove stop words, and filter out records with empty return values.

[0052] S1002: Use TfidfVectorizer to extract text features from the text, perform dimensionality reduction on the extracted features, and construct a word vector matrix.

[0053] Specifically, since there are specific requirements for data when constructing the vector matrix, the fields containing the news text content must be processed again and converted into a format that can be recognized by the computer. TfidfVectorizer is used to extract text features from the text. Since the extracted features reach tens of millions of dimensions, the dimension explosion occurs, and then TruncatedSVD is used to reduce the data dimension. Features related to the classification of power news, such as keywords, subject terms, sentence structure, etc., are extracted from the preprocessed data.

[0054] Step 101: Build a classification model for power news data.

[0055] Among them, the power news data classification model is constructed based on the support vector machine algorithm.

[0056] In some implementations, when executing step 101, the following steps are specifically performed:

[0057] S1010: Obtain a historical power news text dataset and divide it into a training set and a validation set.

[0058] Among them, the training set contains feature data and labels corresponding to the feature data;

[0059] S1011: Initialize the parameters of the power news data classification model.

[0060] S1012: Input the training set into the electric power news data classification model for training to obtain the electric power news data classification model.

[0061] Among them, during the training process, the power news data classification model will learn the relationship between the characteristics of the power news data and its labels, and determine the target hyperplane that separates different categories of power news data.

[0062] In some implementations, the training process is specifically as follows:

[0063] Constructing a hyperplane: The training goal of the support vector machine algorithm is to find a hyperplane that can separate sample data of different categories. In two-dimensional space, this hyperplane is a straight line; in three-dimensional space, it is a plane; in high-dimensional space, it is a hyperplane.

[0064] Maximizing the margin: The support vector machine algorithm finds the optimal hyperplane by maximizing the distance (margin) between the support vector and the hyperplane. The support vector is the sample point closest to the hyperplane and plays a decisive role in the position of the hyperplane.

[0065] Optimization solution: The training process of the support vector machine algorithm can be transformed into the solution of a convex quadratic programming problem. By solving this optimization problem, the optimal hyperplane parameters (i.e., weight vector w and bias term b) can be obtained.

[0066] Soft margin processing: When there is noise or outliers in the data, strictly requiring all samples to meet the condition that the margin is greater than or equal to 1 may lead to overfitting. Therefore, the support vector machine algorithm introduces the concept of soft margin, which allows some samples not to meet the margin requirement, but will penalize these samples. The penalty coefficient C is used to control this trade-off.

[0067] During the training process, the optimal decision hyperplane is found by maximizing the margin, which helps reduce the risk of overfitting and enables the model to maintain good performance on unseen data.

[0068] In some implementations, the power news data classification model is:

[0069]

[0070] In the formula, α represents the Lagrange multiplier, Indicates power news data Xt j The mapped eigenvector, K(Xt j ,z) represents the kernel function of the power news data classification model, w represents the normal vector of the hyperplane, and b represents the intercept of the hyperplane.

[0071] Step 102: Input the word vector matrix into a pre-built electric power news data classification model to obtain a target classification result.

[0072] Specifically, when executing step 102, the following steps may be specifically performed:

[0073] S1020: If the word vector matrix is ​​input into the electric power news data classification model and the output value is less than zero, the electric power news data corresponding to the word vector matrix is ​​classified into the first category.

[0074] S1021: If the word vector matrix is ​​input into the electric power news data classification model and the output value is greater than zero, the electric power news data corresponding to the word vector matrix is ​​classified into the second category.

[0075] S1022: If the word vector matrix is ​​input into the electric power news data classification model and the output value is zero, the electric power news data corresponding to the word vector matrix is ​​classified into the first category or the second category.

[0076] The technical effects achieved by this application are as follows:

[0077] First, it improves work efficiency. News data that originally needed to be manually classified one by one can now be classified after being read once and put into the model, which improves work efficiency and saves labor costs.

[0078] The second is to improve the accuracy of news data classification. Originally, due to subjective judgment and other factors in manual classification, the accuracy of news data classification was about 92%. After using the constructed power news data classification model, the accuracy reached 98%, greatly improving the accuracy of classified data.

[0079] Third, the electric power news data classification model is easy to deploy and simple to use. You only need to install the corresponding library, put the classified data into the corresponding location, and run it to generate results with one click.

[0080] See also Figure 2 FIG. 1 is a schematic diagram of a system for constructing a power news data classification model provided by an embodiment of the present invention, comprising:

[0081] The acquisition unit 200 is used to acquire an electric power news text dataset and construct a word vector matrix based on the electric power news text dataset.

[0082] The construction unit 201 is used to construct a power news data classification model, wherein the power news data classification model is constructed based on a support vector machine algorithm.

[0083] The classification unit 202 is used to input the word vector matrix into a pre-built electric power news data classification model to obtain a target classification result.

[0084] In some implementations, the acquisition unit 200 is specifically configured to:

[0085] Using the pandas data processing library, load the electric power news text data, and process the missing values ​​and duplicate values ​​of the electric power news text data to obtain an initial electric power news text data set;

[0086] Perform Chinese word segmentation, remove stop words, and filter records with empty return values ​​on the initial electric power news text dataset to obtain the electric power news text dataset;

[0087] TfidfVectorizer is used to extract text features from text, and the extracted features are reduced in dimension to construct a word vector matrix.

[0088] In some embodiments, the construction unit 201 is specifically used for:

[0089] Obtain a historical electric power news text dataset and divide it into a training set and a validation set, wherein the training set contains feature data and labels corresponding to the feature data;

[0090] Initialize the parameters of the power news data classification model;

[0091] The training set is input into the electric power news data classification model for training to obtain the electric power news data classification model. During the training process, the electric power news data classification model will learn the relationship between the features of the electric power news data and its labels, and determine the target hyperplane that separates different categories of electric power news data.

[0092] On the other hand, the present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the construction method described in any one of the above embodiments when executing the program.

[0093] On the other hand, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of the construction method described in any one of the above embodiments are implemented.

[0094] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0095] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A method for constructing a power news data classification model, characterized in that: include: Obtaining an electric power news text dataset, and constructing a word vector matrix based on the electric power news text dataset; Constructing a power news data classification model, wherein the power news data classification model is constructed based on a support vector machine algorithm; The word vector matrix is ​​input into a pre-built electric power news data classification model to obtain the target classification result.

2. The construction method according to claim 1, characterized in that: The step of obtaining a power news text dataset and constructing a word vector matrix based on the power news text dataset includes: Using the pandas data processing library, load the electric power news text data, and process the missing values ​​and duplicate values ​​of the electric power news text data to obtain an initial electric power news text data set; Perform Chinese word segmentation, remove stop words, and filter records with empty return values ​​on the initial electric power news text dataset to obtain the electric power news text dataset; TfidfVectorizer is used to extract text features from text, and the extracted features are reduced in dimension to construct a word vector matrix.

3. The construction method according to claim 1, characterized in that: The construction process of the power news data classification model includes: Obtain a historical electric power news text dataset and divide it into a training set and a validation set, wherein the training set contains feature data and labels corresponding to the feature data; Initialize the parameters of the power news data classification model; The training set is input into the electric power news data classification model for training to obtain the electric power news data classification model. During the training process, the electric power news data classification model will learn the relationship between the features of the electric power news data and its labels, and determine the target hyperplane that separates different categories of electric power news data.

4. The construction method according to claim 1, characterized in that: The power news data classification model is: In the formula, Indicates power news data Xt j The mapped eigenvector, K(Xt j ,z) represents the kernel function of the power news data classification model, w represents the normal vector of the hyperplane, b represents the intercept of the hyperplane, and α represents the Lagrange multiplier.

5. The construction method according to claim 4, characterized in that: The word vector matrix is ​​input into the pre-built electric power news data classification model to obtain the target classification results, including: If the word vector matrix is ​​input into the electric power news data classification model and the output value is less than zero, the electric power news data corresponding to the word vector matrix is ​​classified into the first category; If the word vector matrix is ​​input into the electric power news data classification model and the output value is greater than zero, the electric power news data corresponding to the word vector matrix is ​​classified into the second category; If the word vector matrix is ​​input into the electric power news data classification model and the output value is equal to zero, the electric power news data corresponding to the word vector matrix is ​​classified into the first category or the second category.

6. A system for constructing a power news data classification model, characterized in that: include: An acquisition unit, used to acquire a power news text dataset and construct a word vector matrix based on the power news text dataset; A construction unit, used for constructing a power news data classification model, wherein the power news data classification model is constructed based on a support vector machine algorithm; The classification unit is used to input the word vector matrix into a pre-built electric power news data classification model to obtain a target classification result.

7. The construction system according to claim 6, characterized in that: The acquisition unit is specifically used for: Using the pandas data processing library, load the electric power news text data, and process the missing values ​​and duplicate values ​​of the electric power news text data to obtain an initial electric power news text data set; Perform Chinese word segmentation, remove stop words, and filter records with empty return values ​​on the initial electric power news text dataset to obtain the electric power news text dataset; TfidfVectorizer is used to extract text features from text, and the extracted features are reduced in dimension to construct a word vector matrix.

8. The construction system according to claim 6, characterized in that: The building blocks are specifically used for: Obtain a historical electric power news text dataset and divide it into a training set and a validation set, wherein the training set contains feature data and labels corresponding to the feature data; Initialize the parameters of the power news data classification model; The training set is input into the electric power news data classification model for training to obtain the electric power news data classification model. During the training process, the electric power news data classification model will learn the relationship between the features of the electric power news data and its labels, and determine the target hyperplane that separates different categories of electric power news data.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the construction method according to any one of claims 1 to 5 are implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the construction method according to any one of claims 1 to 5 are implemented.

Images