Policy awareness quantitative file analysis method based on LDA topic model

Abstract

The invention provides a policy awareness quantitative file analysis method based on an LDA topic model. The policy awareness quantitative file analysis method comprises the following steps that documents are obtained and preprocessed to obtain a document library; performing word segmentation processing to remove stop words, and processing documents in the document library by adopting a TF-IDF method to obtain a high-frequency word library; constructing an LDA topic model based on the document library and the high-frequency word library; obtaining an optimal topic number of the LDA topic model by adopting a confusion analysis method and a consistency analysis method, and inferring the LDA topic model by adopting a variational inference method to obtain first distribution and second distribution so as to further obtain an optimal topic name; and analyzing the document library according to the optimal theme name. According to the analysis method provided by the invention, the core theme in the document can be extracted and explained, the important attention field and the policy development trend of the policy can be known, the policy key point, the policy association and the policy evolution trend can be conveniently interpreted, and an important reference basis is provided for policy formulators and researchers.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a policy-aware quantitative document analysis method based on an LDA topic model. Background Art

[0002] As the global energy transition accelerates, energy storage technology, as a key technology to solve the volatility and intermittency of renewable energy, has gradually become a key focus of energy policies in countries around the world. As an important part of the future energy system, energy storage technology has become one of the hot areas of concern in academia and industry.

[0003] In the existing research, in terms of policy analysis, some inventions have revealed the logic and driving factors of policy evolution by studying the evolution of energy storage policies in different countries. Some studies have used multi-level perspectives, system dynamics and other methods to analyze the impact of different national policies on the development of energy storage technology. In terms of text analysis methods, existing inventions include content analysis to measure and analyze my country's new energy policy in two stages, sentiment analysis to explore the public's sentimental tendencies and theme characteristics for my country's distributed photovoltaic policies, and the use of LDA theme models to identify themes for electrochemical energy storage battery technology. However, existing research has failed to systematically sort out and summarize the core themes of energy storage policies. Summary of the invention

[0004] In response to the problems existing in the prior art, the present invention provides a policy-aware quantitative document analysis method based on the LDA topic model, which can extract and interpret core topics in documents, understand the key areas of policy focus and policy development trends, facilitate the interpretation of policy priorities and policy connections and policy evolution trends, and provide an important reference basis for policy makers and researchers.

[0005] To achieve the above object, the technical solution adopted by the present invention is as follows: a policy-aware quantitative document analysis method based on LDA topic model, comprising the following steps:

[0006] Acquire and preprocess source files to obtain documents, and build a document library based on the documents;

[0007] Get stop words and build a stop word library based on the stop words;

[0008] Process the documents in the document library through word segmentation to obtain a first word library;

[0009] Removing the stop words in the stop word library from the first word library to obtain a second word library, wherein the second word library is a set of words to be selected;

[0010] Using the TF-IDF method to process the words to be selected in the second vocabulary to obtain high-frequency words, and constructing a high-frequency vocabulary based on the high-frequency words;

[0011] Constructing an LDA topic model based on the document library and the high-frequency word library;

[0012] Using a perplexity analysis method to obtain the topic number interval of the LDA topic model;

[0013] Based on the topic number interval, a consistency analysis method is used to obtain the optimal topic number of the LDA topic model, where the optimal topic number is the number of optimal topics of the LDA topic model;

[0014] Based on the optimal number of topics, a variational inference method is used to infer the LDA topic model to obtain a first distribution and a second distribution; the first distribution is the distribution of the optimal topic of one of the documents in the document library, and the second distribution is the distribution of the high-frequency words in one of the optimal topics;

[0015] Obtaining an optimal topic name for each of the optimal topics according to the first distribution and the second distribution;

[0016] The document library is analyzed according to the optimal topic names.

[0017] Furthermore, the steps of obtaining the document library are:

[0018] Get the preset time period and file source;

[0019] Collecting the source files within the preset time period from the file source;

[0020] The source file is preprocessed to obtain the document, and the document library is constructed according to the document.

[0021] Furthermore, the preprocessing includes format unification and text deduplication.

[0022] Furthermore, the stop words include standard stop words and custom stop words, and the custom stop words are stop words constructed in combination with domain features.

[0023] Furthermore, the steps of constructing the high-frequency word library are:

[0024] Obtain a first number, a first quantity, a second number, a third number and a fourth number; the first number is the number of times one of the words to be selected appears in one of the documents in the document library; the first quantity is the total number of the words to be selected in one of the documents in the document library; the second number is the total number of documents in the document library; the third number is the total number of the documents in the document library that include one of the words to be selected; the fourth number is the total number of the high-frequency words in the high-frequency word library;

[0025] Obtaining a TF value according to the first times and the first quantity, wherein the TF value is a frequency of one of the words to be selected in one of the documents in the document library;

[0026] Obtaining an IDF value according to the second number and the third number, wherein the IDF value is the rarity of one of the words to be selected in the document library;

[0027] Obtaining a TF-IDF value according to the TF value and the IDF value, wherein the TF-IDF value is a weight of one of the words to be selected in the document library;

[0028] sorting the words to be selected in the second vocabulary from high to low according to the TF-IDF value;

[0029] The high-frequency words are obtained by selecting the sorted words to be selected based on the fourth quantity, and the high-frequency word library is constructed according to the high-frequency words.

[0030] Furthermore, the steps of obtaining the optimal number of topics are:

[0031] Obtain a first probability, a fifth quantity, a sixth quantity, and a seventh quantity; the first probability is the probability that one of the high-frequency words is in one of the documents in the document library; the fifth quantity is the total number of the documents in the document library that include one of the high-frequency words; the sixth quantity is the total number of the documents in the document library that include two of the high-frequency words; the seventh quantity is the number of the high-frequency words considered in each topic;

[0032] Obtain the perplexity calculation formula and consistency calculation formula;

[0033] Based on the perplexity calculation formula, obtaining a perplexity curve according to the fourth quantity, the second quantity and the first probability;

[0034] Obtaining the topic number interval according to the perplexity curve;

[0035] Based on the consistency calculation formula, obtaining a consistency curve according to the fifth quantity, the sixth quantity and the seventh quantity;

[0036] The optimal number of topics is obtained by selecting the topic number interval according to the consistency curve.

[0037] Furthermore, the LDA topic model is:

[0038] θ d ~Dir(α),

[0039] φ k ~Dir(β),

[0040] z n ~Multinomial(θ d ),

[0041]

[0042] In the formula, θ d is the distribution of topics of document d in the document library, Dir is the Dirichlet distribution function, and α is the control θ d The sparsity hyperparameter, φ k is the distribution of high-frequency words in topic k, β is the control φ k The sparsity hyperparameter, z n is the topic corresponding to the nth high-frequency word, Multinomial is the Multinomial function, and w n is the nth high-frequency word, For z n Corresponding high-frequency word distribution.

[0043] Furthermore, the variational inference is:

[0044]

[0045] L(q)=E q [logp(w,z,θ,φ)]-E q [logq(θ,φ,z)];

[0046] Where KL is the Kullback-Leibler divergence, q(θ,φ,z) is the variational distribution, p(θ,φ,z|w) is the posterior distribution of the LDA topic model, θ is the distribution of the optimal topic of the document in the document library, φ is the distribution of high-frequency words in the optimal topic, z is the optimal topic, w is the high-frequency word, n is the sequence number of the high-frequency word, N is the fourth number, L(q) is the variational lower bound, E q [logp(w,z,θ,φ)] is the expectation of the joint distribution of the first distribution, the second distribution and the latent variable, E q [logq(θ,φ,z)] is the expectation of the variational distribution.

[0047] Furthermore, the word segmentation processing adopts the tool of jieba word segmentation.

[0048] Furthermore, the first distribution and the second distribution both include visualization data.

[0049] Compared with the prior art, the present invention has the following beneficial effects:

[0050] The present invention obtains a document library through preprocessing and a high-frequency vocabulary library through text mining, and constructs an LDA topic model based on the document library and the high-frequency vocabulary library, and adopts a perplexity analysis method and a consistency analysis method to obtain the optimal number of topics of the LDA topic model, and adopts a variational inference method to infer the document-optimal topic distribution and the optimal topic-high-frequency distribution, and accurately obtains the optimal topic name according to the inference result, so as to extract and interpret the core topics in the document, understand the key focus areas of the policy, policy development trends, and the relationship between the optimal topics.

[0051] The analysis method provided by the present invention helps to reveal the background, goals and impact of policy making, and thus provides a reference for subsequent policy making and implementation. Based on the extracted optimal topics, keywords and visualized data results, the policy focus, policy relevance and policy evolution trends are interpreted. This analysis can provide an in-depth understanding of the intentions and goals behind the policy, thus providing an important reference for policy makers and researchers. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is a flowchart of the policy-aware quantitative document analysis method based on the LDA topic model of the present invention;

[0053] Figure 2 is the number of energy storage related policy documents in the example of the present invention;

[0054] Figure 3 It is a perplexity curve diagram of the LDA topic model in the example of the present invention;

[0055] Figure 4 It is a consistency curve diagram of the LDA topic model in the example of the present invention;

[0056] Figure 5 The document-optimal topic distribution diagram of the LDA topic model in the example of the present invention;

[0057] Figure 6 This is the high-frequency word distribution diagram of the LDA topic model in the example of the present invention. DETAILED DESCRIPTION

[0058] It is worth noting that the methods used in the present invention are all conventional methods unless otherwise specified; the raw materials and devices used are all conventional commercially available products, and their sources are not specifically limited unless otherwise specified.

[0059] See also Figure 1 , a policy-aware quantitative document analysis method based on LDA topic model, including the following steps:

[0060] The source files are obtained and preprocessed to obtain documents, and a document library is constructed according to the documents; preferably, the steps of obtaining the document library are:

[0061] Get the preset time period and file source;

[0062] Collect source files within a preset time period from the file source;

[0063] Preprocess source files to obtain documents, and build a document library based on the documents;

[0064] Preferably, the preprocessing includes format unification and text deduplication;

[0065] Get stop words and build a stop word library based on them; stop words include standard stop words and custom stop words. Standard stop words are usually words that appear frequently but have no actual contribution to the topic or semantics, such as conjunctions, function words, punctuation marks, etc. Custom stop words are stop words built in combination with domain characteristics, that is, stop words built in combination with the characteristics of the energy storage policy research field. In energy storage policy research, some specific words may appear frequently, but their actual contribution to the topic model is low. They are mainly for the standardization and formal expression of policy wording, and are not helpful in distinguishing the topic of the policy: these words mainly include common policy words (such as "documents", "regulations", etc.), document nature words (such as "decision", "notification", etc.) and common policy expressions (such as "to ensure", "in accordance with", etc.), all of which are custom stop words;

[0066] Word segmentation is performed on documents in the document library to obtain a first word library; preferably, the word segmentation tool used is Jieba word segmentation; word segmentation is an important step in text processing. Since Chinese does not have natural separators like English, word segmentation is a basic step in Chinese natural language processing. Jieba is a Chinese word segmentation tool that can segment continuous Chinese sentences into word sequences based on multiple word segmentation algorithms;

[0067] The stop words in the stop word library are removed from the first word library to obtain a second word library, where the second word library is a collection of words to be selected; removing stop words is a very critical part for improving the efficiency and accuracy of text processing;

[0068] The TF-IDF method is used to process the candidate words in the second word library to obtain high-frequency words, and a high-frequency word library is constructed based on the high-frequency words; the TF-IDF method can help measure the importance of words in a document. TF-IDF is used to calculate the weight of each candidate word and identify words with high discrimination in the document. Candidate words with high weights are more likely to represent the main content or theme of the document, while candidates with low weights will be further filtered.

[0069] The steps to build a high-frequency word library are:

[0070] Get the first number f(t,d) and the first quantity N d , a second number N, a third number df(t) and a fourth number T; the first number is the number of times one of the candidate words appears in one of the documents in the document library; the first number is the total number of the candidate words in one of the documents in the document library; the second number N is the total number of documents in the document library; the third number df(t) is the total number of documents in the document library that include one of the candidate words; the fourth number S is the total number of high-frequency words in the high-frequency word library;

[0071] According to the first number f(t,d) and the first number N d Get the TF value. The TF value is the frequency of one of the candidate words in the document library appearing in one of the documents, that is, the word frequency, which means the ratio of the number of times the candidate word t appears in the document d to the total number of candidate words in the document d. The TF value is:

[0072]

[0073] In the formula, tf(t,d) is the frequency of the candidate word t in the document library in document d, f(t,d) is the number of times the candidate word t appears in document d in the document library, N d is the total number of words to be selected in document d of the document library;

[0074] The IDF value is obtained according to the second number N and the third number df(t). The IDF value is the rarity of one of the candidate words in the document library, that is, the inverse document frequency, which is used to measure the rarity of the candidate word t in the document library. The IDF value is:

[0075]

[0076] Where df(t) is the total number of documents d in the document library that include the word t to be selected;

[0077] The TF-IDF value is obtained based on the TF value and the IDF value. The TF-IDF value is the weight of one of the candidate words in the document library. TF-IDF combines the term frequency (TF) and the inverse document frequency (IDF) to measure the importance of the candidate word t in the document d in the document library. The TF-IDF value is:

[0078] TF-IDF(t,d,D)=tf(t,d)×idf(t,D);

[0079] Sort the words to be selected in the second vocabulary from high to low according to the TF-IDF value;

[0080] Based on the fourth number S, the sorted candidate words are selected to obtain high-frequency words, and a high-frequency word library is constructed based on the high-frequency words. That is, after sorting the candidate words in the second word library from high to low according to the TF-IDF value, the first S candidate words are taken as high-frequency words to form a high-frequency word library, that is, the number of high-frequency words in the high-frequency word library is S.

[0081] The LDA topic model is constructed based on the document library and high-frequency word library. During the construction of the LDA topic model, the LDA topic model assumes that each document is generated by a mixture of multiple topics, which can effectively identify the multi-topic structure in the document, reflecting the complexity and diversity of energy storage policies. The LDA topic model first generates the document-topic distribution based on the document library. For each document d, it is assumed that there are K potential topics. The LDA topic model assumes that the topic distribution of document d in the document library is θ d ,θ d Represents the topic weight distribution of document d in the document library. The LDA topic model considers θ d It conforms to the Dirichlet distribution Dir(α). Next, the LDA topic model generates the topic-high-frequency word (topic-word) distribution. For each topic k, the LDA topic model assumes that the high-frequency word distribution in the topic k is φ k ,φ k Represents the probability distribution of high-frequency words in a topic. k It also conforms to the Dirichlet distribution Dir(β). Finally, in each document d, for each high-frequency word w in the document n , from the document library d Extract a topic z from n , i.e., subject z n From the document library, the topic distribution θ corresponding to document d d and extract from the selected topic z n correspond Extract a high-frequency word w from n , that is, high-frequency word w n From topic z n Corresponding high-frequency word distribution Extraction; the LDA topic model is:

[0082] θ d ~Dir(α),

[0083] φ k ~Dir(β),

[0084] zn ~Multinomial(θ d ),

[0085]

[0086] In the formula, θ d is the distribution of topics of document d in the document library, Dir is the Dirichlet distribution function, and α is the control θ d The sparsity hyperparameter, φ k is the distribution of high-frequency words in topic k, β is the control φ k The sparsity hyperparameter, z n is the topic corresponding to the nth high-frequency word, Multinomial is the Multinomial function, and w n is the nth high-frequency word, For z n Corresponding high-frequency word distribution.

[0087] The perplexity analysis method is used to obtain the topic number interval of the LDA topic model;

[0088] Based on the topic number interval, the consistency analysis method is used to obtain the optimal number of topics for the LDA topic model, and the optimal number of topics is the number of optimal topics for the LDA topic model;

[0089] Preferably, the steps for obtaining the optimal number of topics are:

[0090] Obtain a first probability, a fifth quantity, a sixth quantity, and a seventh quantity; the first probability is the probability that one of the high-frequency words is in one of the documents in the document library; the fifth quantity is the total number of documents in the document library that include one of the high-frequency words; the sixth quantity is the total number of documents in the document library that include two of the high-frequency words; the seventh quantity is the number of high-frequency words considered in each topic;

[0091] Obtain the perplexity calculation formula and consistency calculation formula;

[0092] The perplexity calculation formula is:

[0093]

[0094] Where Perplexity(D) is the perplexity value of the LDA topic model, S is the total number of high-frequency words in the high-frequency word library, d is the sequence number of the document in the document library, D is the total number of documents in the document library, P(w d ) is the probability of high-frequency words in document d;

[0095] The consistency calculation formula is:

[0096]

[0097] In the formula, C is the consistency value of the LDA topic model, T k is the kth topic, M is the number of high-frequency words considered in each topic, m and l are the serial numbers of high-frequency words, and w m is the mth high-frequency word in the kth topic, w l is the lth high-frequency word in the kth topic, D(w m ,w l ) is a document library that contains w m 、w l The number of documents, ε is a smoothing parameter, usually a very small positive number to avoid the denominator being zero, D(w l ) is the document library containing w l The number of documents;

[0098] Based on the perplexity calculation formula, according to the fourth quantity S, the second quantity D and the first probability P(w d ) obtain the perplexity curve;

[0099] The topic number interval is obtained according to the perplexity curve; the perplexity analysis method is usually used to evaluate the performance of the LDA topic model under different topic numbers. Generally speaking, the perplexity value decreases as the number of topics increases, indicating that the LDA topic model fits the data better, but when the number of topics is too large, the perplexity value may stabilize or rise again, and the LDA topic model may be overfitting.

[0100] Based on the consistency calculation formula, a consistency curve is obtained according to the fifth quantity, the sixth quantity and the seventh quantity;

[0101] The optimal number of topics is obtained by selecting the interval of the number of topics according to the consistency curve. The consistency value obtained by the consistency analysis method is used to evaluate the semantic rationality of the topic. A higher consistency value indicates that the generated topic is more semantically reasonable and easy to interpret. Usually, the number of topics with a higher consistency value is selected as the optimal number of topics. The consistency analysis method is judged by calculating the co-occurrence frequency between high-frequency words in a certain topic. For every two high-frequency words, if they appear more times in the same document, that is, the co-occurrence frequency is higher, then their semantic correlation is stronger, which makes the consistency of the topic higher.

[0102] Based on the optimal number of topics, the variational inference method is used to infer the LDA topic model to obtain the first distribution and the second distribution; the first distribution is the distribution of the optimal topic of one of the documents in the document library, and the second distribution is the distribution of high-frequency words in one of the optimal topics;

[0103] During the inference process, the goal of the LDA topic model is to infer the optimal topic distribution θ for each document and the high-frequency word distribution φ for each topic given a document set, i.e., a document library. The inference method is to perform approximate inference through variational inference. In the LDA topic model, we usually want to calculate the posterior distribution p(θ, φ, z|w) of the LDA topic model, where w is a high-frequency word. Since the calculation of this posterior distribution is usually infeasible, the goal of variational inference is to approximate the posterior distribution by introducing a simpler distribution, the variational distribution q(θ, φ, z). We hope to find the optimal variational distribution by minimizing the Kullback-Leibler divergence (KL divergence). In order to perform variational inference, we need to select a variational distribution to approximate the true posterior distribution. The usual choice is to use a factorized form, that is, to assume that the latent variables are independent. Finally, by introducing the variational distribution, the lower bound of the log marginal likelihood is derived. Variational inference is:

[0104]

[0105] L(q)=E q [logp(w,z,θ,φ)]-E q [logq(θ,φ,z)];

[0106] Where KL is the Kullback-Leibler divergence, q(θ,φ,z) is the variational distribution, p(θ,φ,z|w) is the posterior distribution of the LDA topic model, θ is the distribution of the optimal topic of the document in the document library, φ is the distribution of high-frequency words in the optimal topic, z is the optimal topic, w is the high-frequency word, n is the sequence number of the high-frequency word, N is the fourth number, L(q) is the variational lower bound, E q [logp(w,z,θ,φ)] is the expectation of the joint distribution of the first distribution, the second distribution and the latent variable, E q [logq(θ,φ,z)] is the expectation of the variational distribution.

[0107] After inference is completed, the LDA topic model outputs the topic distribution θ of document d in the document library. d And the distribution of high-frequency words in topic k φ k ,Through these distributions, we can identify the optimal topics in the documents and the keywords of each optimal topic.

[0108] Preferably, both the first distribution and the second distribution include visualization data.

[0109] Obtaining an optimal topic name for each optimal topic according to the first distribution and the second distribution;

[0110] The purpose of policy analysis is to extract and interpret the best topics, i.e. core topics, in the document to understand the key areas of policy focus, policy development trends, and the relationship between the best topics. This analysis helps to reveal the background, goals, and impact of policy making, and thus provide a reference for subsequent policy making and implementation. Based on the extracted best topics, keywords, and visualized data results, the policy focus, policy linkage, and policy evolution trends are interpreted. This analysis can provide an in-depth understanding of the intentions and goals behind the policy, thus providing an important reference for policymakers and researchers.

[0111] In addition, the present disclosure also has the following beneficial effects:

[0112] First, the present invention adopts a method combining perplexity and consistency to determine the optimal number of topics, which not only considers the statistical performance of the LDA topic model, but also pays attention to the semantic quality of the topic, which can significantly improve the application effect and explanatory power of the LDA topic model.

[0113] Secondly, the present disclosure visualizes the results of the LDA topic model with the help of a visualization analysis tool. From the visualized bubble chart, one can intuitively understand the importance of each topic and the relationship between them.

[0114] Calculation example

[0115] Obtain and preprocess source files to obtain documents, and build a document library based on the documents:

[0116] After preprocessing such as unified format and text deduplication, this example constructs a document library through the processed documents of 65 energy storage-related policy documents issued by the central government from 2019 to 2024, and conducts the following analysis. The document library is the specific number of energy storage-related policy documents. Figure 2 shown.

[0117] The documents in the document library are segmented, stop words are removed, and TF-IDF processing is performed to obtain high-frequency words and build a high-frequency word library. The LDA topic model is built based on the document library and the high-frequency word library. The perplexity curve of the LDA topic model obtained by the perplexity analysis method (see Figure 2 ), and the consistency curve of the LDA topic model obtained by the consistency analysis method (see Figure 3 ).

[0118] From the perplexity curve, we can see that the perplexity shows an overall trend of first decreasing and then stabilizing as the number of topics increases. When the number of topics is in [1,10], the perplexity decreases rapidly, and when the number of topics is in [10,13], the perplexity stabilizes. After 13, the perplexity shows a slight upward trend. Therefore, it can be determined that the number of topics in the LDA topic model is in the range of [10,13]. In order to further determine the optimal number of topics for the LDA topic model, this paper calculates the consistency of the LDA topic model.

[0119] In the LDA topic model, consistency is used to measure the quality of topics generated by the LDA topic model. Its main function is to evaluate the rationality and interpretability of the LDA topic model, that is, whether the generated topics are semantically coherent. From the consistency curve, it can be seen that the consistency of the LDA topic model gradually decreases when the number of topics is in the range of [10,13]. When the number of topics is 10, the consistency of the LDA topic model is the highest. Therefore, the optimal number of topics for the LDA topic model is determined to be 10.

[0120] See also Figure 5-6 ,After determining the optimal number of topics of the LDA topic model, the ,variational inference method is used to infer the LDA topic model, and ,visualized data results of the 10 optimal topics of the LDA topic model can be generated.

[0121] Figure 5 The document-optimal topic bubble distribution diagram in shows the distance relationship between different optimal topics. Each bubble represents an optimal topic, and the size of the bubble is proportional to the marginal distribution of the optimal topic. The distance between the optimal topics indicates their similarity in the latent space. The closer the optimal topics are, the more similar they are in content. As can be seen from the figure, there is a good degree of distinction between the optimal topics 5, 7, 9, and 10, and there is partial overlap between the optimal topics 1, 2, 3, 4, 6, and 8, which indicates that these optimal topics have similar content in the latent space and may have some vocabulary overlap. At the same time, the optimal topic-high-frequency word distribution of the LDA topic model can be obtained as shown in Table 1.

[0122] Table 1 Optimal topic-high frequency word distribution table of LDA topic model

[0123]

[0124] According to the characteristics of high-frequency words under each optimal topic, the 10 optimal topics are named respectively. Specifically:

[0125] The optimal topic 1 reflects the government's strong support for the development of new energy, especially policies related to renewable energy such as photovoltaics and wind power. Keywords, i.e. high-frequency words such as "green" and "low-carbon", show the environmental orientation of the policy, while "development" and "utilization" indicate that the policy encourages enterprises to actively participate in new energy projects and includes support for energy storage technology to improve the utilization efficiency of renewable energy and the stability of the power grid.

[0126] Best Topic 2 This topic focuses on the development of energy storage technology and its position in policy, especially the level of market regulation. Keywords such as "regulation" and "standards" indicate that policies are improving the technical standards and regulatory framework for energy storage to ensure the safety and reliability of energy storage systems, while encouraging market-oriented operations.

[0127] The best theme 3 focuses on the role of local governments in promoting the development of the green energy storage industry. The keywords "green", "industry", "enterprise" and "region" show the focus of the theme's industrial promotion and regional economic reform. The policy focuses on supporting measures for regional economic development, including industrial support policies, standardization promotion and mechanism innovation to promote the regional development and industrialization of green energy storage technology.

[0128] The best theme 4 reflects the policy’s emphasis on energy storage technology R&D and innovative demonstration projects. Keywords such as “demonstration”, “key research” and “platform” indicate that the policy includes supporting the R&D of key energy storage technologies and verifying their application value through demonstration projects, providing a basis for large-scale promotion.

[0129] The best topic 5 focuses on the application of energy storage systems in power dispatching and its management policies. Keywords such as "dispatching", "grid connection" and "management" indicate that its focus is on the coordination and operation of the power system. The policy involves the grid connection, operation and dispatch of energy storage systems and their role in the power system, emphasizing the guarantee of grid stability and operational efficiency, and the establishment of a sound management and maintenance mechanism.

[0130] The optimal topic 6 focuses on the role of energy storage facilities in power grid construction. Keywords such as "power system", "consumption", and "regulation" indicate that the optimal topic 6 involves how to improve the stability and operating efficiency of the power grid system. Policies may encourage the use of energy storage technology to improve the grid's consumption and regulation capabilities to ensure the stable access of new energy and the safe operation of the power grid.

[0131] The best topic 7 explores the policy direction of energy storage marketization, especially the role of energy storage in the electricity spot market. Keywords such as "market", "transaction", and "spot" show that this topic is related to the operation mechanism and information transparency of the electricity market. Policies may involve information disclosure requirements, participation mechanisms in the ancillary service market, and the status of energy storage facilities in electricity market transactions to promote the marketization of energy storage technology.

[0132] The best topic 8 focuses on the construction of energy storage facilities. Keywords such as "charging", "infrastructure" and "demand" show that it is related to the use of new energy and the construction of charging networks. The policy focuses on the role of energy storage facilities as power infrastructure and promotes their application in meeting the demand for new energy electricity, including the improvement of service evaluation standards for industrial enterprises and other users.

[0133] The best topic 9 mainly discussed the cost and price policy of energy storage systems in the transmission and distribution link. Keywords such as "cost", "pricing" and "regulation" indicate that the discussion is mainly about the pricing and government regulation of the transmission and distribution business. The policy mainly revolves around the cost verification of energy storage facilities, the pricing mechanism and the supervision of related expenses to ensure that enterprises can obtain reasonable returns and control the electricity costs at the user end.

[0134] The best topic 10 focuses on power trading contracts and marketization processes. Keywords such as "trading", "contract", and "marketization" indicate that it involves the content of power trading marketization and contract management. The policy focuses on regulating the contract signing and marketization operation of energy storage facilities in medium- and long-term power trading, emphasizing safeguards for users and enterprises to ensure fairness and transparency of transactions. The specific names of each topic are shown in Table 2.

[0135] Table 2 Optimal topic names

[0136] theme Theme Naming Best Theme 1 New energy development and energy storage support Best Theme 2 Energy storage technology and market regulation Best Theme 3 Regional Economy and Green Energy Storage Industry Best Theme 4 Energy Storage Technology Innovation and Demonstration Project Best Theme 5 Energy storage dispatch and power system management Best Theme 6 Power grid and energy storage facility construction Best Theme 7 Energy storage market and trading mechanism Best Theme 8 New energy management and charging facilities Best Theme 9 Energy storage costs and price regulation Top 10 Themes Power Trading and Marketization

[0137] Through the above analysis, we can find that in terms of policy formulation, in recent years, my country has actively promoted the development of new energy, especially green and low-carbon energy such as photovoltaics and wind power, and accelerated the development and construction of new energy projects, and the ability of enterprises to use clean energy has been continuously improved. At the same time, energy storage technology, as an emerging field and an important part of the new power system, has received strong support from the country, and has been gradually applied to grid regulation, improving energy efficiency, and other aspects, and technological innovation and demonstration projects have continued to increase. In addition, promoting the construction and development of a green industrial system is also a key area of ​​national policy attention, especially in the promotion of green technology and energy use in the industrial field, and the stability and absorption capacity of the power system have been improved. At present, the market-oriented reform of electricity is also being continuously promoted, and mechanisms such as the spot market and ancillary service market are gradually being improved, and the enthusiasm of enterprises and users to participate in the market has been enhanced. Finally, my country has gradually established a regulatory system for pricing mechanisms and supervision, and the pricing mechanism of provincial power grid companies has been further clarified and refined.

[0138] Finally, it should be noted that the above content is only used to illustrate the technical solution of the present invention, rather than to limit the scope of protection of the present invention. Simple modifications or equivalent substitutions of the technical solution of the present invention by ordinary technicians in this field do not deviate from the essence and scope of the technical solution of the present invention.

Claims

1. A policy-aware quantitative document analysis method based on LDA topic model, characterized by: The following steps are involved: Acquire and preprocess source files to obtain documents, and build a document library based on the documents; Get stop words and build a stop word library based on the stop words; Process the documents in the document library through word segmentation to obtain a first word library; Removing the stop words in the stop word library from the first word library to obtain a second word library, wherein the second word library is a set of words to be selected; Using the TF-IDF method to process the words to be selected in the second vocabulary to obtain high-frequency words, and constructing a high-frequency vocabulary based on the high-frequency words; Constructing an LDA topic model based on the document library and the high-frequency word library; Using a perplexity analysis method to obtain the topic number interval of the LDA topic model; Based on the topic number interval, a consistency analysis method is used to obtain the optimal topic number of the LDA topic model, where the optimal topic number is the number of optimal topics of the LDA topic model; Based on the optimal number of topics, a variational inference method is used to infer the LDA topic model to obtain a first distribution and a second distribution; the first distribution is the distribution of the optimal topic of one of the documents in the document library, and the second distribution is the distribution of the high-frequency words in one of the optimal topics; Obtaining an optimal topic name for each of the optimal topics according to the first distribution and the second distribution; The document library is analyzed according to the optimal topic names.

2. The policy-aware quantitative document analysis method based on the LDA topic model according to claim 1 is characterized by: The steps of obtaining the document library are: Get the preset time period and file source; Collecting the source files within the preset time period from the file source; The source file is preprocessed to obtain the document, and the document library is constructed according to the document.

3. The policy-aware quantitative document analysis method based on the LDA topic model according to claim 1 is characterized by: The preprocessing includes format unification and text deduplication.

4. The policy-aware quantitative document analysis method based on the LDA topic model according to claim 1 is characterized by: The stop words include standard stop words and custom stop words, and the custom stop words are stop words constructed in combination with domain features.

5. The policy-aware quantitative document analysis method based on the LDA topic model according to claim 1 is characterized by: The steps of constructing the high-frequency word library are: Obtain a first number, a first quantity, a second number, a third number and a fourth number; the first number is the number of times one of the words to be selected appears in one of the documents in the document library; the first quantity is the total number of the words to be selected in one of the documents in the document library; the second number is the total number of documents in the document library; the third number is the total number of the documents in the document library that include one of the words to be selected; the fourth number is the total number of the high-frequency words in the high-frequency word library; Obtaining a TF value according to the first times and the first quantity, wherein the TF value is a frequency of one of the words to be selected in one of the documents in the document library; Obtaining an IDF value according to the second number and the third number, wherein the IDF value is the rarity of one of the words to be selected in the document library; Obtaining a TF-IDF value according to the TF value and the IDF value, wherein the TF-IDF value is a weight of one of the words to be selected in the document library; sorting the words to be selected in the second vocabulary from high to low according to the TF-IDF value; The high-frequency words are obtained by selecting the sorted words to be selected based on the fourth quantity, and the high-frequency word library is constructed according to the high-frequency words.

6. The policy-aware quantitative document analysis method based on the LDA topic model according to claim 5 is characterized by: The steps for obtaining the optimal number of topics are: Obtain a first probability, a fifth quantity, a sixth quantity, and a seventh quantity; the first probability is the probability that one of the high-frequency words is in one of the documents in the document library; the fifth quantity is the total number of the documents in the document library that include one of the high-frequency words; the sixth quantity is the total number of the documents in the document library that include two of the high-frequency words; the seventh quantity is the number of the high-frequency words considered in each topic; Obtain the perplexity calculation formula and consistency calculation formula; Based on the perplexity calculation formula, obtaining a perplexity curve according to the fourth quantity, the second quantity and the first probability; Obtaining the topic number interval according to the perplexity curve; Based on the consistency calculation formula, obtaining a consistency curve according to the fifth quantity, the sixth quantity and the seventh quantity; The optimal number of topics is obtained by selecting the topic number interval according to the consistency curve.

7. The policy-aware quantitative document analysis method based on the LDA topic model according to claim 1 is characterized by: The LDA topic model is: i d ~Dir(a), f k ~Dir(β), With n ~Multinomial(θ d ), In the formula, θ d is the distribution of topics of document d in the document library, Dir is the Dirichlet distribution function, and α is the control θ d The sparsity hyperparameter, φ k is the distribution of high-frequency words in topic k, β is the control φ k The sparsity hyperparameter, z n is the topic corresponding to the nth high-frequency word, Multinomial is the Multinomial function, and w n is the nth high-frequency word, For z n Corresponding high-frequency word distribution.

8. The policy-aware quantitative document analysis method based on the LDA topic model according to claim 7 is characterized by: The variational inference is: L(q)=E q [logp(w,z,θ,φ)]-E q [logq(θ,φ,z)]; Where KL is the Kullback-Leibler divergence, q(θ,φ,z) is the variational distribution, p(θ,φ,z|w) is the posterior distribution of the LDA topic model, θ is the distribution of the optimal topic of the document in the document library, φ is the distribution of high-frequency words in the optimal topic, z is the optimal topic, w is the high-frequency word, n is the sequence number of the high-frequency word, N is the fourth number, L(q) is the variational lower bound, and E q [logp(w,z,θ,φ)] is the expectation of the joint distribution of the first distribution, the second distribution and the latent variable, E q [logq(θ,φ,z)] is the expectation of the variational distribution.

9. The policy-aware quantitative document analysis method based on the LDA topic model according to claim 1 is characterized by: The word segmentation process uses the jieba word segmentation tool.

10. The policy-aware quantitative document analysis method based on the LDA topic model according to claim 1, characterized in that: The first distribution and the second distribution each include visualization data.

Images