Text analysis method and system

Abstract

The embodiment of the invention discloses a text analysis method and system, and the method comprises the steps: obtaining a certain number of policy texts, carrying out the preprocessing of the policy texts, and obtaining the preprocessing data; inputting the preprocessed data into a self-adaptive model for training to obtain an optimal self-adaptive model; the method comprises the following steps: receiving preprocessed data through an input layer, carrying out division processing on the preprocessed data through a multi-level division layer to obtain divided data, mapping the divided data to a low-dimensional vector space through an embedding presentation layer to obtain an embedded vector of the divided data, carrying out topic classification on the embedded vector through a topic modeling layer, and carrying out topic classification on the embedded vector. Performing structured interpretation on the embedded vectors after subject classification through an output layer, and outputting structured data of a policy relationship corresponding to the policy text; and inputting a to-be-analyzed policy text into the optimal adaptive model to obtain structured data of a policy relationship corresponding to the to-be-analyzed policy text. The problem that industrial policies cannot be deeply, comprehensively and accurately analyzed in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a text analysis method and system. Background Art

[0002] With the acceleration of the global digital transformation process, the penetration of information technology in various industries continues to deepen, and emerging industries are booming. The technological development trends, market demand changes, and policy support in various industries play a vital role in the prediction of industrial development paths and the formulation of supporting policies. Especially in the global environment where international competition is becoming increasingly fierce, reasonable policy guidance and effective market support mechanisms are key elements in enhancing industrial innovation capabilities and international competitiveness, which makes it imperative to build a digital platform to support industrial development strategies, and the establishment and improvement of the industrial policy database is the top priority.

[0003] At present, industrial policy data mainly comes from various channels, including various policy documents, industry reports and government planning documents, etc. However, the current industrial policy analysis methods have many limitations, which seriously affect the comprehensive understanding and effective use of policy connotations.

[0004] In existing industrial policy analysis practices, traditional methods are mainly based on basic text retrieval techniques, such as keyword matching and simple text classification.

[0005] These methods have obvious deficiencies when dealing with policy documents. First, in terms of cross-document analysis, existing technologies mostly focus on the content of a single policy document and lack the ability to systematically compare the internal connections between different policy documents. This makes it impossible to track and analyze the evolution of policies and to grasp the coherence of policy development from a macro perspective. Second, from the perspective of analysis, traditional methods are limited to similarity analysis at the lexical level. This approach ignores the more complex connections and potential strategic intentions contained in the policy content, making the analysis results too superficial and unable to deeply explore the deep meaning of the policy. Third, in terms of policy relevance judgment, existing technologies are not accurate enough. Especially in a complex policy environment, it is difficult to capture the deep connections between different policy documents, and subtle policy differences are easily overlooked, which in turn affects the accurate assessment of the overall impact of the policy. Finally, in the face of large-scale literature data, due to the low efficiency of existing analysis methods, they are struggling to process massive policy documents and text data, and cannot achieve accurate and efficient analysis, thus limiting the effective use of large-scale industrial policy data.

[0006] To sum up, the traditional industrial policy analysis method can no longer meet the current industrial development's demand for policy analysis. A new and more effective policy analysis method is urgently needed to overcome the limitations of existing technologies and achieve a more in-depth, comprehensive and accurate analysis of industrial policies. Summary of the invention

[0007] The purpose of the embodiments of the present invention is to provide a method and system for text analysis to solve the problem that the prior art cannot conduct in-depth, comprehensive and accurate analysis of industrial policies.

[0008] To achieve the above object, an embodiment of the present invention provides a text analysis method, which specifically includes:

[0009] Acquire a certain number of policy texts, preprocess the policy texts, and obtain preprocessed data;

[0010] Constructing an adaptive model, inputting the preprocessed data into the adaptive model for training to obtain an optimal adaptive model, wherein the adaptive model includes an input layer, a multi-level division layer, an embedding representation layer, a topic modeling layer and an output layer connected in sequence;

[0011] The preprocessed data is received through an input layer, the preprocessed data is partitioned through a multi-level partitioning layer to obtain partitioned data, the partitioned data is mapped to a low-dimensional vector space through an embedding representation layer to obtain an embedding vector of the partitioned data, the embedded vector is subject-classified through a topic modeling layer, and the embedded vector after subject classification is structured interpreted through an output layer to output structured data of policy text corresponding to a policy relationship;

[0012] The policy text to be analyzed is input into the optimal adaptive model to obtain structured data of the policy relationship corresponding to the policy text to be analyzed.

[0013] On the basis of the above technical solution, the present invention can also be improved as follows:

[0014] Furthermore, the obtaining of a certain number of policy texts and preprocessing of the policy texts to obtain preprocessed data includes:

[0015] Clean the policy text to remove irrelevant characters, extra spaces and formatting marks in the policy text;

[0016] Convert policy texts into a uniform, standardized coding format;

[0017] Break down the policy text into words or phrases.

[0018] Furthermore, the pre-processed data is partitioned by a multi-level partitioning layer to obtain partitioned data, including:

[0019] Dividing and processing the pre-processed data based on policy dimensions and policy labels;

[0020] Among them, the policy dimensions include economy, politics, social welfare, technological innovation and environment;

[0021] The policy labels under the economic dimension include performance orientation, economic institutions, economic development, attracting FDI, enterprise development, industrial planning, policy tools, and coordination; the policy labels under the political dimension include political institutions, government efficiency, public security and social stability, united front, and legal development; the policy labels under the social welfare dimension include employment, people's livelihood and medical care, urbanization, agriculture, culture, education, and tourism; the policy labels under the technological innovation dimension include technology and innovation; and the policy labels under the environmental dimension include environment.

[0022] Furthermore, mapping the partitioned data to a low-dimensional vector space through an embedding representation layer to obtain an embedding vector of the partitioned data includes:

[0023] According to the semantic relationship and occurrence frequency of words or phrases in the divided data, a corresponding embedding vector is generated for each divided data.

[0024] Furthermore, the topic classification of the embedded vectors through the topic modeling layer includes:

[0025] Based on the distribution characteristics and semantic similarity of the embedding vectors, each embedding vector is assigned to the corresponding topic category.

[0026] Furthermore, the text analysis method further includes:

[0027] Dividing the preprocessed data into a training set, a validation set, and a test set;

[0028] Training the adaptive model based on the training set;

[0029] Performing a performance evaluation on the trained adaptive model based on the validation set to obtain an adaptive model that meets the performance requirements;

[0030] The prediction result of the adaptive model that meets the performance condition is evaluated based on the test set to obtain an evaluation index corresponding to the adaptive model.

[0031] Furthermore, the text analysis method further includes:

[0032] The parameters of the adaptive model are adjusted and updated through an adaptive learning algorithm so that the adaptive model is continuously optimized to adapt to new policy texts.

[0033] A text analysis system, comprising:

[0034] A preprocessing module, used for acquiring a certain number of policy texts, preprocessing the policy texts, and obtaining preprocessed data;

[0035] A construction module, used for constructing an adaptive model, inputting the preprocessed data into the adaptive model for training to obtain an optimal adaptive model, wherein the adaptive model includes an input layer, a multi-level division layer, an embedding representation layer, a topic modeling layer and an output layer connected in sequence;

[0036] The preprocessed data is received through an input layer, the preprocessed data is partitioned through a multi-level partitioning layer to obtain partitioned data, the partitioned data is mapped to a low-dimensional vector space through an embedding representation layer to obtain an embedding vector of the partitioned data, the embedded vector is subject-classified through a topic modeling layer, and the embedded vector after subject classification is structured interpreted through an output layer to output structured data of policy text corresponding to a policy relationship;

[0037] The prediction module is used to input the policy text to be analyzed into the optimal adaptive model to obtain structured data of the policy relationship corresponding to the policy text to be analyzed.

[0038] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the steps of the method are implemented when the processor executes the computer program.

[0039] A non-transitory computer-readable storage medium stores a computer program, which implements the steps of the method when executed by a processor.

[0040] The embodiments of the present invention have the following advantages:

[0041] The text analysis method of the present invention can deeply mine the semantic information of the policy text through the collaborative work of the multi-level division layer, the embedding representation layer and the topic modeling layer. The multi-level division layer can analyze the policy text from different angles, the embedding representation layer maps the divided data to the low-dimensional vector space to obtain the embedding vector, effectively capturing the semantic relationship in the text, and the topic modeling layer further classifies these embedding vectors by subject, thereby avoiding the limitation of the traditional method based only on surface keyword matching, more accurately understanding the policy content, and improving the accuracy of policy relationship analysis.

[0042] Preprocessing the policy text before inputting it into the adaptive model can clean and standardize the data, allowing the adaptive model to better focus on key information. During the processing of the adaptive model, the various layers cooperate with each other to fully consider the various characteristics of the policy text. Unlike traditional analysis methods, it will not ignore the deep logical relationship behind the policy, reduce information omissions, and present the relationship between policies more accurately.

[0043] The structure of the adaptive model enables it to analyze policy texts from multiple dimensions. Unlike traditional methods that only focus on the content of a single policy document or simple lexical similarity analysis, this method can compare and analyze policies at multiple levels and dimensions, thereby building policy relationships more systematically. Whether it is the similarities and differences in policy content or the associations in the process of policy evolution, they can all be fully considered, achieving a systematic comparison across documents.

[0044] An adaptive model is constructed and trained with a large amount of preprocessed data to obtain the optimal model. The model is designed to adapt to large-scale policy text data. Compared with the inefficiency of traditional analysis methods in processing massive policy documents and text data, the model of this method can efficiently output the structured data of the corresponding policy relationship for the policy text to be analyzed after training, which improves the efficiency of processing large-scale data.

[0045] The output layer outputs structured data of policy texts corresponding to policy relationships. This structured form is convenient for storage, query and further analysis. It provides convenience for both the rapid retrieval of large numbers of policy documents and the construction and analysis of complex policy relationship networks, further improving processing efficiency.

[0046] By deeply processing the policy texts, we can accurately identify the similarities and differences in policy content and deeply explore the policy evolution process. This allows policymakers to clearly understand the development context of the policy and discover the trends and laws of policy changes, thereby providing a strong basis for formulating more scientific, reasonable, forward-looking and coherent policies, which will help enhance the industry's innovation capabilities and international competitiveness. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the implementation methods of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the implementation methods or the description of the prior art. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other implementation drawings can be derived from the provided drawings without creative work.

[0048] The structures, proportions, sizes, etc. illustrated in this specification are only used to match the contents disclosed in the specification so as to facilitate understanding and reading by persons familiar with the technology. They are not used to limit the conditions under which the present invention can be implemented, and therefore have no substantial technical significance. Any structural modification, change in proportion or adjustment of size shall still fall within the scope of the technical contents disclosed in the present invention without affecting the effects and purposes that can be achieved by the present invention.

[0049] Figure 1 A flowchart of the method for text analysis of the present invention;

[0050] Figure 2 The structure diagram of the text analysis system of the present invention;

[0051] Figure 3 A schematic diagram of the physical structure of an electronic device provided by the present invention;

[0052] Figure 4 It is a schematic diagram of the five policy dimensions and 20 policy labels of the present invention.

[0053] The accompanying drawings are denoted as follows:

[0054] Preprocessing module 10 , construction module 20 , prediction module 30 , electronic device 40 , processor 401 , memory 402 , bus 403 . DETAILED DESCRIPTION

[0055] The following is a description of the implementation of the present invention by specific embodiments. People familiar with the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] Figure 1 Flow chart of the text analysis method embodiment of the present invention, as shown in Figure 1 As shown, a text analysis method provided by an embodiment of the present invention includes the following steps:

[0057] S101, obtaining a certain number of policy texts, preprocessing the policy texts, and obtaining preprocessed data;

[0058] Specifically: Clean the policy text to remove irrelevant characters, extra spaces and formatting marks in the policy text;

[0059] Convert policy texts into a uniform, standardized coding format;

[0060] Break down the policy text into words or phrases.

[0061] Preprocessing the policy text before inputting it into the adaptive model can clean and standardize the data, allowing the adaptive model to better focus on key information. During the processing of the adaptive model, the various layers cooperate with each other to fully consider the various characteristics of the policy text. Unlike traditional analysis methods, it will not ignore the deep logical relationship behind the policy, reduce information omissions, and present the relationship between policies more accurately.

[0062] S102, constructing an adaptive model, inputting the preprocessed data into the adaptive model for training, so as to obtain an optimal adaptive model;

[0063] Specifically, the adaptive model includes an input layer, a multi-level division layer, an embedding representation layer, a topic modeling layer and an output layer which are connected in sequence.

[0064] Divide the policy text into training, validation and test sets;

[0065] Train the adaptive model based on the training set;

[0066] Perform performance evaluation on the trained adaptive model based on the validation set to obtain an adaptive model that meets the performance requirements;

[0067] The prediction results of the adaptive model that meets the performance conditions are evaluated based on the test set, and the evaluation index corresponding to the adaptive model is obtained.

[0068] Based on the validation set, the performance of the trained adaptive model is evaluated to obtain an adaptive model that meets the performance conditions; based on the test set, the similarity calculation results of the adaptive model that meets the performance conditions are evaluated to obtain the evaluation index corresponding to the adaptive model. The performance of the adaptive model is evaluated to obtain a percentage score (i.e., the highest score is 100 points and the lowest score is 0 points), and the adaptive model with a score greater than the set value is determined based on the percentage score. For example, an adaptive model with a score greater than 90 points is an adaptive model that meets the performance conditions.

[0069] An evaluation index is calculated for the adaptive model that meets the performance conditions to obtain the evaluation index of the adaptive model. An evaluation value corresponding to each evaluation index is calculated to represent the capability value of the adaptive model on the evaluation index.

[0070] In the process of model construction, the present invention introduces a multilayer embedded clustering model (Multilayer EmbeddedClustering Model), which independently projects text terms into 20 policy label vector spaces based on the high-order semantic affinity of the text. Due to the ambiguity of terms and high-dimensional feature mapping, some words may have overlapping semantic embeddings and thus be assigned to multiple policy dimensions. For the focus terms with divergent encodings, the Adaptive Iterative Labeling Model is used. After four rounds of nested iterations, the encoding confidence reaches 80%.

[0071] Based on the constructed policy dimension system, each document is encoded into a 20-dimensional policy vector space, where the components of each dimension serve as potential proxy variables for government resource allocation on a specific policy dimension. Through Hierarchical Bayesian Inference, it is assumed that local governments present a conjugate pattern of "policy density-resource allocation" in resource distribution, that is, high-frequency representations are generated for high-priority policy dimensions, while low-priority dimensions show relatively sparse feature distributions. The results show that the adaptive model successfully captures the complex differences in local government resource allocation across dimensions and time series.

[0072] An adaptive model is constructed and trained with a large amount of preprocessed data to obtain the optimal model. The model is designed to adapt to large-scale policy text data. Compared with the inefficiency of traditional analysis methods in processing massive policy documents and text data, the model of this method can efficiently output the structured data of the corresponding policy relationship for the policy text to be analyzed after training, which improves the efficiency of processing large-scale data.

[0073] S103, receiving preprocessed data through the input layer, performing partitioning processing on the preprocessed data through the multi-level partitioning layer to obtain partitioned data, mapping the partitioned data to a low-dimensional vector space through the embedding representation layer to obtain an embedding vector of the partitioned data, subject-classifying the embedded vector through the topic modeling layer, and performing structured interpretation on the embedded vector after subject classification through the output layer to output structured data of policy text corresponding to policy relationship;

[0074] Specifically, the pre-processed data is partitioned by a multi-level partitioning layer to obtain the partitioned data, including:

[0075] Dividing and processing the preprocessed data based on policy dimensions and policy labels;

[0076] like Figure 4 As shown, among them, the policy dimensions include economy, politics, social welfare, technological innovation and environment;

[0077] The policy labels under the economic dimension include performance orientation, economic institutions, economic development, attracting FDI, enterprise development, industrial planning, policy tools, and coordination; the policy labels under the political dimension include political institutions, government efficiency, public security and social stability, united front, and legal development; the policy labels under the social welfare dimension include employment, people's livelihood and medical care, urbanization, agriculture, culture, education, and tourism; the policy labels under the technological innovation dimension include technology and innovation; and the policy labels under the environmental dimension include environment.

[0078] Mapping the partitioned data to a low-dimensional vector space through an embedding representation layer to obtain an embedded vector of the partitioned data includes:

[0079] According to the semantic relationship and occurrence frequency of words or phrases in the divided data, a corresponding embedding vector is generated for each divided data.

[0080] The topic classification of the embedded vectors through the topic modeling layer includes:

[0081] Based on the distribution characteristics and semantic similarity of the embedding vectors, each embedding vector is assigned to the corresponding topic category.

[0082] Through the collaborative work of the multi-level division layer, the embedding representation layer and the topic modeling layer, the semantic information of the policy text can be deeply mined. The multi-level division layer can analyze the policy text from different angles. The embedding representation layer maps the divided data to a low-dimensional vector space to obtain the embedding vector, effectively capturing the semantic relationship in the text. The topic modeling layer further classifies these embedding vectors by subject, thus avoiding the limitations of traditional methods based only on surface keyword matching, more accurately understanding the policy content, and improving the accuracy of policy relationship analysis. The output layer outputs the structured data of the policy text corresponding to the policy relationship. This structured form is convenient for storage, query and further analysis, which provides convenience for both the rapid retrieval of a large number of policy documents and the construction and analysis of complex policy relationship networks, and further improves processing efficiency.

[0083] S104, inputting the policy text to be analyzed into the optimal adaptive model to obtain structured data of the policy relationship corresponding to the policy text to be analyzed;

[0084] Specifically, the structure of the adaptive model enables it to analyze policy texts from multiple dimensions. Unlike traditional methods that only focus on the content of a single policy document or simple lexical similarity analysis, this method can compare and analyze policies at multiple levels and dimensions, thereby constructing policy relationships in a more systematic way. Whether it is the similarities and differences in policy content or the associations in the process of policy evolution, they can all be fully considered, achieving a systematic comparison across documents.

[0085] By deeply processing the policy texts, we can accurately identify the similarities and differences in policy content and deeply explore the policy evolution process. This allows policymakers to clearly understand the development context of the policy and discover the trends and laws of policy changes, thereby providing a strong basis for formulating more scientific, reasonable, forward-looking and coherent policies, which will help enhance the industry's innovation capabilities and international competitiveness.

[0086] The text analysis method further includes:

[0087] The parameters of the adaptive model are adjusted and updated through an adaptive learning algorithm so that the adaptive model is continuously optimized to adapt to new policy texts.

[0088] This paper constructs a twenty-dimensional policy distribution theme for each policy report released through an adaptive model, and then captures the distribution of resource allocation characteristics in each policy dimension. The heterogeneity of resource distribution in key policy dimensions may be driven by the synergy of multiple hierarchical variables such as city characteristics, time series effects, and individual effects of leaders. In order to strip away the potential influence of political leaders on resource allocation patterns, a combination of factorization machine and latent semantic decomposition is used to deconstruct individual independence in multi-factor interaction through a hierarchical Bayesian inference process, ultimately achieving multi-dimensional composite analysis and deep modeling of strategy distribution patterns.

[0089] Figure 2 This is a system architecture diagram for text analysis of the present invention; Figure 2 As shown, a text analysis system provided by an embodiment of the present invention includes the following steps:

[0090] A preprocessing module 10 is used to obtain a certain number of policy texts, preprocess the policy texts, and obtain preprocessed data;

[0091] The pre-processing module 10 is also used for:

[0092] Clean the policy text to remove irrelevant characters, extra spaces and formatting marks in the policy text;

[0093] Convert policy texts into a uniform, standardized coding format;

[0094] Break down the policy text into words or phrases.

[0095] A construction module 20 is used to construct an adaptive model, input the preprocessed data into the adaptive model for training to obtain an optimal adaptive model, wherein the adaptive model includes an input layer, a multi-level division layer, an embedding representation layer, a topic modeling layer and an output layer connected in sequence;

[0096] The preprocessed data is received through an input layer, the preprocessed data is partitioned through a multi-level partitioning layer to obtain partitioned data, the partitioned data is mapped to a low-dimensional vector space through an embedding representation layer to obtain an embedding vector of the partitioned data, the embedded vector is subject-classified through a topic modeling layer, and the embedded vector after subject classification is structured interpreted through an output layer to output structured data of policy text corresponding to a policy relationship;

[0097] The step of dividing the preprocessed data by a multi-level division layer to obtain the divided data includes:

[0098] Dividing and processing the pre-processed data based on policy dimensions and policy labels;

[0099] Among them, the policy dimensions include economy, politics, social welfare, technological innovation and environment;

[0100] The policy labels under the economic dimension include performance orientation, economic institutions, economic development, attracting FDI, enterprise development, industrial planning, policy tools, and coordination; the policy labels under the political dimension include political institutions, government efficiency, public security and social stability, united front, and legal development; the policy labels under the social welfare dimension include employment, people's livelihood and medical care, urbanization, agriculture, culture, education, and tourism; the policy labels under the technological innovation dimension include technology and innovation; and the policy labels under the environmental dimension include environment.

[0101] Mapping the partitioned data to a low-dimensional vector space through an embedding representation layer to obtain an embedded vector of the partitioned data includes:

[0102] According to the semantic relationship and occurrence frequency of words or phrases in the divided data, a corresponding embedding vector is generated for each divided data.

[0103] The topic classification of the embedded vectors through the topic modeling layer includes:

[0104] Based on the distribution characteristics and semantic similarity of the embedding vectors, each embedding vector is assigned to the corresponding topic category.

[0105] Dividing the preprocessed data into a training set, a validation set, and a test set;

[0106] Training the adaptive model based on the training set;

[0107] Performing a performance evaluation on the trained adaptive model based on the validation set to obtain an adaptive model that meets the performance requirements;

[0108] The prediction result of the adaptive model that meets the performance condition is evaluated based on the test set to obtain an evaluation index corresponding to the adaptive model.

[0109] The parameters of the adaptive model are adjusted and updated through an adaptive learning algorithm so that the adaptive model is continuously optimized to adapt to new policy texts.

[0110] The prediction module 30 is used to input the policy text to be analyzed into the optimal adaptive model to obtain structured data of the policy relationship corresponding to the policy text to be analyzed.

[0111] Figure 3 A schematic diagram of the physical structure of an electronic device provided in an embodiment of the present invention, such as Figure 3 As shown, the electronic device 40 includes: a processor 401 (processor), a memory 402 (memory) and a bus 403;

[0112] The processor 401 and the memory 402 communicate with each other via the bus 403;

[0113] The processor 401 is used to call the program instructions in the memory 402 to execute the methods provided by the above-mentioned method embodiments, for example, including: obtaining a certain number of policy texts, preprocessing the policy texts to obtain preprocessed data; constructing an adaptive model, inputting the preprocessed data into the adaptive model for training to obtain an optimal adaptive model, wherein the adaptive model includes an input layer, a multi-level division layer, an embedding representation layer, a topic modeling layer and an output layer connected in sequence; receiving the preprocessed data through the input layer, dividing the preprocessed data through the multi-level division layer to obtain divided data, mapping the divided data to a low-dimensional vector space through the embedding representation layer to obtain an embedded vector of the divided data, subject-classifying the embedded vector through the topic modeling layer, and structuredly interpreting the embedded vector after subject classification through the output layer to output structured data of policy text corresponding to policy relationship; inputting the policy text to be analyzed into the optimal adaptive model to obtain structured data of policy relationship corresponding to the policy text to be analyzed.

[0114] The present embodiment provides a non-transitory computer-readable storage medium, which stores computer instructions. The computer instructions enable a computer to execute the methods provided by the above-mentioned method embodiments, for example, including: obtaining a certain number of policy texts, preprocessing the policy texts to obtain preprocessed data; constructing an adaptive model, inputting the preprocessed data into the adaptive model for training to obtain an optimal adaptive model, wherein the adaptive model includes an input layer, a multi-level division layer, an embedding representation layer, a topic modeling layer and an output layer connected in sequence; receiving the preprocessed data through the input layer, dividing the preprocessed data through the multi-level division layer to obtain divided data, mapping the divided data to a low-dimensional vector space through the embedding representation layer to obtain an embedded vector of the divided data, subject-classifying the embedded vector through the topic modeling layer, and structuredly interpreting the embedded vector after subject classification through the output layer to output structured data of policy text corresponding to policy relationship; inputting the policy text to be analyzed into the optimal adaptive model to obtain structured data of policy relationship corresponding to the policy text to be analyzed.

[0115] A person skilled in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk, etc., various storage media that can store program codes.

[0116] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0117] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or some parts of the embodiment.

[0118] Although the present invention has been described in detail above by general description and specific embodiments, it is obvious to those skilled in the art that some modifications or improvements can be made to the present invention. Therefore, these modifications or improvements made without departing from the spirit of the present invention all belong to the scope of protection claimed by the present invention.

Claims

1. A method for text analysis, characterized in that: The text analysis method specifically includes: Acquire a certain number of policy texts, preprocess the policy texts, and obtain preprocessed data; Constructing an adaptive model, inputting the preprocessed data into the adaptive model for training to obtain an optimal adaptive model, wherein the adaptive model includes an input layer, a multi-level division layer, an embedding representation layer, a topic modeling layer and an output layer connected in sequence; The preprocessed data is received through an input layer, the preprocessed data is partitioned through a multi-level partitioning layer to obtain partitioned data, the partitioned data is mapped to a low-dimensional vector space through an embedding representation layer to obtain an embedding vector of the partitioned data, the embedded vector is subject-classified through a topic modeling layer, and the embedded vector after subject classification is structured interpreted through an output layer to output structured data of policy text corresponding to a policy relationship; The policy text to be analyzed is input into the optimal adaptive model to obtain structured data of the policy relationship corresponding to the policy text to be analyzed.

2. The method for text analysis according to claim 1, characterized in that: The step of obtaining a certain number of policy texts and preprocessing the policy texts to obtain preprocessed data includes: Clean the policy text to remove irrelevant characters, extra spaces and formatting marks in the policy text; Convert policy texts into a uniform, standardized coding format; Break down the policy text into words or phrases.

3. The method for text analysis according to claim 1, characterized in that: The step of dividing the preprocessed data by a multi-level division layer to obtain the divided data includes: Dividing and processing the pre-processed data based on policy dimensions and policy labels; Among them, the policy dimensions include economy, politics, social welfare, technological innovation and environment; The policy labels under the economic dimension include performance orientation, economic institutions, economic development, attracting FDI, enterprise development, industrial planning, policy tools, and coordination; the policy labels under the political dimension include political institutions, government efficiency, public security and social stability, united front, and legal development; the policy labels under the social welfare dimension include employment, people's livelihood and medical care, urbanization, agriculture, culture, education, and tourism; the policy labels under the technological innovation dimension include technology and innovation; and the policy labels under the environmental dimension include environment.

4. The method for text analysis according to claim 1, characterized in that: Mapping the partitioned data to a low-dimensional vector space through an embedding representation layer to obtain an embedded vector of the partitioned data includes: According to the semantic relationship and occurrence frequency of words or phrases in the divided data, a corresponding embedding vector is generated for each divided data.

5. The method for text analysis according to claim 1, characterized in that: The topic classification of the embedded vectors through the topic modeling layer includes: Based on the distribution characteristics and semantic similarity of the embedding vectors, each embedding vector is assigned to the corresponding topic category.

6. The method for text analysis according to claim 1, characterized in that: The text analysis method further includes: Dividing the preprocessed data into a training set, a validation set, and a test set; Training the adaptive model based on the training set; Performing a performance evaluation on the trained adaptive model based on the validation set to obtain an adaptive model that meets the performance requirements; The prediction result of the adaptive model that meets the performance condition is evaluated based on the test set to obtain an evaluation index corresponding to the adaptive model.

7. The method for text analysis according to claim 1, characterized in that: The text analysis method further includes: The parameters of the adaptive model are adjusted and updated through an adaptive learning algorithm so that the adaptive model is continuously optimized to adapt to new policy texts.

8. A text analysis system, characterized in that: include: A preprocessing module, used for acquiring a certain number of policy texts, preprocessing the policy texts, and obtaining preprocessed data; A construction module, used for constructing an adaptive model, inputting the preprocessed data into the adaptive model for training to obtain an optimal adaptive model, wherein the adaptive model includes an input layer, a multi-level division layer, an embedding representation layer, a topic modeling layer and an output layer connected in sequence; The preprocessed data is received through an input layer, the preprocessed data is partitioned through a multi-level partitioning layer to obtain partitioned data, the partitioned data is mapped to a low-dimensional vector space through an embedding representation layer to obtain an embedding vector of the partitioned data, the embedded vector is subject-classified through a topic modeling layer, and the embedded vector after subject classification is structured interpreted through an output layer to output structured data of policy text corresponding to a policy relationship; The prediction module is used to input the policy text to be analyzed into the optimal adaptive model to obtain structured data of the policy relationship corresponding to the policy text to be analyzed.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A non-transitory computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images