Text Analysis Methods and Systems

Abstract

This invention discloses a method and system for text analysis, comprising: acquiring a certain number of policy texts; preprocessing the policy texts to obtain preprocessed data; inputting the preprocessed data into an adaptive model for training to obtain an optimal adaptive model; receiving the preprocessed data through an input layer; partitioning the preprocessed data through a multi-level partitioning layer to obtain partitioned data; mapping the partitioned data to a low-dimensional vector space through an embedding representation layer to obtain embedding vectors of the partitioned data; classifying the embedding vectors into topics through a topic modeling layer; and outputting structured data of the policy relationships corresponding to the policy texts through a structured interpretation of the topic-classified embedding vectors through an output layer; and inputting the policy texts to be analyzed into the optimal adaptive model to obtain structured data of the policy relationships corresponding to the policy texts to be analyzed. This addresses the problem in existing technologies that cannot provide in-depth, comprehensive, and accurate analysis of industrial policies.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to a method and system for text analysis. Background Technology

[0002] With the acceleration of global digital transformation and the deepening penetration of information technology into various industries, emerging industries are booming. The dynamics of technological development, changes in market demand, and policy support in each industry play a crucial role in predicting industrial development paths and formulating supporting policies. Especially in an increasingly competitive global environment, reasonable policy guidance and effective market support mechanisms are key elements for enhancing industrial innovation capabilities and international competitiveness. This makes building a digital platform to support industrial development strategies an urgent priority, with the establishment and improvement of an industrial policy database being of paramount importance.

[0003] Currently, industrial policy data mainly comes from various channels, including policy documents, industry reports, and government planning documents. However, current industrial policy analysis methods have many limitations, which seriously affect the comprehensive understanding and effective utilization of policy implications.

[0004] In existing industrial policy analysis practices, traditional methods are mainly based on basic text retrieval techniques, such as keyword matching and simple text classification.

[0005] These methods have significant shortcomings when processing policy documents. First, in cross-document analysis, existing technologies mostly focus on the content of individual policy documents, lacking the ability to systematically compare the inherent connections between different policy documents. This makes it impossible to trace and analyze the evolution of policies and to grasp the continuity of policy development from a macro perspective. Second, from an analytical perspective, traditional methods are limited to similarity analysis at the lexical level. This approach ignores the more complex relationships and potential strategic intentions contained in policy content, making the analysis results too superficial and unable to delve into the deeper meaning of policies. Third, existing technologies lack accuracy in determining policy correlations. Especially in complex policy environments, due to the difficulty in capturing the deep connections between different policy documents, subtle policy differences are easily overlooked, thus affecting the accurate assessment of the overall policy impact. Finally, when faced with large-scale literature data, existing analytical methods are inefficient and struggle to process massive amounts of policy documents and text data, failing to achieve accurate and efficient analysis, thus limiting the effective use of large-scale industrial policy data.

[0006] In conclusion, traditional industrial policy analysis methods can no longer meet the current needs of industrial development for policy analysis. There is an urgent need for a new and more effective policy analysis method to overcome the limitations of existing technologies and achieve a more in-depth, comprehensive and accurate analysis of industrial policies. Summary of the Invention

[0007] The purpose of this invention is to provide a text analysis method and system to solve the problem that existing technologies cannot provide in-depth, comprehensive and accurate analysis of industrial policies.

[0008] To achieve the above objectives, embodiments of the present invention provide a text analysis method, the method specifically including:

[0009] Obtain a certain number of policy texts, preprocess the policy texts, and obtain preprocessed data;

[0010] An adaptive model is constructed, and the preprocessed data is input into the adaptive model for training to obtain the optimal adaptive model. The adaptive model includes an input layer, a multi-level partitioning layer, an embedding representation layer, a topic modeling layer, and an output layer that are connected in sequence.

[0011] The preprocessed data is received through the input layer, and the preprocessed data is divided through a multi-level partitioning layer to obtain partitioned data. The partitioned data is mapped to a low-dimensional vector space through the embedding representation layer to obtain the embedding vectors of the partitioned data. The embedding vectors are classified into topics through the topic modeling layer. The embedded vectors after topic classification are structured and interpreted through the output layer to output structured data of policy relationships corresponding to policy texts.

[0012] The policy text to be analyzed is input into the optimal adaptive model to obtain structured data of the policy relationships corresponding to the policy text to be analyzed.

[0013] Based on the above technical solution, the present invention can be further improved as follows:

[0014] Furthermore, the step of acquiring a certain number of policy texts and preprocessing the policy texts to obtain preprocessed data includes:

[0015] Clean the policy text to remove irrelevant characters, extra spaces, and formatting marks.

[0016] Convert policy texts into a standardized encoding format;

[0017] Break down policy texts into words or phrases.

[0018] Furthermore, the step of dividing the preprocessed data through multi-level partitioning to obtain partitioned data includes:

[0019] The preprocessed data is divided and processed based on policy dimensions and policy tags;

[0020] The policy dimension includes economic, political, social welfare, technological innovation, and environmental aspects.

[0021] The policy tags under the economic dimension include performance orientation, economic institutions, economic development, attracting FDI, enterprise development, industrial planning, policy tools, and coordination. The policy tags under the political dimension include political institutions, government efficiency, public security and social stability, united front, and legal development. The policy tags under the social welfare dimension include employment, people's livelihood and medical care, urbanization, agriculture, culture, education, and tourism. The policy tags under the technological innovation dimension include technology and innovation. The policy tags under the environmental dimension include the environment.

[0022] Further, the step of mapping the partitioned data to a low-dimensional vector space through an embedding representation layer to obtain the embedding vector of the partitioned data includes:

[0023] Based on the semantic relationships and frequency of occurrence of words or phrases in the segmented data, a corresponding embedding vector is generated for each segmented data.

[0024] Furthermore, the step of classifying the embedded vectors through the topic modeling layer includes:

[0025] Based on the distribution characteristics and semantic similarity of the embedding vectors, each embedding vector is assigned to a corresponding topic category.

[0026] Furthermore, the text analysis method further includes:

[0027] The preprocessed data is divided into a training set, a validation set, and a test set;

[0028] The adaptive model is trained based on the training set;

[0029] The trained adaptive model is evaluated based on the validation set to obtain an adaptive model that meets the performance requirements.

[0030] The evaluation index corresponding to the adaptive model is obtained by evaluating the prediction results of the adaptive model that meet the performance conditions based on the test set.

[0031] Furthermore, the text analysis method further includes:

[0032] The parameters of the adaptive model are adjusted and updated by an adaptive learning algorithm, so that the adaptive model is continuously optimized to adapt to new policy texts.

[0033] A text analysis system, comprising:

[0034] The preprocessing module is used to acquire a certain number of policy texts, preprocess the policy texts, and obtain preprocessed data.

[0035] A construction module is used to build an adaptive model. The preprocessed data is input into the adaptive model for training to obtain the optimal adaptive model. The adaptive model includes an input layer, a multi-level partitioning layer, an embedding representation layer, a topic modeling layer, and an output layer connected in sequence.

[0036] The preprocessed data is received through the input layer, and the preprocessed data is divided through a multi-level partitioning layer to obtain partitioned data. The partitioned data is mapped to a low-dimensional vector space through the embedding representation layer to obtain the embedding vectors of the partitioned data. The embedding vectors are classified into topics through the topic modeling layer. The embedded vectors after topic classification are structured and interpreted through the output layer to output structured data of policy relationships corresponding to policy texts.

[0037] The prediction module is used to input the policy text to be analyzed into the optimal adaptive model to obtain structured data of the policy relationships corresponding to the policy text to be analyzed.

[0038] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the steps of the method described herein.

[0039] A non-transitory computer-readable storage medium having a computer program stored thereon, the computer program implementing the steps of the method when executed by a processor.

[0040] The embodiments of the present invention have the following advantages:

[0041] The text analysis method of this invention, through the collaborative work of a multi-level segmentation layer, an embedding representation layer, and a topic modeling layer, can deeply mine the semantic information of policy texts. The multi-level segmentation layer can analyze policy texts from different perspectives, the embedding representation layer maps the segmented data to a low-dimensional vector space to obtain embedding vectors, effectively capturing semantic relationships in the text, and the topic modeling layer further classifies these embedding vectors into topics. This avoids the limitations of traditional methods that rely solely on surface keyword matching, enabling a more accurate understanding of policy content and improving the accuracy of policy relationship analysis.

[0042] Preprocessing policy texts before inputting them into an adaptive model cleans and standardizes the data, allowing the model to better focus on key information. During processing, the adaptive model's layers work together to comprehensively consider various features of the policy text, avoiding the oversight of underlying logical relationships that traditional analysis methods often overlook. This reduces information omissions and more accurately presents the relationships between policies.

[0043] The adaptive model's structure enables it to analyze policy texts from multiple dimensions. Unlike traditional methods that focus only on the content of a single policy document or simple vocabulary-level similarity analysis, this method can compare and analyze policies at multiple levels and dimensions, thus constructing policy relationships more systematically. Whether it's the similarity or differences in policy content, or the connections in the policy evolution process, a comprehensive consideration can be given, achieving systematic comparisons across documents.

[0044] An adaptive model is constructed and trained on a large amount of preprocessed data to obtain the optimal model. This model is designed to adapt to large-scale policy text data. Compared with the inefficiency of traditional analysis methods when processing massive policy documents and text data, the model of this method can efficiently output structured data of the corresponding policy relationships for the policy text to be analyzed after training, thus improving the efficiency of processing large-scale data.

[0045] The output layer outputs structured data of policy relationships corresponding to policy texts. This structured format facilitates storage, querying, and further analysis, providing convenience for both rapid retrieval of large numbers of policy documents and the construction and analysis of complex policy relationship networks, thereby further improving processing efficiency.

[0046] In-depth processing of policy texts allows for the accurate identification of similarities and differences in policy content, and enables a thorough exploration of policy evolution. This allows policymakers to clearly understand the development trajectory of policies, discover trends and patterns in policy changes, and thus provide a strong basis for formulating more scientific, rational, forward-looking, and consistent policies, contributing to enhancing industrial innovation capabilities and international competitiveness. Attached Figure Description

[0047] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings in the following description are merely exemplary, and those skilled in the art can derive other embodiments based on the provided drawings without creative effort.

[0048] The structures, proportions, sizes, etc. illustrated in this specification are only for the purpose of assisting those skilled in the art in understanding and reading the content disclosed herein, and are not intended to limit the conditions under which the present invention can be implemented. Therefore, they have no substantial technical significance. Any modifications to the structure, changes in the proportions, or adjustments to the size, without affecting the effects and objectives that the present invention can produce, should still fall within the scope of the technical content disclosed in the present invention.

[0049] Figure 1 This is a flowchart of the text analysis method of the present invention;

[0050] Figure 2 This is an architecture diagram of the text analysis system of the present invention;

[0051] Figure 3 This is a schematic diagram of the physical structure of the electronic device provided by the present invention;

[0052] Figure 4 This is a schematic diagram of the five policy dimensions and 20 policy labels of this invention.

[0053] The attached figures are labeled as follows:

[0054] The system includes a preprocessing module 10, a building module 20, a prediction module 30, an electronic device 40, a processor 401, a memory 402, and a bus 403. Detailed Implementation

[0055] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0056] Figure 1 This is a flowchart illustrating an embodiment of the text analysis method of the present invention, as shown below. Figure 1 As shown, an embodiment of the present invention provides a text analysis method comprising the following steps:

[0057] S101: Obtain a certain number of policy texts, preprocess the policy texts, and obtain preprocessed data;

[0058] Specifically: perform a cleaning operation on the policy text to remove irrelevant characters, extra spaces, and formatting marks;

[0059] Convert policy texts into a standardized encoding format;

[0060] Break down policy texts into words or phrases.

[0061] Preprocessing policy texts before inputting them into an adaptive model cleans and standardizes the data, allowing the model to better focus on key information. During processing, the adaptive model's layers work together to comprehensively consider various features of the policy text, avoiding the oversight of underlying logical relationships that traditional analysis methods often overlook. This reduces information omissions and more accurately presents the relationships between policies.

[0062] S102, Construct an adaptive model by inputting the preprocessed data into the adaptive model for training to obtain the optimal adaptive model;

[0063] Specifically, the adaptive model consists of an input layer, a multi-level partitioning layer, an embedded representation layer, a topic modeling layer, and an output layer connected in sequence.

[0064] The policy texts were divided into training, validation, and test sets.

[0065] Train an adaptive model based on the training set;

[0066] The performance of the trained adaptive model is evaluated based on the validation set to obtain an adaptive model that meets the performance conditions.

[0067] The prediction results of the adaptive model that meets the performance conditions are evaluated based on the test set, and the evaluation index corresponding to the adaptive model is obtained.

[0068] The trained adaptive model is evaluated on the validation set to obtain the adaptive model that meets the performance conditions. The similarity calculation results of the adaptive models that meet the performance conditions are evaluated on the test set to obtain the evaluation index corresponding to the adaptive model. The performance evaluation of the adaptive model yields a percentage score (i.e., the highest score is 100 points and the lowest is 0 points). Adaptive models with scores greater than a set value are determined based on these percentage scores; for example, adaptive models with scores greater than 90 points are considered to meet the performance conditions.

[0069] The performance conditions of the adaptive model are evaluated by calculating the evaluation index, and the evaluation value corresponding to each evaluation index is calculated. The evaluation value is used to represent the capability value of the adaptive model on the evaluation index.

[0070] In constructing the model, this invention introduces a multilayer embedded clustering model, which independently projects text terms into 20 policy label vector spaces based on the high-order semantic affinity of the text. Due to the polysemy of terms and high-dimensional feature mapping, some terms may have overlapping semantic embeddings, thus being assigned to multiple policy dimensions. For focal terms where encoding discrepancies arise, an adaptive iterative labeling model is used. After four rounds of nested iterations, the encoding confidence reaches 80%.

[0071] Based on the constructed policy dimension system, each document is encoded into a 20-dimensional policy vector space, where the components of each dimension serve as potential proxy variables for government resource allocation on a specific policy dimension. Through Hierarchical Bayesian inference, it is assumed that local governments exhibit a conjugate pattern of "policy density-resource allocation" in resource distribution, meaning that high-priority policy dimensions are represented frequently, while low-priority dimensions show relatively sparse feature distribution. The results show that this adaptive model successfully captures the complex differences in local government resource allocation across dimensions and time series.

[0072] An adaptive model is constructed and trained on a large amount of preprocessed data to obtain the optimal model. This model is designed to adapt to large-scale policy text data. Compared with the inefficiency of traditional analysis methods when processing massive policy documents and text data, the model of this method can efficiently output structured data of the corresponding policy relationships for the policy text to be analyzed after training, thus improving the efficiency of processing large-scale data.

[0073] S103: The preprocessed data is received through the input layer, and the preprocessed data is divided through the multi-level partitioning layer to obtain partitioned data. The partitioned data is mapped to a low-dimensional vector space through the embedding representation layer to obtain the embedding vectors of the partitioned data. The embedding vectors are classified into topics through the topic modeling layer. The embedded vectors after topic classification are structured and interpreted through the output layer to output the structured data of policy relationships corresponding to the policy text.

[0074] Specifically, the step of dividing the preprocessed data through multi-level partitioning to obtain partitioned data includes:

[0075] The preprocessed data is divided and processed based on policy dimensions and policy tags;

[0076] like Figure 4 As shown, the policy dimensions include economic, political, social welfare, technological innovation, and environmental aspects;

[0077] The policy tags under the economic dimension include performance orientation, economic institutions, economic development, attracting FDI, enterprise development, industrial planning, policy tools, and coordination. The policy tags under the political dimension include political institutions, government efficiency, public security and social stability, united front, and legal development. The policy tags under the social welfare dimension include employment, people's livelihood and medical care, urbanization, agriculture, culture, education, and tourism. The policy tags under the technological innovation dimension include technology and innovation. The policy tags under the environmental dimension include the environment.

[0078] The step of mapping the partitioned data to a low-dimensional vector space through an embedding representation layer to obtain the embedding vector of the partitioned data includes:

[0079] Based on the semantic relationships and frequency of occurrence of words or phrases in the segmented data, a corresponding embedding vector is generated for each segmented data.

[0080] The topic classification of embedded vectors through a topic modeling layer includes:

[0081] Based on the distribution characteristics and semantic similarity of the embedding vectors, each embedding vector is assigned to a corresponding topic category.

[0082] By leveraging the collaborative efforts of a multi-layered partitioning layer, an embedding representation layer, and a topic modeling layer, the semantic information of policy texts can be deeply mined. The multi-layered partitioning layer analyzes policy texts from different perspectives; the embedding representation layer maps the partitioned data to a low-dimensional vector space to obtain embedding vectors, effectively capturing semantic relationships within the text; and the topic modeling layer further classifies these embedding vectors by topic, thus avoiding the limitations of traditional methods that rely solely on surface keyword matching, leading to a more accurate understanding of policy content and improved accuracy in policy relationship analysis. The output layer outputs structured data of policy relationships corresponding to the policy text. This structured format facilitates storage, retrieval, and further analysis, providing convenience for both rapid retrieval of large amounts of policy documents and the construction and analysis of complex policy relationship networks, further enhancing processing efficiency.

[0083] S104, Input the policy text to be analyzed into the optimal adaptive model to obtain structured data of the policy relationships corresponding to the policy text to be analyzed;

[0084] Specifically, the adaptive model's structure enables it to analyze policy texts from multiple dimensions. Unlike traditional methods that focus only on the content of a single policy document or simple vocabulary-level similarity analysis, this method can compare and analyze policies at multiple levels and dimensions, thus constructing policy relationships more systematically. Whether it's the similarity or differences in policy content, or the connections in the policy evolution process, a comprehensive consideration can be given, achieving systematic comparisons across documents.

[0085] In-depth processing of policy texts allows for the accurate identification of similarities and differences in policy content, and enables a thorough exploration of policy evolution. This allows policymakers to clearly understand the development trajectory of policies, discover trends and patterns in policy changes, and thus provide a strong basis for formulating more scientific, rational, forward-looking, and consistent policies, contributing to enhancing industrial innovation capabilities and international competitiveness.

[0086] The text analysis method further includes:

[0087] The parameters of the adaptive model are adjusted and updated by an adaptive learning algorithm, so that the adaptive model is continuously optimized to adapt to new policy texts.

[0088] This invention constructs a 20-dimensional policy distribution theme for each published policy report using an adaptive model, thereby capturing its resource allocation characteristics across various policy dimensions. The heterogeneity of resource distribution across key policy dimensions may be driven by multiple hierarchical variables, including city characteristics, time-series effects, and individual leader effects. To isolate the potential influence of political leaders on resource allocation patterns, a combination of Factorization Machine and Latent Semantic Decomposition is employed. Through hierarchical Bayesian inference, the individual independence within the multi-factor interactions is deconstructed, ultimately achieving a multi-dimensional composite analysis and deep modeling of policy distribution patterns.

[0089] Figure 2 This is a system architecture diagram of a text analysis embodiment of the present invention; as shown below. Figure 2 As shown in the figure, an embodiment of the present invention provides a text analysis system, which includes the following steps:

[0090] Preprocessing module 10 is used to acquire a certain number of policy texts, preprocess the policy texts, and obtain preprocessed data;

[0091] The preprocessing module 10 is also used for:

[0092] Clean the policy text to remove irrelevant characters, extra spaces, and formatting marks.

[0093] Convert policy texts into a standardized encoding format;

[0094] Break down policy texts into words or phrases.

[0095] The construction module 20 is used to construct an adaptive model, and input the preprocessed data into the adaptive model for training to obtain the optimal adaptive model. The adaptive model includes an input layer, a multi-level partitioning layer, an embedding representation layer, a topic modeling layer and an output layer connected in sequence.

[0096] The preprocessed data is received through the input layer, and the preprocessed data is divided through a multi-level partitioning layer to obtain partitioned data. The partitioned data is mapped to a low-dimensional vector space through the embedding representation layer to obtain the embedding vectors of the partitioned data. The embedding vectors are classified into topics through the topic modeling layer. The embedded vectors after topic classification are structured and interpreted through the output layer to output structured data of policy relationships corresponding to policy texts.

[0097] The process of dividing the preprocessed data through a multi-level partitioning layer to obtain partitioned data includes:

[0098] The preprocessed data is divided and processed based on policy dimensions and policy tags;

[0099] The policy dimension includes economic, political, social welfare, technological innovation, and environmental aspects.

[0100] The policy tags under the economic dimension include performance orientation, economic institutions, economic development, attracting FDI, enterprise development, industrial planning, policy tools, and coordination. The policy tags under the political dimension include political institutions, government efficiency, public security and social stability, united front, and legal development. The policy tags under the social welfare dimension include employment, people's livelihood and medical care, urbanization, agriculture, culture, education, and tourism. The policy tags under the technological innovation dimension include technology and innovation. The policy tags under the environmental dimension include the environment.

[0101] The step of mapping the partitioned data to a low-dimensional vector space through an embedding representation layer to obtain the embedding vector of the partitioned data includes:

[0102] Based on the semantic relationships and frequency of occurrence of words or phrases in the segmented data, a corresponding embedding vector is generated for each segmented data.

[0103] The topic classification of embedded vectors through a topic modeling layer includes:

[0104] Based on the distribution characteristics and semantic similarity of the embedding vectors, each embedding vector is assigned to a corresponding topic category.

[0105] The preprocessed data is divided into a training set, a validation set, and a test set;

[0106] The adaptive model is trained based on the training set;

[0107] The trained adaptive model is evaluated based on the validation set to obtain an adaptive model that meets the performance requirements.

[0108] The evaluation index corresponding to the adaptive model is obtained by evaluating the prediction results of the adaptive model that meet the performance conditions based on the test set.

[0109] The parameters of the adaptive model are adjusted and updated by an adaptive learning algorithm, so that the adaptive model is continuously optimized to adapt to new policy texts.

[0110] The prediction module 30 is used to input the policy text to be analyzed into the optimal adaptive model to obtain structured data of the policy relationship corresponding to the policy text to be analyzed.

[0111] Figure 3 This is a schematic diagram of the physical structure of an electronic device provided in an embodiment of the present invention, such as... Figure 3 As shown, the electronic device 40 includes: a processor 401, a memory 402, and a bus 403;

[0112] The processor 401 and the memory 402 communicate with each other via the bus 403.

[0113] The processor 401 is used to call program instructions in the memory 402 to execute the methods provided in the above-described method embodiments, such as: acquiring a certain number of policy texts, preprocessing the policy texts to obtain preprocessed data; constructing an adaptive model, inputting the preprocessed data into the adaptive model for training to obtain an optimal adaptive model, wherein the adaptive model includes an input layer, a multi-level partitioning layer, an embedding representation layer, a topic modeling layer, and an output layer connected in sequence; receiving the preprocessed data through the input layer, partitioning the preprocessed data through the multi-level partitioning layer to obtain partitioned data, mapping the partitioned data to a low-dimensional vector space through the embedding representation layer to obtain the embedding vectors of the partitioned data, performing topic classification on the embedding vectors through the topic modeling layer, and performing structured interpretation on the topic-classified embedding vectors through the output layer to output structured data of policy relationships corresponding to the policy texts; inputting the policy texts to be analyzed into the optimal adaptive model to obtain structured data of policy relationships corresponding to the policy texts to be analyzed.

[0114] This embodiment provides a non-transitory computer-readable storage medium that stores computer instructions. These instructions cause a computer to execute the methods provided in the above-described method embodiments. For example, the methods include: acquiring a certain number of policy texts; preprocessing the policy texts to obtain preprocessed data; constructing an adaptive model; inputting the preprocessed data into the adaptive model for training to obtain an optimal adaptive model. The adaptive model includes a sequentially connected input layer, a multi-level partitioning layer, an embedding representation layer, a topic modeling layer, and an output layer. The input layer receives the preprocessed data; the multi-level partitioning layer partitions the preprocessed data to obtain partitioned data; the embedding representation layer maps the partitioned data to a low-dimensional vector space to obtain embedding vectors; the topic modeling layer performs topic classification on the embedding vectors; and the output layer performs structured interpretation on the topic-classified embedding vectors to output structured data of the policy relationships corresponding to the policy texts. Finally, the policy text to be analyzed is input into the optimal adaptive model to obtain structured data of the policy relationships corresponding to the policy text to be analyzed.

[0115] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various storage media capable of storing program code, such as ROM, RAM, magnetic disk, or optical disk.

[0116] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0117] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of each embodiment or some parts of the embodiments.

[0118] Although the present invention has been described in detail above with general descriptions and specific embodiments, modifications or improvements can be made to it, which will be obvious to those skilled in the art. Therefore, all such modifications or improvements made without departing from the spirit of the present invention fall within the scope of protection claimed by the present invention.

Claims

1. A text analysis method, characterized in that, The text analysis method specifically includes: Obtain a certain number of policy texts, preprocess the policy texts, and obtain preprocessed data; An adaptive model is constructed, and the preprocessed data is input into the adaptive model for training to obtain the optimal adaptive model. The adaptive model includes an input layer, a multi-level partitioning layer, an embedding representation layer, a topic modeling layer, and an output layer that are connected in sequence. In the process of building the model, a multi-level embedding clustering model was introduced. Based on the high-order semantic affinity of the text, the text terms were independently projected into 20 policy label vector spaces. For the focal terms that caused the coding to diverge, the adaptive iterative label model was used for nested iteration. The preprocessed data is received through the input layer, and the preprocessed data is divided through a multi-level partitioning layer to obtain partitioned data. The partitioned data is mapped to a low-dimensional vector space through the embedding representation layer to obtain the embedding vectors of the partitioned data. The embedding vectors are classified into topics through the topic modeling layer. The embedded vectors after topic classification are structured and interpreted through the output layer to output structured data of policy relationships corresponding to policy texts. The process of dividing the preprocessed data through a multi-level partitioning layer to obtain partitioned data includes: The preprocessed data is divided and processed based on policy dimensions and policy tags; The policy dimension includes economic, political, social welfare, technological innovation, and environmental aspects. The policy tags under the economic dimension include performance orientation, economic institutions, economic development, attracting FDI, enterprise development, industrial planning, policy tools, and coordination. The policy tags under the political dimension include political institutions, government efficiency, public security and social stability, united front, and legal development. The policy tags under the social welfare dimension include employment, people's livelihood and medical care, urbanization, agriculture, culture, education, and tourism. The policy tags under the technological innovation dimension include technology and innovation. The policy tags under the environmental dimension include the environment. The step of mapping the partitioned data to a low-dimensional vector space through an embedding representation layer to obtain the embedding vector of the partitioned data includes: Based on the semantic relationships and frequency of occurrence of words or phrases in the segmented data, generate corresponding embedding vectors for each segmented data; The method of classifying embedded vectors through a topic modeling layer includes: Based on the distribution characteristics and semantic similarity of the embedding vectors, each embedding vector is assigned to a corresponding topic category; The policy text to be analyzed is input into the optimal adaptive model to obtain structured data of the policy relationships corresponding to the policy text to be analyzed.

2. The text analysis method according to claim 1, characterized in that, The process of acquiring a certain number of policy texts and preprocessing the policy texts to obtain preprocessed data includes: Clean the policy text to remove irrelevant characters, extra spaces, and formatting marks. Convert policy texts into a standardized encoding format; Break down policy texts into words or phrases.

3. The text analysis method according to claim 1, characterized in that, The text analysis method further includes: The preprocessed data is divided into a training set, a validation set, and a test set; The adaptive model is trained based on the training set; The trained adaptive model is evaluated based on the validation set to obtain an adaptive model that meets the performance requirements. The evaluation index corresponding to the adaptive model is obtained by evaluating the prediction results of the adaptive model that meet the performance conditions based on the test set.

4. The text analysis method according to claim 1, characterized in that, The text analysis method further includes: The parameters of the adaptive model are adjusted and updated by an adaptive learning algorithm, so that the adaptive model is continuously optimized to adapt to new policy texts.

5. A text analysis system, characterized in that, include: The preprocessing module is used to acquire a certain number of policy texts, preprocess the policy texts, and obtain preprocessed data. A construction module is used to build an adaptive model. The preprocessed data is input into the adaptive model for training to obtain the optimal adaptive model. The adaptive model includes an input layer, a multi-level partitioning layer, an embedding representation layer, a topic modeling layer, and an output layer connected in sequence. In the process of building the model, a multi-level embedding clustering model was introduced. Based on the high-order semantic affinity of the text, the text terms were independently projected into 20 policy label vector spaces. For the focal terms that caused the coding to diverge, the adaptive iterative label model was used for nested iteration. The preprocessed data is received through the input layer, and the preprocessed data is divided through a multi-level partitioning layer to obtain partitioned data. The partitioned data is mapped to a low-dimensional vector space through the embedding representation layer to obtain the embedding vectors of the partitioned data. The embedding vectors are classified into topics through the topic modeling layer. The embedded vectors after topic classification are structured and interpreted through the output layer to output structured data of policy relationships corresponding to policy texts. The preprocessed data is divided into layers based on policy dimensions and policy tags through a multi-level segmentation process; The policy dimension includes economic, political, social welfare, technological innovation, and environmental aspects. The policy tags under the economic dimension include performance orientation, economic institutions, economic development, attracting FDI, enterprise development, industrial planning, policy tools, and coordination. The policy tags under the political dimension include political institutions, government efficiency, public security and social stability, united front, and legal development. The policy tags under the social welfare dimension include employment, people's livelihood and medical care, urbanization, agriculture, culture, education, and tourism. The policy tags under the technological innovation dimension include technology and innovation. The policy tags under the environmental dimension include the environment. The embedding representation layer generates a corresponding embedding vector for each segment of data based on the semantic relationships and frequency of occurrence of words or phrases in the segmented data. By using a topic modeling layer based on the distribution characteristics and semantic similarity of the embedding vectors, each embedding vector is assigned to a corresponding topic category; The prediction module is used to input the policy text to be analyzed into the optimal adaptive model to obtain structured data of the policy relationships corresponding to the policy text to be analyzed.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 4.

7. A non-transitory computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 4.

Images