Social governance text processing method and device, electronic equipment and storage medium

Abstract

The invention provides a social governance text processing method, which comprises the following steps of: on the basis of a self-defined dictionary, respectively preprocessing each social governance text to obtain livelihood characteristic data corresponding to the social governance text; calling a document theme generation model to analyze the livelihood characteristic data, and generating livelihood analysis information about the social governance text, wherein the livelihood analysis information at least comprises livelihood themes associated with the social governance texts, distribution probabilities of livelihood segmented words on the livelihood themes and distribution probabilities of the social governance texts on the livelihood themes; and according to the livelihood analysis information, constructing a display page convenient to read. The invention further provides a social governance text processing device, electronic equipment and a storage medium.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to a method, device, electronic device and storage medium for processing social governance texts. Background Art

[0002] At present, in order to improve social governance capabilities and resolve social conflicts, relevant departments have opened a variety of channels to obtain public opinion and public sentiment information. Among them, the livelihood service hotline is an important channel for the general public to express their demands. Since the opening of the livelihood service hotline platform, various provinces, cities and counties have accumulated a large amount of livelihood hotline data. This information mainly includes the time, location and content of the reported incident, and contains rich local social hot spot information, providing rich data mining resources for text data analysis and mining for social governance.

[0003] However, complaint text data is usually short text, and the data sparsity problem is relatively serious; the complaint content is generally manually entered by government officials, and there are problems such as non-standard and inconsistent expressions, and there are also noises such as typos; in addition, with the improvement of the level of people's livelihood services, the amount of data has increased. Based on the above, it is more difficult to manually analyze complaint text data, and the hidden problems of a large amount of unstructured data are also difficult to manually mine. There is no technical means in the relevant technology that can convert unstructured data into structured data and convert these text data into reasonable knowledge representation. Summary of the invention

[0004] In order to solve at least one of the above technical problems, the present disclosure provides a method, device, electronic device and storage medium for processing social governance texts.

[0005] According to one aspect of the present disclosure, a method for processing social governance texts is proposed, including: pre-processing each social governance text based on a custom dictionary to obtain people's livelihood feature data corresponding to the social governance text; calling a document topic generation model to analyze the people's livelihood feature data to generate people's livelihood analysis information about the social governance text, wherein the people's livelihood analysis information at least includes people's livelihood themes associated with each social governance text, the distribution probability of people's livelihood segmentation on people's livelihood themes, and the distribution probability of each social governance text on people's livelihood themes; and constructing an easy-to-read display page based on the people's livelihood analysis information.

[0006] In some embodiments, based on the custom dictionary, each social governance text is preprocessed separately to obtain the people's livelihood feature data corresponding to the social governance text, including: cleaning the social governance text, at least removing the missing values, repeated values, noise data and redundant data in the social governance text to obtain a clean text; based on the custom dictionary, the clean text is segmented to construct a corpus formed by multiple people's livelihood keywords; and each of the social governance texts and the corresponding corpus is converted into a matrix form to construct the people's livelihood feature data composed of a text matrix and a word segmentation matrix, wherein the text matrix includes the document number, document sequence number, people's livelihood keyword number and its word frequency, and the word segmentation matrix includes the people's livelihood keyword, the number of the people's livelihood keyword and its word frequency.

[0007] In some embodiments, the clean text is segmented based on the custom dictionary to construct a corpus formed by multiple livelihood keywords, including: based on the geographic information dictionary and the social governance event dictionary in the custom dictionary, the clean text is segmented to obtain multiple livelihood segmentations; according to the stop word removal dictionary in the custom dictionary, stop words are removed from the multiple livelihood segmentations to screen out multiple livelihood keywords used to characterize the livelihood information of the social governance text; and each of the livelihood keywords is integrated to form the corpus constructed by each of the livelihood keywords.

[0008] In some embodiments, constructing an easy-to-read display page based on the people's livelihood analysis information includes: analyzing the people's livelihood analysis information to mine related data of the social governance text, wherein the related data includes at least multiple hot events and their development trends, and the number of people's livelihood topics; and processing the related data in a display form to form an easy-to-read display page.

[0009] In some implementations, before constructing an easy-to-read display page based on the livelihood analysis information, it also includes: establishing a visual interactive clustering system to obtain error correction constraint instructions for the livelihood analysis information to optimize the document topic generation model.

[0010] In some embodiments, before preprocessing each social governance text based on the custom dictionary to obtain the people's livelihood feature data corresponding to the social governance text, it includes: according to the application scenario of the social governance text, constructing a custom dictionary suitable for the application scenario, wherein the custom dictionary at least includes a geographic information dictionary, a social governance event dictionary and a stop word removal dictionary.

[0011] In some embodiments, it also includes: extracting any livelihood keyword from any livelihood theme in the social governance text; performing semantic analysis on the livelihood keyword, and binding multiple livelihood keywords with the semantics to the corresponding same constraint identifier; mapping the multiple livelihood keywords associated with the constraint identifier to the extracted livelihood theme.

[0012] According to another aspect of the present disclosure, a device for processing social governance texts is proposed, including: a preprocessing module, used to preprocess each social governance text based on a custom dictionary, and obtain people's livelihood feature data corresponding to the social governance text; a data analysis module, used to call a document topic generation model to analyze the people's livelihood feature data, and generate people's livelihood analysis information about the social governance text, wherein the people's livelihood analysis information at least includes the people's livelihood themes associated with each of the social governance texts, the distribution probability of the people's livelihood participles on the people's livelihood themes, and the distribution probability of each of the social governance texts on the people's livelihood themes; and a display module, used to construct an easy-to-read display page according to the people's livelihood analysis information, wherein the display page includes a constraint setting area.

[0013] According to another aspect of the present disclosure, an electronic device is proposed, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for processing social governance text as described in any of the above-mentioned embodiments is implemented.

[0014] According to another aspect of the present disclosure, a readable storage medium is proposed, in which execution instructions are stored. When the execution instructions are executed by a processor, they are used to implement the method for processing social governance texts as described in any of the above-mentioned embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings illustrate exemplary embodiments of the present disclosure and together with the description serve to explain the principles of the present disclosure. These drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this specification.

[0016] Figure 1 The present invention is a flowchart of the method for processing social governance texts disclosed herein.

[0017] Figure 2 This is an architectural diagram of the social governance text processing method disclosed in this disclosure.

[0018] Figure 3 This is a schematic diagram of the associated data display page for the public service hotline scenario.

[0019] Figure 4The figure is a schematic diagram of a related data display page for an application scenario of the present disclosure.

[0020] Figure 5 This is a block diagram of a processing device for the social governance text disclosed in the present invention. DETAILED DESCRIPTION

[0021] The present disclosure is further described in detail below in conjunction with the accompanying drawings and implementations. It is understood that the specific implementations described herein are only used to explain the relevant content, rather than to limit the present disclosure. It should also be noted that, for ease of description, only the parts related to the present disclosure are shown in the accompanying drawings.

[0022] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in the present disclosure can be combined with each other. The technical solution of the present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0023] Unless otherwise specified, the exemplary embodiments / embodiments shown will be understood as providing exemplary features of various details of some ways in which the technical concept of the present disclosure can be implemented in practice. Therefore, unless otherwise specified, the features of the various embodiments / embodiments can be combined, separated, interchanged and / or rearranged without departing from the technical concept of the present disclosure.

[0024] The use of cross-hatching and / or shading in the accompanying drawings is generally used to make the boundaries between adjacent components clear. As such, unless otherwise specified, the presence or absence of cross-hatching or shading does not convey or indicate any preference or requirement for the specific materials, material properties, dimensions, proportions, commonalities between the components shown, and / or any other characteristics, attributes, properties, etc. of the components. In addition, in the accompanying drawings, the sizes and relative sizes of the components may be exaggerated for clarity and / or descriptive purposes. When the exemplary embodiments can be implemented differently, the specific process sequence can be performed in a different order than described. For example, two successively described processes can be performed substantially simultaneously or in an order opposite to the described order. In addition, the same figure numbers represent the same components.

[0025] When a component is referred to as being "on" or "over," "connected to," or "coupled to" another component, the component may be directly on, directly connected to, or directly coupled to the other component, or intervening components may be present. However, when a component is referred to as being "directly on," "directly connected to," or "directly coupled to" another component, there are no intervening components. For this purpose, the term "connected" may refer to a physical connection, an electrical connection, etc., with or without intervening components.

[0026] The terms used herein are for the purpose of describing specific embodiments, and are not intended to be restrictive. As used herein, unless the context clearly indicates otherwise, the singular forms "one (kind, person)" and "said (the)" are also intended to include plural forms. In addition, when the terms "comprise" and / or "include" and their variations are used in this specification, it is explained that there are stated features, integral bodies, steps, operations, parts, assemblies and / or their groups, but it is not excluded that there are or add one or more other features, integral bodies, steps, operations, parts, assemblies and / or their groups. It should also be noted that, as used herein, the terms "substantially", "approximately" and other similar terms are used as approximate terms and not as degree terms, so that they are used to explain the inherent deviations of the measured values, calculated values ​​and / or the values ​​provided that will be recognized by those of ordinary skill in the art.

[0027] Figure 1 The following is a flowchart of the method for processing social governance texts disclosed in the present invention. Figure 1 The social governance text processing method S100 disclosed in the present invention is described in detail.

[0028] The social governance text processing method S100 includes the following steps:

[0029] Step S102: pre-process each social governance text based on the custom dictionary to obtain the people's livelihood feature data corresponding to the social governance text.

[0030] Step S104, calling the document topic generation model to analyze the people's livelihood feature data, and generating people's livelihood analysis information about the social governance text, wherein the people's livelihood analysis information at least includes the people's livelihood topics associated with each social governance text, the distribution probability of people's livelihood segmentation on the people's livelihood topics, and the distribution probability of each social governance text on the people's livelihood topics.

[0031] Step S106, constructing an easy-to-read display page based on the livelihood analysis information.

[0032] Social governance texts are information on public sentiment and public opinion obtained through various feedback channels, mainly including complaints from the public on various livelihood topics. Feedback channels usually include livelihood service hotlines. Social governance texts are usually recorded based on the oral statements of the public and are unstructured data. Every feedback from each citizen will generate a corresponding social governance text, so the amount of data in social governance texts is relatively large.

[0033] Preprocessing is the process of extracting people's livelihood keywords from social governance texts. Through preprocessing, missing values, repeated values, noise data and redundant data in social governance texts can be eliminated, and the word segmentations that best reflect the social governance texts can be screened out.

[0034] The people’s livelihood feature data is the preprocessing result of the social governance text, which is used to characterize the relationship between the social governance text and the people’s livelihood keywords, including the text matrix and the word segmentation matrix.

[0035] The custom dictionary at least includes a geographic information dictionary, a social governance event dictionary, and a discontinued dictionary. The geographic information dictionary is a data set constructed based on historical documents, which contains various geographic names in real life scenarios and custom names that represent geographic information. Calling the geographic information dictionary avoids the cost of calling online map data and improves the efficiency of identifying spatial geographic information in social governance texts. In addition, since many custom names composed of non-geographical words are difficult to accurately identify, the preset geographic information dictionary avoids the omission of information or recognition errors in the process of obtaining spatial geographic information. By referring to the geographic information dictionary and the social governance event dictionary, the word segmentation results are more accurate and meaningful; by referring to the discontinued dictionary, meaningless words in the analysis of social governance texts are eliminated, reducing the processing volume of the subsequent document topic generation model and improving its execution efficiency.

[0036] Among them, spatial geographic information refers to the geographical terms mentioned in social governance texts that are related to the information reflected, usually representing the location of the event or the location of the object of the subject's appeal.

[0037] The document topic generation model is a model used to analyze people's livelihood characteristic data. Through analysis, information such as hot events mapped by social governance texts can be obtained. Compared with manual mining of hidden information, it has more accurate and efficient mining capabilities.

[0038] In some embodiments, the specific implementation of step S102 includes: cleaning the social governance text, at least removing missing values, repeated values, noise data and redundant data in the social governance text to obtain clean text; based on a custom dictionary, performing word segmentation on the clean text to construct a corpus formed by multiple people's livelihood keywords; and converting each social governance text and the corresponding corpus into a matrix form to construct people's livelihood feature data composed of a text matrix and a word segmentation matrix, wherein the text matrix includes document number, document sequence number, people's livelihood keyword number and its word frequency, and the word segmentation matrix includes people's livelihood keywords, people's livelihood keyword numbers and their word frequencies.

[0039] In some embodiments, based on the custom dictionary, the clean text is segmented to construct a corpus formed by multiple livelihood keywords, including: based on the geographic information dictionary and the social governance event dictionary in the custom dictionary, the clean text is segmented to obtain multiple livelihood segmentations; according to the stop dictionary in the custom dictionary, the stop words of the multiple livelihood segmentations are removed to screen out multiple livelihood keywords for representing the livelihood information of the social governance text; and the various livelihood keywords are integrated to form a corpus constructed by the various livelihood keywords.

[0040] In some embodiments, step S106 includes: analyzing the people's livelihood analysis information to mine related data of the social governance text, wherein the related data includes at least multiple hot events and their development trends, and the number of people's livelihood topics; and processing the related data in a display form to form an easy-to-read display page.

[0041] In some implementations, before step S106, it also includes: establishing a visual interactive clustering system to obtain error correction constraint instructions for people's livelihood analysis information.

[0042] In some embodiments, before step S102, it includes: according to the application scenario of the social governance text, building a custom dictionary suitable for the application scenario, wherein the custom dictionary includes at least a geographic information dictionary, a social governance event dictionary and a deactivated dictionary.

[0043] In some embodiments, it also includes: extracting any livelihood keyword from any livelihood theme in the social governance text; performing semantic analysis on the livelihood keywords, and binding multiple livelihood keywords with semantics to the corresponding same constraint identifier; mapping multiple livelihood keywords associated with the constraint identifier to the extracted livelihood theme.

[0044] Figure 2 This is a diagram of the processing method of the social governance text disclosed in this disclosure. Figure 2 The approach to social governance texts is elaborated in more detail.

[0045] Data collection and establishment of training sample library. The data may include government service livelihood hotline data of any region (such as a prefecture-level city), and use this data as social governance text. Specifically, the social governance text of the government service livelihood hotline of any region is obtained from the citizen hotline intelligent system of any region, and more than 10,000 pieces of social governance text data are obtained. The livelihood documents collected in each scene are stored in the corresponding database to form a training text library, so as to facilitate the subsequent preprocessing of each social governance text.

[0046] Construct a geographic information dictionary. Existing geographic information word segmentation dictionaries are mostly at the province, city, district, county, township, community, and road levels. The coverage of the word segmentation dictionary has a great impact on the word segmentation results, and the accuracy of the word segmentation must be guaranteed. This disclosure automatically identifies geographic location entities in historical documents and establishes a geographic location information dictionary, which not only greatly improves the word segmentation accuracy of the complaint corpus, but also accurately extracts the geographic location, making it convenient for higher-level management departments to conduct statistics and analysis on livelihood complaints occurring in communities, streets, and grids; further discover problems existing in each administrative division, and handle various problems in a centralized and graded manner, thereby achieving the government's refined governance and refined services for society.

[0047] The construction process of the geographic information dictionary is as follows:

[0048] First, call the online map to obtain all the geographical terms and longitudes and latitudes contained in the keywords of the historical documents (i.e., training texts) to establish a geographical name database (i.e., geographical information dictionary); for words that do not have longitudes and latitudes, classify them as non-geographic terms to form a custom name database (i.e., custom dictionary). When identifying spatial geographic information in social governance texts, traverse each keyword in the geographical name database and the custom name database. If no keyword is found, call the geographical data in the online map, thereby reducing the number of calls to the online map structure and improving efficiency.

[0049] Secondly, optimize the geographic location information dictionary. The geographic information dictionary obtained above may contain some useless information. For example, all feedback opinions in Beijing's administrative districts involve "Beijing". In this scenario, mentioning "Beijing" is of little significance for locating the associated location of the complaint. It can be extracted as useless information, while also avoiding interference in the front-end display process. The method of removing useless data is to obtain the contour data of a certain area on Baidu Maps and form a shaps file; project each geographic term in the geographic location information dictionary on the shaps file and remove those points that fall outside the contour boundary.

[0050] Thirdly, historical documents and spatial geographic information are integrated and stored in the database. Based on the historical documents, the regional code, occurrence time and the lowest level geographic term information (including nouns and coordinates) in each historical document are added to each data.

[0051] Finally, by comparing the livelihood characteristic data with the geographic location information dictionary, the inaccurate word segmentation is corrected, and the geographic name database and the custom name database are expanded.

[0052] Data preprocessing. Perform preprocessing operations such as cleaning, word segmentation, stop word removal, and feature extraction on the original social governance text data. Including: cleaning the collected messy and unstructured raw data to remove missing values, duplicate values, noise data, redundant data, etc.; based on the above-mentioned custom dictionary library, calling the precise mode of the jieba database to perform word segmentation on the social governance text to form multiple livelihood word segmentations; using the stop word list in the stop word removal tool and the custom stop dictionary to remove meaningless words, and finally obtain effective word segmentation results (that is, a corpus composed of multiple livelihood keywords).

[0053] In the word segmentation process, single words in the data are filtered out, and only the word frequency matrix of nouns and verbs is produced. After processing, the social governance text obtains a text matrix containing document information and a word segmentation matrix containing word information. The text matrix includes the document number, document sequence number, livelihood keyword number and its word frequency, and the word segmentation matrix includes livelihood keywords, livelihood keyword numbers and their word frequencies. Using the data provided in the text matrix and word segmentation matrix, the relationship between social governance text and livelihood keywords can be constructed.

[0054] Text feature extraction is an important part of natural language processing and a key step in text analysis. It is one of the effective methods to reduce the dimension of text data and improve the performance of task completion. This paper adopts the TF-IDF (termfrequency–inverse document frequency) algorithm to extract keywords from social governance texts and assign word weights.

[0055] Topic modeling and optimization. The document topic generation model disclosed in the present invention can be modeled using the LDA (Latent Dirichlet Allocation, document topic generation model) model. The document topic generation model can be used to represent each short social governance text as a vector in a low-dimensional space. The space of low-dimensional topic vectors avoids the sparsity problem and helps to explore the implicit associations of various people's livelihood topics, thereby reducing the impact of data row attributes on measuring the similarity between documents. Then, use the gensim library to establish a corpus for the preprocessed people's livelihood keywords and vectorize the corpus; establish a bag of words model, and extract important topics through the document topic generation model.

[0056] After the preprocessed data is input into the document topic generation model, a cluster of implicit topics is obtained. Then, all the test texts are converted into topic representations, and then the approximate iterative method of extracting sample values ​​is used to indirectly calculate the model parameters and obtain the probability distribution of social governance texts on the topic set, d = {K1, K2, K3, ..., KT}, where T is the number of topics and K is the probability. The document topic generation model uses the bag of words method, treating each document as a word frequency vector, thereby converting document information into digital information that is easy to model. For each document in the corpus, the document topic generation model defines the following generation process:

[0057] For each document, extract a topic from the topic distribution;

[0058] Extract a word from the word distribution corresponding to the extracted topic;

[0059] Repeat the above steps until every word in the document is traversed.

[0060] Algorithm flow:

[0061] choose parameterθ~P(θ)

[0062] for each of then N words W n

[0063] Choose a topic Z n ~P(Z|θ)

[0064] Choose a word W n ~P(W|Z)

[0065] Among them, θ is a livelihood topic vector, each column of the vector represents the probability of each livelihood topic appearing in the document, and the vector is a non-negative normalized vector; P(θ) is the distribution of θ P(θ)~Dir(α), N and W n is the total number of words and the nth word; Z n represents the selected livelihood theme, P(Z|θ) represents the probability distribution of livelihood theme Z given θ, specifically the value of θ, that is, P(Z=i|θ)=θ i , P(W|Z) represents the probability distribution of livelihood topic W given Z.

[0066] However, during the model training process, some phenomena were found, such as keywords belonging to the same topic being assigned to different topics, and keywords that have nothing to do with each other being assigned to the same topic. In order to avoid these problems, the present invention implements a visual interactive clustering system, and users can add constraint identifiers to the topic model through word clouds, such as stipulating that a group of words must appear in the same topic, and iteratively optimize the topic model.

[0067] In addition, the present disclosure proposes a visual interactive clustering system, and the visual display effect adds constraints to incorrect model calculation results, making the model calculation results more accurate. The system display page consists of two parts: a word cloud and a topic bubble chart. The word cloud shows the similarity of the topic distribution of the segmented words. The segmented words with similar distributions are close to each other, and the segmented words with large differences are far away from each other. When the segmented words with similar semantics are far away, they can be brought closer by adding constraint identifiers; that is, the user can select the segmented words that need to be constrained according to the visual word cloud chart, and add appropriate constraint identifiers to them to correct the model output results. The user can also click on a group of segmented words continuously to add constraints, iteratively optimize the model, and make the model results more accurate.

[0068] The generation process of the optimized document topic generation model is as follows:

[0069] For each social governance text, a livelihood topic is extracted from the topic distribution (each branch generated by the topic distribution may be a livelihood keyword or a constraint identifier); a livelihood keyword or constraint identifier is extracted from the word distribution corresponding to the extracted topic; if a livelihood keyword is extracted, the livelihood keyword is selected; if a constraint identifier is extracted, the corresponding constraint index is selected, and then a livelihood keyword belonging to the constraint is generated from the livelihood keyword distribution of this constraint.

[0070] Repeat the above steps until every livelihood keyword in the social governance text has been traversed.

[0071] This model introduces two control parameters α, β and a hyperparameter constraining the distribution

[0072] α: The distribution P(θ) requires a vector parameter, which is used to generate a topic θ vector and obtain the topic distribution probability on each document.

[0073] β: The probability distribution matrix P(W|Z) of livelihood keywords corresponding to each topic.

[0074] Through the generation process of the document topic generation model, it can be seen that the model mainly learns and trains two control parameters α, β and the parameters of the constraint label distribution from the given input corpus. Learning these three parameters determines the model.

[0075] Figure 3 This is a schematic diagram of the associated data display page for the public service hotline scenario. Figure 4 The figure is a schematic diagram of a related data display page for an application scenario of the present disclosure.

[0076] like Figure 3 and Figure 4 As shown, when the results are visualized, the display page includes several parts such as the distance between subject words, the top 30 most frequent words, the five hot events, and the frequently occurring word cloud of the text. The distance between subject words is obtained through multidimensional scaling analysis, and each topic is represented by a bubble. The importance of each livelihood topic can be represented by a circle with a number. The larger the circle, the more important the corresponding livelihood topic. The similarity between various livelihood topics can be represented by the interval between the centers of the circles. Of course, if too many topics are extracted, there will be a bubble fitting problem during visualization. In order to solve this problem, the present disclosure also refers to the perplexity obtained by calculation, first uses the perplexity to judge the approximate range of the number of topics, and then combines the situation of the visualized topic density distribution map to judge the quality of the model, and continuously adjusts the number of livelihood topics until the model is optimal, and then determines the optimal number of topics.

[0077] exist Figure 3 The report shows the results of people's livelihood analysis of various social governance texts in the scenario of people's livelihood service hotline. The identified hot events include property management, urban environment, education management, etc. Among them, there are 2,619 cases of property management, 1,741 cases of urban environment, and 848 cases of education management.

[0078] The people's livelihood themes corresponding to the five types of hot events are displayed in the form of bubbles on the left side of the page. The larger the data volume of the related events, the larger the area of ​​the circle.

[0079] The most frequent words include property, unpaid, student, noise, worker, vehicle, school, contractor, house, teacher, farmland, teacher, handover, contractor, disturbing, cleanup, kindergarten, collection, refund, construction, payment, water fee, project fee, construction site, etc., and the word frequency is displayed in the form of statistical bars. Of course, you can set the display of other word frequencies according to the page display requirements.

[0080] The number of documents for the five major hot events is displayed in the form of a bar chart on the display page.

[0081] The hottest words in the text are displayed in the form of bubbles. Keywords with similarity are closer; conversely, they are farther away. If there are keywords with similar semantics but far away, you can manually set constraint marks to feed back to the model for optimization.

[0082] Figure 4This is the display page of another application scenario, and its display information is consistent with that of the livelihood service hotline scenario. Hot events include five categories: housing security, compensation, living security, labor security, and illegal complaints. The most frequently used words include: community, owner, developer, handling, property, real estate certificate, house, child, resident, delivery, house, installation, household registration, fee, unreasonable, staff, reading, elementary school, housing authority, school, appraisal, life, fee, resident, safety hazard, parking space, commercial housing, electric car, payment, water leakage, etc., and according to the selected livelihood theme, the keywords involved will be highlighted for easy query.

[0083] The method for processing social governance text data disclosed in the present invention ensures the accuracy of word segmentation of geographic information in social governance text documents by constructing a customized social governance text dictionary and a geographic information dictionary; further reduces the number of useless words by customizing the removal of stop words, and realizes the binding of related word segmentations by setting constraint identifiers for word segmentations, thereby ensuring the credibility of the analysis results; utilizes the document topic generation model to obtain data such as key topics, topic word segmentation sets, and related probability distributions, and implements a visual interactive clustering system to optimize the model; finally, statistically analyzes the model calculation results and mines hot events, and finally displays all analysis and mining results on a large-screen visualization, converts disordered complaint information into ordered text data, and mines deeper useful information hidden in social governance text documents, providing data support for urban management and decision-making.

[0084] Figure 5 This is a block diagram of a processing device for the social governance text disclosed in the present invention.

[0085] like Figure 5 As shown, the present disclosure proposes a social governance text processing device 1000, including: a preprocessing module 1002, used to preprocess each social governance text based on a custom dictionary, and obtain the people's livelihood feature data corresponding to the social governance text; a data analysis module 1004, used to call the document topic generation model to analyze the people's livelihood feature data, and generate people's livelihood analysis information about the social governance text, wherein the people's livelihood analysis information at least includes the people's livelihood themes associated with each social governance text, the distribution probability of people's livelihood segmentation on the people's livelihood themes, and the distribution probability of each social governance text on the people's livelihood themes; and a display module 1006, used to construct an easy-to-read display page according to the people's livelihood analysis information, wherein the display page includes a constraint setting area.

[0086] The various modules of the social governance text processing device 1000 are used to execute the social governance text processing method. The execution principles and implementation steps of each module can be referred to above and will not be repeated here.

[0087] The device may include a corresponding module for performing each or several steps in the above flowchart. Therefore, each step or several steps in the above flowchart may be performed by a corresponding module, and the device may include one or more modules in these modules. The module may be one or more hardware modules specifically configured to perform the corresponding steps, or implemented by the processor 1300 configured to perform the corresponding steps, or stored in a computer-readable medium for implementation by the processor 1300, or implemented by some combination.

[0088] The hardware structure can be implemented using a bus architecture. The bus architecture can include any number of interconnected buses 1100 and bridges, depending on the specific application and overall design constraints of the hardware. The bus 1100 connects various other circuits 1400 including one or more processors 1300, memory 1300, and / or hardware modules together. The bus 1100 can also connect various other circuits 1400 such as peripherals, voltage regulators, power management circuits, external antennas, etc.

[0089] The bus 1100 may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Component (EISA) bus, etc. The bus 1100 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the figure only uses one connecting line, but does not mean that there is only one bus or one type of bus 1100.

[0090] Any process or method description in the flowchart or otherwise described herein may be understood to represent a module, fragment or portion of a code including one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present disclosure includes additional implementations, in which functions may not be performed in the order shown or discussed, including in a substantially simultaneous manner or in a reverse order according to the functions involved, which should be understood by those skilled in the art to which the embodiments of the present disclosure belong. The processor 1300 performs the various methods and processes described above. For example, the method implementation in the present disclosure may be implemented as a software program, which is tangibly contained in a machine-readable medium, such as a memory 1300. In some embodiments, part or all of the software program may be loaded and / or installed via the memory 1300 and / or a communication interface. When the software program is loaded into the memory 1300 and executed by the processor 1300, one or more steps in the method described above may be performed. Alternatively, in other embodiments, the processor 1300 may be configured to perform one of the above methods in any other appropriate manner (e.g., by means of firmware).

[0091] The logic and / or steps represented in the flowchart or otherwise described herein may be embodied in any readable storage medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including processor 1300, or other system that can fetch instructions from and execute instructions on an instruction execution system, apparatus, or device), or in combination with such instruction execution systems, apparatuses, or devices.

[0092] For the purposes of this specification, a "readable storage medium" may be any device that can contain, store, communicate, propagate or transmit a program for use with or in conjunction with an instruction execution system, device or apparatus. More specific examples of readable storage media (a non-exhaustive list) include the following: an electrical connection with one or more wirings (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and editable read-only memory (EPROM or flash memory), a fiber optic device, and a portable read-only memory (CDROM). In addition, the readable storage medium may even be paper or other suitable medium on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a memory.

[0093] It should be understood that the various parts of the present disclosure can be implemented in hardware, software, or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented in software stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0094] A person skilled in the art may understand that all or part of the steps of implementing the above-mentioned implementation method may be completed by instructing related hardware through a program, and the program may be stored in a readable storage medium, which, when executed, includes one or a combination of the steps of the implementation method.

[0095] In addition, each functional unit in each embodiment of the present disclosure may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a readable storage medium. The storage medium may be a read-only memory, a disk or an optical disk, etc.

[0096] Those skilled in the art should understand that the above embodiments are only for the purpose of clearly illustrating the present disclosure, and are not intended to limit the scope of the present disclosure. For those skilled in the art, other changes or modifications may be made based on the above disclosure, and these changes or modifications are still within the scope of the present disclosure.

Claims

1. A method for processing social governance text, characterized in that: include: Based on the custom dictionary, each social governance text is preprocessed to obtain the livelihood characteristic data corresponding to the social governance text; Calling a document topic generation model to analyze the livelihood feature data to generate livelihood analysis information about the social governance texts, wherein the livelihood analysis information at least includes livelihood topics associated with each of the social governance texts, distribution probabilities of the livelihood participles on the livelihood topics, and distribution probabilities of each of the social governance texts on the livelihood topics; and Based on the livelihood analysis information, an easy-to-read display page is constructed.

2. The method for processing social governance text according to claim 1 is characterized in that: Based on the custom dictionary, each social governance text is preprocessed to obtain the livelihood feature data corresponding to the social governance text, including: Cleaning the social governance text, at least removing missing values, duplicate values, noise data and redundant data in the social governance text, to obtain a clean text; Based on the custom dictionary, the clean text is segmented to construct a corpus formed by a plurality of livelihood keywords; and Each of the social governance texts and the corresponding corpus is converted into a matrix form to construct the people's livelihood feature data consisting of a text matrix and a word segmentation matrix, wherein the text matrix includes the document number, document sequence number, people's livelihood keyword number and its word frequency, and the word segmentation matrix includes the people's livelihood keywords, the people's livelihood keyword numbers and their word frequencies.

3. The method for processing social governance text according to claim 2 is characterized in that: The step of performing word segmentation processing on the clean text based on the custom dictionary and constructing a corpus formed by a plurality of livelihood keywords includes: Based on the geographic information dictionary and the social governance event dictionary in the custom dictionary, the clean text is segmented to obtain multiple livelihood segmentations; According to the stop word dictionary in the custom dictionary, the plurality of livelihood participles are removed from the stop words, and the plurality of livelihood keywords used to characterize the livelihood information of the social governance text are screened out; and Integrate the various livelihood keywords to form the corpus constructed by the various livelihood keywords.

4. The method for processing social governance text according to claim 1 is characterized in that: The step of constructing a display page that is easy to read based on the livelihood analysis information includes: Analyze the livelihood analysis information to mine the associated data of the social governance text, wherein the associated data at least includes a plurality of hot events and their development trends, and the number of the livelihood topics; and The associated data is processed in a display form to form a display page that is easy to read.

5. The method for processing social governance text according to claim 4 is characterized in that: Before constructing a display page that is easy to read according to the livelihood analysis information, the method further includes: A visual interactive clustering system is established to obtain error correction constraint instructions for the livelihood analysis information to optimize the document topic generation model.

6. The method for processing social governance text according to claim 1 or 2, characterized in that: Before preprocessing each social governance text based on the custom dictionary and obtaining the livelihood feature data corresponding to the social governance text, the method includes: According to the application scenario of the social governance text, a custom dictionary suitable for the application scenario is constructed, wherein the custom dictionary at least includes a geographic information dictionary, a social governance event dictionary and a stop word removal dictionary.

7. The method for processing social governance text according to claim 1 is characterized in that: Also includes: Extract any livelihood keyword from any livelihood theme in the social governance text; Performing semantic analysis on the livelihood keywords, and binding a plurality of livelihood keywords having the semantics to the same corresponding constraint identifier; The plurality of livelihood keywords associated with the constraint identifier are mapped to the extracted livelihood theme.

8. A social governance text processing device, characterized in that: include: A preprocessing module, used to preprocess each social governance text based on a custom dictionary to obtain livelihood feature data corresponding to the social governance text; A data analysis module, used to call a document topic generation model to analyze the livelihood feature data, and generate livelihood analysis information about the social governance text, wherein the livelihood analysis information at least includes the livelihood topics associated with each of the social governance texts, the distribution probability of the livelihood participles on the livelihood topics, and the distribution probability of each of the social governance texts on the livelihood topics; as well as A display module is used to construct an easy-to-read display page based on the livelihood analysis information, wherein the display page includes a constraint condition setting area.

9. An electronic device, characterized in that: It includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the method for processing social governance texts as described in any one of claims 1 to 7.

10. A readable storage medium, characterized in that: The readable storage medium stores execution instructions, which, when executed by a processor, are used to implement the method for processing social governance texts described in any one of claims 1 to 7.

Images