A big data analysis method and system based on community governance data

Through preprocessing and sentiment analysis of community group chat records, using algorithms such as word2vec and BiLSTM, we can identify positive and negative emotional tendencies and topics in chat records in real time, solving the problem of lagging response in community governance and achieving more accurate community governance manage.

CN113569008BActive Publication Date: 2024-11-26NANJING QIXIA DISTRICT CIVIL AFFAIRS SERVICE CENT
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110816825.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-07-20
Publication Date
2024-11-26
Estimated Expiration
2041-07-20

AI Technical Summary

Technical Problem

It is difficult for the existing technology to effectively analyze and utilize positive and negative emotional tendencies in community group chat records, resulting in lagging or inaccurate community governance responses.

Method used

Using a big data analysis method based on community governance data, by collecting and preprocessing chat records, using algorithms such as word2vec, K-means and BiLSTM to generate word vectors, conduct sentiment analysis and public opinion classification, and identify positive and negative emotional tendencies in chat records in real time. and topics, build a public opinion classification table to realize real-time analysis and classification.

Benefits of technology

It improves the accuracy and efficiency of sentiment analysis, can quickly identify key topics and events that community residents are concerned about, and promotes timely response and management of community governance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113569008B_ABST
    Figure CN113569008B_ABST
Patent Text Reader

Abstract

The present invention proposes a big data analysis method and system based on community governance data, collects community group chat records, extracts the main body of chat content as a corpus; performs word segmentation operation on the corpus according to the user's stop word list and jieba word segmentation table, generates word vectors of word segmentation using the word2vec neural network algorithm, performs cluster analysis on the word vectors using the K-means algorithm, and constructs a public opinion classification table; annotates the corpus with positive and negative sentiment tendencies, and trains a sentiment analysis model based on the BiLSTM algorithm; receives real-time community group chat records, extracts the main body of chat content for word segmentation and vectorization operations, obtains similar word sets on the one hand, compares the similar word sets with the public opinion classification table to determine the comprehensive public opinion classification result, and on the other hand, inputs the word vector of the target sentence into the sentiment analysis model to determine the sentiment tendency label. The present invention can well analyze the topics and events that community residents focus on, and promote better community governance.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of natural language analysis, and in particular to a big data analysis method and system based on community governance data. Background Art

[0002] Currently, we can obtain residents' chat dynamics in real time through community groups. By using big data natural language processing technology, we can analyze the positive and negative sentiment tendencies of community group chat records to effectively and timely reflect residents' needs. For example, after obtaining a target chat record sentence, we can analyze the sentiment tendency in real time, such as the sentence "The street light next to Building 24 is broken and needs to be repaired." The sentiment tendency is a negative label. When we obtain a negative sentence, we can respond quickly and arrange for a repairman to repair it. This will achieve very good results in community governance. Summary of the invention

[0003] The purpose of the present invention is to propose a big data analysis method and system based on community governance data.

[0004] The technical solution to achieve the purpose of the present invention is: a big data analysis method based on community governance data, comprising the following steps:

[0005] Step 1: Collect community group chat records, extract the main body of chat content as a corpus, and store it in a database;

[0006] Step 2: Segment the corpus according to the user's stop word list and Jieba word segmentation table, and generate word vectors for the segmentation using the word2vec neural network algorithm;

[0007] Step 3: Use the K-means algorithm to perform cluster analysis on the word vectors in step 2, determine cluster topics and topic keywords, and construct a public opinion classification table;

[0008] Step 4, annotating the corpus with positive and negative sentiment tendencies, and training the sentiment analysis model. The training process of the sentiment analysis model is as follows: first, the word vector sequence of step 2 is processed using the BiLSTM algorithm, and then the processing result is mapped using the hyperbolic tangent function, and the sentiment category with the largest mapping value is used as the predicted sentiment tendency label through softmax, and finally, the loss function is calculated according to the predicted sentiment tendency label and the true positive and negative sentiment tendency label, and the positive and negative sentiment tendency analysis model parameters are adjusted according to the loss function to complete the training;

[0009] Step 5: Receive real-time community group chat records, extract the main body of the chat content for word segmentation and vectorization operations, calculate the similarity with the word vector obtained in step 2 to obtain a similar word set, compare the similar word set with the topic keywords in the public opinion classification table to determine the comprehensive public opinion classification result of the target sentence, and at the same time input the word segmentation vector of the target sentence into the trained sentiment analysis model to obtain the sentiment tendency label of the target sentence.

[0010] Furthermore, in step 1 and step 5, the specific method for extracting the chat content body from the community group chat record is:

[0011] The chat records are preprocessed to remove advertisements, weather, system prompts, emoticons, blank lines, and nicknames from the chat records to obtain the main body of the chat content.

[0012] Furthermore, in step 3, the K-means algorithm is used to perform cluster analysis on the word vectors in step 2, determine cluster topics and topic keywords, and construct a public opinion classification table, where:

[0013] The clustering topics are large categories, including repair and maintenance, environmental sanitation, medical education, community autonomy, parking management, security patrol, convenient life, vegetable market management, civilized neighborhood life, and conflict mediation; the topic keywords are the word vectors corresponding to the word points whose distance from the cluster center is less than the set threshold.

[0014] Furthermore, in step 4, the corpus is labeled with positive and negative sentiment tendencies, and a sentiment analysis model is trained. The specific method in which the BiLSTM algorithm processes the word vector sequence in step 2 is as follows:

[0015] The word vector sequence is input into the BiLSTM encoding end for encoding to obtain the word context semantic features, and then the sentence semantic features are obtained by vector splicing, and the semantic features are input into the BiLSTM decoding end for decoding; the input dimension of the BiLSTM encoding end is determined by the sentence length ratio and sentence length distribution in the corpus.

[0016] Furthermore, step 4 also includes the following steps:

[0017] The corpus is divided into training set and validation set according to the data volume of 1:10 for training the sentiment analysis model.

[0018] Furthermore, in step 5, the specific method of comparing the similar word set with the topic keywords in the public opinion classification table to determine the comprehensive public opinion classification result of the target sentence is:

[0019] Compare the similar word set with the topic keywords in the public opinion classification table. If there is more than one topic keyword in the similar word set, the comprehensive public opinion classification result of the target sentence is determined to be the cluster topic with the largest number of corresponding topic keywords. If there is only one topic keyword in the similar word set, the comprehensive public opinion classification result of the target sentence is determined to be the cluster topic where the corresponding topic keyword is located. If there is no topic keyword in the similar word set, the comprehensive public opinion classification result of the target sentence is determined to be others. Finally, the comprehensive public opinion classification result of the target sentence is obtained.

[0020] Furthermore, step 5 also includes the following steps:

[0021] The target sentences and the classification labels of the positive and negative sentiment tendencies of the target sentences are stored in the database to expand the corpus.

[0022] A big data analysis system based on community governance data implements big data analysis based on community governance data based on the big data analysis method.

[0023] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements big data analysis based on community governance data based on the big data analysis method.

[0024] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements big data analysis based on community governance data based on the big data analysis method.

[0025] Compared with the prior art, the present invention has the following significant advantages: 1) After preprocessing the community group chat records, irrelevant information in the sentences is removed, the main body of the text is retained, and the semantics of the word features in the context are accurately reflected, which improves the credibility of the context semantic feature sequence. 2) The real-time community group chat records are classified into topics and positive and negative sentiment tendency labels, which can well analyze the topics and events that community residents focus on, and promote better community governance. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 is a flow chart of a big data analysis method based on community governance data of the present invention;

[0027] Figure 2 1 is an example diagram of sentence analysis of a social chat record of the present invention, wherein (a) is a graph showing the proportion of each length of sentences, and (b) is a graph showing the proportion of sentence length 43;

[0028] Figure 3 It is a schematic diagram of the sentiment analysis model training of the present invention;

[0029] Figure 4 It is a schematic diagram of the principle of predicting positive and negative emotional tendencies of the present invention;

[0030] Figure 5 It is a schematic diagram of the word vector model generated by the present invention;

[0031] Figure 6 It is a schematic diagram of the public opinion classification of the present invention. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0033] A big data analysis method based on community governance data includes the following steps:

[0034] Step 1: collect community group chat records, extract the main body of chat content as a corpus, and store it in a database; the specific method for extracting the main body of chat content from the community group chat records is:

[0035] The chat records are preprocessed to remove advertisements, weather, system prompts, emoticons, blank lines, and nicknames from the chat records, thus obtaining the main body of the chat content, which is used as the corpus.

[0036] Step 2: Segment the corpus according to the user's stop word list and Jieba word segmentation table, and generate word vectors for the segmentation using the word2vec neural network algorithm;

[0037] Step 3: Use the K-means algorithm to perform cluster analysis on the word vectors in step 2, determine cluster topics and topic keywords, and construct a public opinion classification table; where:

[0038] The clustering topics are large categories, including repair and maintenance, environmental sanitation, medical education, community autonomy, parking management, security patrol, convenient life, vegetable market management, civilized neighborhood life, and conflict mediation; the topic keywords are the word vectors corresponding to the word points whose distance from the cluster center is less than the set threshold.

[0039] Step 4, annotating the corpus with positive and negative sentiment tendencies, and training the sentiment analysis model. The training process of the sentiment analysis model is as follows: first, the word vector sequence of step 2 is processed using the BiLSTM algorithm, and then the processing result is mapped using the hyperbolic tangent function, and the sentiment category with the largest mapping value is used as the predicted sentiment tendency label through softmax, and finally, the loss function is calculated according to the predicted sentiment tendency label and the true positive and negative sentiment tendency label, and the positive and negative sentiment tendency analysis model parameters are adjusted according to the loss function to complete the training;

[0040] The corpus can be divided into a training set and a validation set according to a data volume of 1:10 for training the sentiment analysis model. In the sentiment analysis model, the specific method of the BiLSTM algorithm to process the word vector sequence is as follows:

[0041] The word vector sequence is input into the BiLSTM encoding end for encoding to obtain the word context semantic features, and then the sentence semantic features are obtained by vector splicing, and the semantic features are input into the BiLSTM decoding end for decoding; the input dimension of the BiLSTM encoding end is determined by the sentence length ratio and sentence length distribution in the corpus.

[0042] Step 5: Receive real-time community group chat records, extract the main body of the chat content for word segmentation and vectorization operations, perform similarity calculation with the word vector obtained in step 2 to obtain a similar word set, compare the similar word set with the topic keywords in the public opinion classification table to determine the comprehensive public opinion classification result of the target sentence, and at the same time input the word segmentation vector of the target sentence into the trained sentiment analysis model to obtain the sentiment tendency label of the target sentence;

[0043] After obtaining the positive and negative sentiment tendency classification labels of the target sentence, the target sentence and the positive and negative sentiment tendency classification labels of the target sentence can be stored in the database for expanding the corpus. In the comprehensive public opinion classification, the specific method of comparing the similar word set with the topic keywords of the public opinion classification table to determine the comprehensive public opinion classification result of the target sentence is:

[0044] Compare the similar word set with the topic keywords in the public opinion classification table. If there is more than one topic keyword in the similar word set, the comprehensive public opinion classification result of the target sentence is determined to be the cluster topic with the largest number of corresponding topic keywords. If there is only one topic keyword in the similar word set, the comprehensive public opinion classification result of the target sentence is determined to be the cluster topic where the corresponding topic keyword is located. If there is no topic keyword in the similar word set, the comprehensive public opinion classification result of the target sentence is determined to be others. Finally, the comprehensive public opinion classification of the target sentence is obtained.

[0045] The present invention also proposes a big data analysis system based on community governance data, and implements big data analysis based on community governance data based on the big data analysis method.

[0046] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements big data analysis based on community governance data based on the big data analysis method.

[0047] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements big data analysis based on community governance data based on the big data analysis method.

[0048] The present invention adopts a big data approach to solve the problem of analyzing the positive and negative sentiment tendencies of chat records in community groups. It can store massive data, and also improve the accuracy of the positive and negative sentiment tendency analysis of community group chat records under the analysis of massive data. The public opinion analysis of large amounts of data can accurately understand the issues and events that community residents are focused on and apply them to community governance.

[0049] Example

[0050] In order to verify the effectiveness of the solution of the present invention, the following simulation experiment is carried out.

[0051] In this embodiment, community group chat records (2.8 million) are collected. Since there is a lot of useless information in the chat records, data processing is performed first. Using the character matching principle, the sentence containing: "copy" and "Alipay" or "scan the code to receive the red envelope" removes the Alipay red envelope message; the sentence containing "[" and "】" or " <msg>'Remove advertising messages; remove weather messages if the sentence contains '[weather]' or 'temperature' and '℃'; remove useless characters in the message, such as blank line characters:' '、' '、' ', emoticon character: '\[{1}(.+)]', remove '@nickname' in a loop, and finally leave the main body of the chat content as the training set.

[0052] The training set is divided into a training set and a validation set in a ratio of 10:1. The training set and the validation set are labeled with positive and negative labels to create a dictionary: '1' is positive and '0' is negative. The sentence length ratio and sentence length distribution analysis in step 5 are performed on the training set, such as Figure 2 (a) is the proportion of each length of sentences, such as Figure 2 (b) The sentence length is 43 and the proportion of length is 0.91, so the BiLSTM input width is set to batch_size 43.

[0053] The processed community group chat records are segmented according to the user's stop word list and jieba word segmentation table, and then the Skip-gram model in word2vec is used to train the word vector model. The word vector model can be used for word similarity calculation and the construction of public opinion classification table. Figure 5 The generation principle and process of the word vector model are shown. The number of segmented words is V, the Skip-gram parameter skip_window is c / 2, which indicates the sliding window size, and the parameter num_skips is c, which indicates that c words are randomly selected from the sliding window of a central word. Each word is one-hot mapped to a V*1 matrix, and the dimension of the final word vector is set to 300. The dimension of the weight matrix w between the initial input layer and the hidden layer is 300*V. In the forward propagation: the one-hot (V*1) of the input central word and the weight matrix w (300*V) are multiplied as the hidden layer vector, which is expressed as: Where x is the one-hot of the input center word, the weight between the initial hidden layer and the output layer is recorded as w′, the dimension is 300*V, the hidden layer vector h and w′ are multiplied c times to obtain c*V output nodes, and the net input of each node is expressed as: where u c,j represents the net input of the jth word at the cth context position in the output layer, v' wj is the jth column vector of w'. Since each input node shares w', u cj =u j In order to facilitate probability representation, the vector u cj After softmax, c V*1 vectors are obtained, and each dimension of each V*1 vector represents a word in the corpus. The word represented by the position with the highest probability in the cth V*1 vector is the word in the cth context position predicted by the central word. Skip-Gram does not output a multinomial distribution at the output layer, but c multinomial distributions. The output of the jth word at the cth context position is expressed as:

[0054]

[0055] where w c,j is represented as the jth word in the cth context position of the output layer; w o,c represents the actual word at the cth context position, wI represents the central word of the input; p(w c,j =w o,c |wI) indicates that when a given input word is used as the central word, the probability that the word at the cth context position is the actual word at the cth context position is obtained; back propagation and stochastic gradient descent update weights: the loss function E is defined as:

[0056]

[0057] in is the index of the actual word in the cth upper and lower positions. The loss function E takes the derivative of w′ and obtains the gradient from the hidden layer to the output layer as:

[0058]

[0059] According to stochastic gradient descent, the weight update equation between the hidden layer and the output layer is expressed as:

[0060]

[0061] Where η is the learning rate, the loss function E takes the derivative of w, and the gradient of the weight from the input layer to the hidden layer is expressed as:

[0062]

[0063] According to stochastic gradient descent, the weight update equation between the input layer and the hidden layer is expressed as:

[0064]

[0065] In this embodiment, the preprocessed training set is segmented, and the word vector sequence obtained by training the word2vec 300-dimensional word vector model above is represented as S = [v1, v2, ..., v n ], S is the word vector sequence and v is the word vector, which is used as the input data of BiLSTM. The forward LSTM obtains the contextual semantic information of the word and outputs the state vector. The forward context vector is expressed as: The reverse LSTM obtains the contextual semantic information of the word and outputs a state vector. The reverse context vector is expressed as: Combined with the bidirectional output state of BiLSTM as the contextual semantics of the word, the contextual semantics is expressed as And generate the semantic feature sequence S1=[h1,h2,...,h n ], S1 is the context semantic sequence; S1 is used as the input of the LSTM network to generate a mapping in the sentiment classification space. The specific calculation process is as follows:

[0066] f j =σ(W f [S j-1 ,h j ])

[0067] i j =σ(W i [S j-1 ,h j ])

[0068]

[0069] o j =σ(W o [S j-1 ,h j ])

[0070] S j =o j *tanh(C j )

[0071] The LSTM model is composed of the input word h at time j. j , cell state C j , temporary cell state Hidden state S j , forget gate f j , Memory Gate j , output gate o j The calculation process of LSTM can be summarized as follows: by forgetting the information in the cell state and memorizing new information, the information useful for subsequent moment calculations can be transmitted, while the useless information is discarded, and the hidden state S is output at each time step. j , where forgetting, memory and output are determined by the hidden state S of the previous moment j-1 and the current input h j The calculated forget gate f j , Memory Gate j , output gate o j To control, we finally get a sequence S with the same number of word segments as the target sentence j . Use the softmax function to calculate the mapping S j The probability of belonging to different labels in the sentiment classification space is calculated as:

[0072]

[0073] Among them, p(l j,t ) is the prediction of community group chat records and sentences l j Positive and negative labels are the probabilities of category t; w m is the mth row of the weight matrix in the time-distributed fully connected layer network; S j is the output of the LSTM network decoding end; the positive and negative sentiment tendency classification labels with the highest probability are finally selected as the final output labels, which are the predicted label values ​​of the target sentence. The loss function is calculated using cross entropy based on the true positive and negative classification labels and predicted label values ​​of all validation sets. The calculation formula is:

[0074]

[0075] Where loss is the calculated loss value; It is a Boolean value indicating whether the community group chat record data m belongs to category t. If the Boolean value is true, it is 1 for positive, otherwise it is 0 for negative. is the probability that the sample data m of the community group chat record belongs to category t. Since the loss function falls between [0,1], the larger the loss value, the smaller the correct value. The loss values ​​are compared cyclically, and the model with the smaller loss value is retained. The optimal model is selected in iteration. Figure 3 The generation method and schematic diagram of training the optimal positive and negative sentiment tendency analysis model are demonstrated.

[0076] In addition, the k-means algorithm is used to perform cluster analysis on the word2vec 300-dimensional word vector trained above, using the clustering theme as the major category and the words closest to the center point as the topic keywords to obtain a public opinion classification table. The clustering results can be divided into the following major categories according to the ranking of topic popularity: repair and maintenance, environmental sanitation, medical education, community autonomy, parking management, public security patrol, convenient life, vegetable market management, civilized neighborhood life, and conflict mediation.

[0077] In this embodiment, the flask framework is used to build an interface for receiving real-time community group chat records. After receiving the real-time target sentence, the text is first preprocessed to remove advertisements, weather, system prompts, emoticons, blank lines, and nicknames in the text. The corpus is segmented according to the user's stop word list and the Jieba word segmentation table to obtain a word vector sequence, and then the sentence-level feature sequence is obtained through vector splicing and input into the model to obtain the positive and negative sentiment tendency scores, thereby obtaining the positive and negative sentiment tendency label category to which the sentence belongs, and the sentence and the result are stored in the database for use in expanding the corpus; at the same time, the word vector of each word segmentation of the sentence is compared with the word vector model for similarity, and the top 5 most similar words are obtained. The most similar words are fully matched with the topic keywords of the public opinion table to determine the major classification of the sentence and generate the public opinion analysis results.

[0078] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0079] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.< / msg>

Claims

1. A big data analysis method based on community governance data, It is characterized in that The steps include: Step 1: Collect community group chat records, extract the main body of chat content as a corpus, and store it in a database; Step 2: Segment the corpus according to the user's stop word list and Jieba word segmentation table, and generate word vectors for the segmentation using the word2vec neural network algorithm; Step 3: Use the K-means algorithm to perform cluster analysis on the word vectors in step 2, determine cluster topics and topic keywords, and construct a public opinion classification table; Step 4, annotating the corpus with positive and negative sentiment tendencies, and training the sentiment analysis model. The training process of the sentiment analysis model is as follows: first, the word vector sequence of step 2 is processed using the BiLSTM algorithm, and then the processing result is mapped using the hyperbolic tangent function, and the sentiment category with the largest mapping value is used as the predicted sentiment tendency label through softmax, and finally, the loss function is calculated according to the predicted sentiment tendency label and the true positive and negative sentiment tendency label, and the positive and negative sentiment tendency analysis model parameters are adjusted according to the loss function to complete the training; Step 5: Receive real-time community group chat records, extract the main body of the chat content for word segmentation and vectorization operations, perform similarity calculation with the word vector obtained in step 2 to obtain a similar word set, compare the similar word set with the topic keywords in the public opinion classification table to determine the comprehensive public opinion classification result of the target sentence, and at the same time input the word segmentation vector of the target sentence into the trained sentiment analysis model to obtain the sentiment tendency label of the target sentence; In step 1 and step 5, the specific method for extracting the chat content body from the community group chat record is: Pre-process the chat records, remove advertisements, weather, system prompts, emoticons, blank lines, and nicknames from the chat records, and obtain the main body of the chat content; Step 2: Segment the corpus according to the user's stop word list and Jieba word segmentation table, and use the word2vec neural network algorithm to generate word vectors for the segmentation. Specifically: The word vector model is trained using the Skip-gram model in word2vec. The number of segmented words is V, the Skip-gram parameter skip_window is c / 2, which indicates the sliding window size, and the parameter num_skips is c, which means that c words are randomly selected from the sliding window of a central word, and each word is one-hot mapped to a V*1 matrix. The dimension of the final word vector is set to 300, and the dimension of the weight matrix w between the initial input layer and the hidden layer is 300*V. In the forward propagation: the one-hot of the input central word and the weight matrix w are multiplied as the hidden layer vector, expressed as: h=w T x, where x is the one-hot of the input center word. The weight between the initial hidden layer and the output layer is recorded as w′, with a dimension of 300*V. The hidden layer vector h and w′ are multiplied c times to obtain c*V output nodes. The net input of each node is expressed as: where u c,j represents the net input of the jth word at the cth context position in the output layer, v' wj is the jth column vector of w'. Since each input node shares w', u cj =u j ; In order to facilitate probability representation, vector u cj After softmax, c V*1 vectors are obtained. Each dimension of each V*1 vector represents a word in the corpus. The word represented by the position with the highest probability in the cth V*1 vector is the word at the cth context position predicted by the central word. Skip-Gram does not output a multinomial distribution at the output layer, but c multinomial distributions. The output probability of the jth word at the cth context position is expressed as: where w c,j Represented as the jth word in the cth context position of the output layer; w o,c represents the actual word at the cth context position, wI represents the central word of the input; p(w c,j =w o,c |wI) represents the probability that the word at the cth context position is the actual word at the cth context position when a given input word is used as the central word; Back propagation, stochastic gradient descent update weights: define the loss function E as: in is the index of the actual word in the cth upper and lower positions. The loss function E takes the derivative of w′ and obtains the gradient from the hidden layer to the output layer as: According to stochastic gradient descent, the weight update equation between the hidden layer and the output layer is expressed as: Where η is the learning rate, the loss function E takes the derivative of w, and the gradient of the weight from the input layer to the hidden layer is expressed as: According to stochastic gradient descent, the weight update equation between the input layer and the hidden layer is expressed as: In step 3, the K-means algorithm is used to perform cluster analysis on the word vectors in step 2, determine the cluster topics and topic keywords, and construct a public opinion classification table, where: The clustering topics are large categories, including repair and maintenance, environmental sanitation, medical education, community autonomy, parking management, public security patrol, convenient life, vegetable market management, civilized neighborhood life, and conflict mediation; the topic keywords are the word vectors corresponding to the word vectors whose distance from the cluster center is less than the set threshold; The preprocessed training set is segmented, and the word vector sequence obtained by training the word2vec 300-dimensional word vector model above is represented as S = [v 1 ,v 2 ,...,v n ], S is the word vector sequence and v is the word vector, which is used as the input data of BiLSTM. The forward LSTM obtains the contextual semantic information of the word and outputs the state vector. The forward context vector is expressed as: The reverse LSTM obtains the contextual semantic information of the word and outputs a state vector. The reverse context vector is expressed as: Combined with the bidirectional output state of BiLSTM as the contextual semantics of the word, the contextual semantics is expressed as And generate the semantic feature sequence S of the target sentence 1 =[h 1 ,h 2 ,...,h n ],S 1 is the context semantic sequence; S 1 As the input of the LSTM network, the specific calculation process of generating the mapping in the sentiment classification space is as follows: S j =o j *tanh(C j ) The LSTM model is composed of the input word h at time j. j , cell state C j , temporary cell state Hidden state S j , forget gate f j , Memory Gate j , output gate o j The calculation process of LSTM can be summarized as follows: by forgetting the information in the cell state and memorizing new information, the information useful for subsequent moment calculations can be transmitted, while the useless information is discarded, and the hidden state S is output at each time step. j , where forgetting, memory and output are determined by the hidden state S of the previous moment j-1 and the current input h j The calculated forget gate f j , Memory Gate j , output gate o j To control, we finally get a sequence S with the same number of word segments as the target sentence j ; Use the softmax function to calculate the mapping S j The probability of belonging to different labels in the sentiment classification space is calculated as: Among them, p(l j,t ) is the prediction of community group chat records and sentences l j Positive and negative labels are the probabilities of category t; w m is the mth row of the weight matrix in the time-distributed fully connected layer network; S j is the output of the LSTM network decoding end; finally, the positive and negative sentiment tendency classification label with the maximum probability is selected as the final output label, which is the predicted label value of the target sentence; the loss function is calculated using cross entropy based on the true positive and negative classification labels and predicted label values ​​of all validation sets, and the calculation formula is: Where loss is the calculated loss value; It is a Boolean value indicating whether the community group chat record data m belongs to category t. If the Boolean value is true, it is 1 for positive, otherwise it is 0 for negative. It is the probability that the sample data m of the community group chat record belongs to category t. Since the loss function falls between [0,1], the larger the loss value, the smaller the correct value. The loss values ​​are compared cyclically, and the model with the smaller loss value is retained. The optimal model is selected in iteration.

2. According to the big data analysis method based on community governance data according to claim 1, It is characterized in that In step 4, the corpus is labeled with positive and negative sentiment tendencies, and the sentiment analysis model is trained. The specific method of the BiLSTM algorithm to process the word vector sequence in step 2 is as follows: The word vector sequence is input into the BiLSTM encoder for encoding, the word context semantic features are obtained, and then the sentence semantic features are obtained by vector concatenation, and the semantic features are input into the BiLSTM decoder for decoding; The input dimension of the BiLSTM encoding end is determined by the sentence length ratio and sentence length distribution in the corpus.

3. According to the big data analysis method based on community governance data according to claim 1, It is characterized in that Step 4 also includes the following steps: The corpus is divided into training set and validation set according to the data volume of 1:10 for training the sentiment analysis model.

4. According to the big data analysis method based on community governance data according to claim 1, It is characterized in that In step 5, the specific method of comparing the similar word set with the topic keywords in the public opinion classification table to determine the comprehensive public opinion classification result of the target sentence is: Compare the similar word set with the topic keywords in the public opinion classification table. If there is more than one topic keyword in the similar word set, the comprehensive public opinion classification result of the target sentence is determined to be the cluster topic with the largest number of corresponding topic keywords. If there is only one topic keyword in the similar word set, the comprehensive public opinion classification result of the target sentence is determined to be the cluster topic where the corresponding topic keyword is located. If there is no topic keyword in the similar word set, the comprehensive public opinion classification result of the target sentence is determined to be others. Finally, the comprehensive public opinion classification result of the target sentence is obtained.

5. According to the big data analysis method based on community governance data according to claim 1, It is characterized in that Step 5 also includes the following steps: The target sentences and the classification labels of the positive and negative sentiment tendencies of the target sentences are stored in the database to expand the corpus.

6. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements big data analysis based on community governance data based on the big data analysis method described in any one of claims 1 to 5.

7. A computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the big data analysis based on community governance data is implemented based on the big data analysis method described in any one of claims 1 to 5.