A public opinion detection and analysis system

Abstract

The present invention discloses a public opinion detection and analysis system thereof, which relates to the technical field of public opinion detection and analysis, and includes: a word sense association module, which is used to construct a word sense association tree of characters, wherein the root node is any character, the child nodes are characters related to the root node, and the child nodes are assigned values ​​according to the co-occurrence frequency, so that the sum of the values ​​of all child nodes is one; a semantic association module, which is used to construct a word sense association tree of each child node based on the child node of the word sense association tree as the root node, until a preset number of layers is reached to obtain a semantic association tree; a keyword detection module, which is used to match and analyze the semantic association trees of the document to be analyzed and the keyword, and obtain a set of public opinion keywords for the document to be analyzed. By constructing a word association tree, a co-occurrence frequency matrix and a multi-level similarity matching calculation model, high accuracy, dynamic adaptability and high robustness of keyword recognition are achieved, and the accuracy and stability of keyword recognition in complex contexts are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of public opinion detection and analysis, and specifically to a public opinion detection and analysis system. Background Art

[0002] With the rise of applications such as social media, intelligent customer service, and user experience analytics, demand for keyword spotting technology is increasing in areas such as social opinion monitoring, user behavior analysis, and mental health monitoring. Keyword spotting technology analyzes user behavior in text, voice, or video to identify their emotional state, helping systems or services more accurately understand user needs. However, existing keyword spotting technology faces the following limitations in practical applications: Inadequate keyword spotting accuracy: Existing keyword spotting typically relies on keyword dictionaries or datasets based on specific keyword tags. This approach works well for corpora containing specific keywords, but often performs poorly for complex contexts or text with implicit emotional expressions. For example, existing models struggle to effectively identify sentences containing homophones, resulting in low recognition accuracy. Lack of semantic dynamism: Most keyword spotting systems rely on static word associations or feature engineering and lack the ability to analyze dynamic semantic relationships between words. In particular, existing models struggle to accurately capture the polysemy of words or the shifting emotional tendencies of words in different contexts, compromising keyword identification accuracy. Summary of the Invention

[0003] Based on the above-mentioned shortcomings of the prior art, the purpose of the present invention is to provide a public opinion detection and analysis system to solve the above-mentioned technical problems.

[0004] To achieve the above objectives, the present invention provides the following technical solutions: a public opinion detection and analysis system, comprising:

[0005] The word sense association module is used to construct a word sense association tree for each word, where the root node is any word and the child nodes are words related to the root node. The child nodes are assigned values ​​based on the co-occurrence frequency so that the sum of the values ​​of all child nodes is one.

[0006] A semantic connection module is used to construct a word sense connection tree for each child node based on the child node of the word sense connection tree as a root node, until a preset number of layers is reached to obtain a semantic connection tree;

[0007] The keyword detection module is used to match and analyze the semantic connection tree of the document to be analyzed and the keywords to obtain the public opinion keyword set of the document to be analyzed.

[0008] The present invention is further configured such that the semantic connection module comprises: a data processing unit, a node construction unit and a node assignment unit;

[0009] A data processing unit is used to extract characters from historical texts and count their occurrence frequencies to generate a character frequency table, calculate the co-occurrence frequencies between characters, and generate a co-occurrence frequency matrix;

[0010] A node construction unit is used to select the corresponding word as the root node according to the descending order of the word frequency table, extract the words connected to the root node from the co-occurrence matrix, and fill the child nodes according to the order of co-occurrence frequency;

[0011] The node assignment unit is used to assign a value to each child node according to the co-occurrence frequency of the word corresponding to the child node and the root node, and the sum of all child nodes is one.

[0012] The present invention is further configured to calculate the co-occurrence frequencies between characters and generate a co-occurrence frequency matrix, including:

[0013] Create a word pair co-occurrence matrix C, where the rows and columns of the word pair co-occurrence matrix are all words that appear in the historical text;

[0014] Set the window size of the word pair and perform co-occurrence calculation for each word w i Traverse and find other words w within the window range j , and update the value of the corresponding position in the co-occurrence matrix C. The update logic is: C ′ [w i ,w j ]=C[w i ,w j ]+1,C ′ [w i ,w j ] is the number of times after updating, C[w i ,w j ] is other characters within the window range j The initial value is zero. Whenever other characters w appear within the window range, j , the value is increased by one, and after the traversal is completed, the statistical word w i and the word w j The number of co-occurrences of

[0015] The co-occurrence frequency is calculated based on the co-occurrence times to generate a co-occurrence frequency matrix between the characters. The calculation logic of the co-occurrence frequency is: P(w i ,w j ) is the word w i and the word w j The co-occurrence frequency of k is the number of all words in the co-occurrence matrix with word w. i Index of co-occurring words.

[0016] The present invention is further configured to assign a value to each child node based on the co-occurrence frequency of the word corresponding to the child node and the root node, and the sum of all child nodes is one. The calculation logic of the value assigned to each child node is: Among them, v(w j ) is the word w j The value assigned to the child node, where N is the number of child nodes.

[0017] The present invention is further configured such that the construction logic of the semantic connection tree includes:

[0018] Extracting a set of child nodes of the current node from the semantic connection tree, starting from the first child node, taking the current child node as the root node, creating a new semantic connection tree for the current child node, and repeatedly completing the set of child nodes;

[0019] For each new semantic connection tree, assign a value to each child node based on the co-occurrence frequency of the word corresponding to the child node and its parent node, and the sum of all child nodes is one;

[0020] Repeat until the preset number of layers is reached to obtain a complete semantic connection tree.

[0021] The present invention is further configured such that, in the keyword detection module, keywords are generated by presetting, and a semantic connection tree of the keywords is generated based on the presetting keywords;

[0022] When matching and analyzing the semantic connection tree of the document to be analyzed and the keyword, and obtaining the public opinion keyword set of the document to be analyzed, it includes:

[0023] Perform word segmentation on the document to be analyzed to obtain the word sequence of the document to be analyzed;

[0024] Mapping the first character of the segmented word in the word sequence to the root node of the semantic connection tree of the keyword, and mapping the remaining characters in the word sequence to the corresponding nodes in the semantic connection tree of the keyword;

[0025] Calculate the matching degree between the segmented words in the word sequence of the document to be analyzed and the semantic connection tree of the keywords, and input all words with matching degree greater than a preset matching degree threshold into the keyword candidate set;

[0026] Calculate the semantic influence factor of each segmented word in the keyword candidate set, and select the top M words with the highest semantic influence factors as the final public opinion keyword set.

[0027] The present invention is further configured such that the logic of word segmentation processing is as follows: formatting the document to be analyzed, removing punctuation and paragraphs, obtaining a text set of the document to be analyzed, obtaining the pinyin of each word, and matching it with the pinyin of the root node of the semantic connection tree of the keyword; when a match is found, starting from the matched word, obtaining the same number of words as the preset number of layers, setting them as segmentation words, and collecting all the segmentation words to obtain a word sequence of the document to be analyzed.

[0028] The present invention is further configured such that the calculation logic of the matching degree between the segmented words in the word sequence of the document to be analyzed and the semantic connection tree of the keywords includes:

[0029] Obtain the number of repeated words in the semantic connection tree between the segmented words and keywords in the word sequence, and calculate the repetition similarity;

[0030] Calculate the co-occurrence similarity based on the values ​​assigned to the repeated words in the semantic connection tree;

[0031] Calculate the hierarchical similarity based on the hierarchy of repeated words in the semantic connection tree;

[0032] The matching degree of the semantic connection tree between the segmented words and the keywords is calculated based on repetition similarity, co-occurrence similarity and hierarchical similarity.

[0033] The present invention is further configured such that the calculation logic of the repeated similarity is: Among them, R s is the repeat similarity, N r is the number of repeated words in the segmented words and the semantic connection tree, N t is the total number of words in the segmented words; the calculation logic of the co-occurrence similarity is: Among them, C s is the co-occurrence similarity, v(w r ) is the distribution value of the rth word in the semantic connection tree that is repeated in the segmented word and the semantic connection tree in the keyword; the calculation logic of the hierarchical similarity is: Among them, L s is the hierarchical similarity, D r is the absolute difference between the number of layers of the r-th word repeated in the segmentation word and the semantic connection tree; the calculation logic of the matching degree between the segmentation word and the semantic connection tree of the keyword is: M d =α*R s +β*C s +γ*L s , where M d is the matching degree of the semantic connection tree between the segmented words and the keywords, α, β and γ are weight coefficients of similarity, all greater than zero, and α+β+γ=1.

[0034] The present invention is further configured such that the calculation logic of the semantic impact factor of each segmented word in the keyword candidate set is: Among them, F si is the semantic influence factor of each word in the keyword candidate set, f(S(word) i ) is the frequency of the lth segmentation word in the word sequence appearing in the document to be analyzed, and L is the number of segmentation words in the word sequence.

[0035] The present invention provides a public opinion detection and analysis system, comprising: a word sense connection module, for constructing a word sense connection tree of characters, wherein the root node is any character, the child nodes are characters related to the root node, and the child nodes are assigned values ​​according to the co-occurrence frequency, so that the sum of the values ​​of all child nodes is one; a semantic connection module, for constructing a word sense connection tree of each child node with the child node of the word sense connection tree as the root node, until a preset number of layers is reached to obtain a semantic connection tree; a keyword detection module, for matching and analyzing the semantic connection trees of the document to be analyzed and the keyword, to obtain a set of public opinion keywords for the document to be analyzed, and the beneficial effects produced include:

[0036] 1. Improved emotion recognition accuracy: By building a word association tree, the system can analyze the dynamic semantic associations between words, allowing the system to dynamically adjust the emotional weight of words based on the context. This dynamic analysis model based on semantic associations significantly improves emotion recognition accuracy in complex contexts, especially when processing text containing polysemous words or homophones. It can more accurately judge the user's emotional state than traditional static dictionaries or emotion labeling methods.

[0037] 2. Enhance the dynamic adaptability of sentiment analysis: By constructing a co-occurrence frequency matrix and a keyword detection module, the keywords of a document can be effectively analyzed, allowing the system of the present invention to dynamically adapt to keyword changes in scenarios such as public opinion monitoring and long-term user interaction analysis;

[0038] 3. Improve the robustness and stability of emotion recognition. By introducing multi-level similarity and matching calculations, the system of the present invention demonstrates greater robustness and stability when processing texts with different contexts and emotional expressions. Whether identifying keywords in a single text or tracking keyword trends over time, the present invention can provide consistent and reliable emotion analysis results, meeting diverse emotion recognition needs.

[0039] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without inventive efforts. In the drawings:

[0041] Figure 1 The figure is a structural diagram of a public opinion detection and analysis system showing an exemplary embodiment of the present invention. DETAILED DESCRIPTION

[0042] The following describes the embodiments of the present invention with reference to the accompanying drawings and preferred embodiments. Those skilled in the art will readily appreciate the other advantages and benefits of the present invention from the disclosure herein. The present invention may also be implemented or applied through various other specific embodiments, and the various details in this specification may be modified or altered based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are intended only to illustrate the present invention and are not intended to limit the scope of protection of the present invention.

[0043] It should be noted that the illustrations provided in the following embodiments are merely schematic illustrations of the basic concept of the present invention. Therefore, the illustrations only show components related to the present invention and are not drawn according to the number, shape, and size of components in actual implementation. In actual implementation, the type, quantity, and proportion of each component may be changed arbitrarily, and the component layout may also be more complex.

[0044] In the following description, numerous details are discussed to provide a more thorough explanation of the embodiments of the present invention. However, it will be apparent to those skilled in the art that the embodiments of the present invention may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring the embodiments of the present invention.

[0045] A public opinion detection and analysis system, such as Figure 1 As shown, including:

[0046] The word sense association module is used to construct a word sense association tree for each word, where the root node is any word and the child nodes are words related to the root node. The child nodes are assigned values ​​based on the co-occurrence frequency so that the sum of the values ​​of all child nodes is one.

[0047] A semantic connection module is used to construct a word sense connection tree for each child node based on the child node of the word sense connection tree as a root node, until a preset number of layers is reached to obtain a semantic connection tree;

[0048] The keyword detection module is used to match and analyze the semantic connection tree of the document to be analyzed and the keywords to obtain the public opinion keyword set of the document to be analyzed.

[0049] The present invention is further configured such that the semantic connection module comprises: a data processing unit, a node construction unit and a node assignment unit;

[0050] The data processing unit is used to extract characters from historical texts and count their occurrence frequencies, generate a character frequency table, calculate the co-occurrence frequencies between characters, and generate a co-occurrence frequency matrix. Specifically, the historical text is formatted, including removing punctuation marks, spaces, and line breaks to ensure that only valid Chinese characters are retained, and the text is converted into a unified character encoding (such as UTF-8) to ensure consistency in subsequent processing. The processed text characters are traversed one by one, the Chinese characters therein are extracted, and other non-Chinese characters are ignored. The extracted Chinese characters are stored in a list or set for frequency statistics. A dictionary or hash table data structure is used to record the number of times each Chinese character appears. For each Chinese character, if the Chinese character already exists in the dictionary, its count value is increased by one. If the Chinese character does not exist, the Chinese character is added to the dictionary and its count value is initialized to one. The character frequency table is sorted in descending order according to the frequency of occurrence.

[0051] A node construction unit is used to select the corresponding word as the root node according to the descending order of the word frequency table, extract the words connected to the root node from the co-occurrence matrix, and fill the child nodes according to the order of co-occurrence frequency;

[0052] The node assignment unit is used to assign a value to each child node according to the co-occurrence frequency of the word corresponding to the child node and the root node, and the sum of all child nodes is one.

[0053] The present invention is further configured to calculate the co-occurrence frequencies between characters and generate a co-occurrence frequency matrix, including:

[0054] Create a word pair co-occurrence matrix C, where the rows and columns of the word pair co-occurrence matrix are all words that appear in the historical text;

[0055] Set the window size of the word pair and perform co-occurrence calculation for each word w i Traverse and find other words w within the window range j , and update the value of the corresponding position in the co-occurrence matrix C. The update logic is: C ′ [w i ,w j ]=C[w i ,w j ]+1,C ′ [w i ,w j ] is the number of times after updating, C[w i ,w j] is other characters within the window range j The initial value is zero. Whenever other characters w appear within the window range, j , the value is increased by one, and after the traversal is completed, the statistical word w i and the word w j Specifically, the window range is preferably 2-10. There is no limit on the specific value here, but a window exceeding 10 will exponentially increase the computational complexity when generating a semantic connection tree.

[0056] The co-occurrence frequency is calculated based on the co-occurrence times to generate a co-occurrence frequency matrix between the characters. The calculation logic of the co-occurrence frequency is: P(w i ,w j ) is the word w i and the word w j The co-occurrence frequency of k is the number of all words in the co-occurrence matrix with word w. i The index of the co-occurring characters. Specifically, the above calculation logic generates a character pair co-occurrence frequency matrix by counting the number of co-occurrences between characters, quantifying the co-occurrence relationship between characters. The co-occurrence frequency matrix reflects the probability of each character appearing with other characters in the text at the same time;

[0057] The present invention is further configured to assign a value to each child node based on the co-occurrence frequency of the word corresponding to the child node and the root node, and the sum of all child nodes is one. The calculation logic of the value assigned to each child node is: Among them, v(w j ) is the word w j The value assigned to a child node, where N is the number of child nodes. Specifically, the calculation logic above assigns a weight (i.e., assigned value) to each child node by calculating the co-occurrence frequency of the child node with the root node, reflecting the child node's importance in the relevance tree. The assigned value for each child node is calculated based on its co-occurrence frequency with the root node, ensuring that the sum of the assigned values ​​of all child nodes is 1. This quantifies the influence or relevance of a child node relative to the root node.

[0058] The present invention is further configured such that the construction logic of the semantic connection tree includes:

[0059] Extracting a set of child nodes of the current node from the semantic connection tree, starting from the first child node, taking the current child node as the root node, creating a new semantic connection tree for the current child node, and repeatedly completing the set of child nodes;

[0060] For each new semantic connection tree, assign a value to each child node based on the co-occurrence frequency of the word corresponding to the child node and its parent node, and the sum of all child nodes is one;

[0061] Repeat until the preset number of layers is reached to obtain a complete semantic connection tree. Specifically, the semantic connection tree is recursively constructed to reflect the hierarchical relationship between words in a layer-by-layer expansion manner. Specifically, starting from the root node in the word meaning connection tree, child nodes are extracted layer by layer and weights are calculated based on co-occurrence frequency to form a semantic connection tree with a hierarchical structure. After reaching the preset number of layers, a complete semantic connection tree is obtained. The assigned value of each child node is based on its association with its parent node, and the sum of the assigned values ​​of each layer is guaranteed to be 1.

[0062] The present invention is further configured such that, in the keyword detection module, keywords are generated by presetting, and a semantic connection tree of the keywords is generated based on the pre-set keywords; specifically, in the keyword detection module, a semantic connection tree is generated by a preset keyword set. The semantic connection tree of keywords is used as a reference structure when detecting public opinion, so as to identify words or phrases semantically associated with the preset keywords in the document to be analyzed. The pre-set keyword semantic connection tree helps to improve the efficiency and accuracy of detection, concentrates the calculation in the semantic space related to the keywords, and avoids the calculation of irrelevant words in the entire text;

[0063] When matching and analyzing the semantic connection tree of the document to be analyzed and the keyword, and obtaining the public opinion keyword set of the document to be analyzed, it includes:

[0064] The document to be analyzed is segmented to obtain a word sequence of the document to be analyzed; the present invention is further configured such that the logic of the word segmentation processing is: formatting the document to be analyzed, removing punctuation and paragraphs, obtaining a text set of the document to be analyzed, obtaining the pinyin of each word, and matching it with the pinyin of the root node of the semantic connection tree of the keyword; when a match is found, starting from the matched word, obtaining the same number of words as the preset number of layers, setting them as segmented words, and collecting all the segmented words to obtain a word sequence of the document to be analyzed.

[0065] The first character of the segmented word in the word sequence is mapped to the root node of the semantic connection tree of the keyword, and the remaining characters in the word sequence are mapped to the corresponding nodes in the semantic connection tree of the keyword. Specifically, the word sequence in the document to be analyzed is mapped word by word with the semantic connection tree of the keyword to determine the relevance of each segmented word in the word sequence with the keyword. The mapping can help the system quickly determine whether the word has a semantic association with the preset keyword, thereby improving the efficiency and accuracy of keyword detection.

[0066] Calculate the matching degree of the semantic connection tree of the segmented words in the word sequence of the document to be analyzed and the keywords, and input all words with matching degrees greater than a preset matching degree threshold into the keyword candidate set; the present invention is further configured such that the calculation logic of the matching degree of the semantic connection tree of the segmented words in the word sequence of the document to be analyzed and the keywords includes: obtaining the number of repeated words in the semantic connection tree of the segmented words in the word sequence and the keywords, and calculating the repetition similarity; calculating the co-occurrence similarity according to the assigned values ​​of the repeated words in the semantic connection tree; calculating the hierarchical similarity according to the hierarchical level of the repeated words in the semantic connection tree; calculating the matching degree of the semantic connection tree of the segmented words and the keywords according to the repetition similarity, co-occurrence similarity and hierarchical similarity. The present invention is further configured such that the calculation logic of the repetition similarity is: Among them, R s is the repeat similarity, N r is the number of repeated words in the segmented words and the semantic connection tree, N t is the total number of words in the segmented words; the calculation logic of the co-occurrence similarity is: Among them, C s is the co-occurrence similarity, v(w r ) is the distribution value of the rth word in the semantic connection tree that is repeated in the segmented word and the semantic connection tree in the keyword; the calculation logic of the hierarchical similarity is: Among them, L s is the hierarchical similarity, D r is the absolute difference between the number of layers of the r-th word repeated in the segmentation word and the semantic connection tree; the calculation logic of the matching degree between the segmentation word and the semantic connection tree of the keyword is: M d =α*R s +β*C s +γ*L s , where M dis the matching degree of the semantic connection tree between the segmentation words and the keywords, α, β and γ are the weight coefficients of similarity, all greater than zero, and α+β+γ=1. Specifically, the above logic calculates the matching degree of the segmentation words in the word sequence in the document to be analyzed and the semantic connection tree of the keywords. By quantifying multiple dimensions of matching degree, including repetition similarity, co-occurrence similarity and hierarchical similarity, the relevance between the segmentation words and keywords in the text to be analyzed is judged. The specific steps include calculating repetition similarity, co-occurrence similarity, hierarchical similarity, and combining these similarities to calculate the final matching degree; repetition similarity measures the number of overlapping words in the semantic connection tree between the segmentation words in the word sequence to be analyzed and the keywords, and is used to quantify the degree of literal overlap between the segmentation words and the keywords' semantic connection tree. The more overlap, the stronger the semantic association between the two words may be; co-occurrence similarity measures the sum of the co-occurrence frequencies of the subnodes of the segmentation words and the keywords in the semantic connection tree, and is used to quantify the co-occurrence relationship between the segmentation words and the keywords in the semantic connection tree. A higher assigned value indicates a more important node, and a larger sum of co-occurrence similarities indicates a higher relevance between two words in the semantic connection tree. Hierarchical similarity measures the hierarchical proximity of the corresponding subnodes in the semantic connection tree of the segmented word and the keyword, and is used to assess the relative position of the two words in the semantic connection tree. The closer the hierarchies, the stronger their semantic relationship is likely to be. By combining these three similarity metrics with a weighted sum, we can obtain a quantitative matching index, which is used to ultimately determine the semantic relevance between the segmented word and the keyword.

[0067] Calculate the semantic influence factor of each segmented word in the keyword candidate set, and select the top M words with the highest semantic influence factors as the final public opinion keyword set. The present invention is further configured such that the calculation logic of the semantic influence factor of each segmented word in the keyword candidate set is: Among them, F si is the semantic influence factor of each word in the keyword candidate set, f(S(word) i ) is the frequency of the lth segmentation word in the word sequence in the document to be analyzed, and L is the number of segmentation words in the word sequence. Specifically, the above calculation logic evaluates the importance of the word in the text by calculating the semantic influence factor of each segmentation word in the keyword candidate set. The semantic influence factor comprehensively considers the frequency of occurrence and matching of the word in the text to be analyzed, and weights these two factors to obtain a comprehensive indicator reflecting the importance of the word. According to the size of the semantic influence factor, the top M words with the strongest semantic relevance can be selected as the final public opinion keyword set; through the calculation of the semantic influence factor, the frequency of occurrence of the word can be combined with the semantic matching degree to more comprehensively evaluate the importance of the word in the text, ensuring that the selected keywords are highly semantically relevant to the topic.

[0068] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer program are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state drive.

[0069] It should be understood that the term "and / or" as used herein simply describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A alone, A and B together, or B alone. A and B can be singular or plural. Furthermore, the character " / " as used herein generally indicates an "or" relationship between the associated objects, but it may also indicate an "and / or" relationship. For specific understanding, please refer to the context.

[0070] In this application, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or plural.

[0071] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0072] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0073] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0074] In the several embodiments provided in this application, it should be understood that the disclosed system can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0075] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0076] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0077] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0078] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A public opinion detection and analysis system, characterized in that: include: The word sense association module is used to construct a word sense association tree for each word, where the root node is any word and the child nodes are words related to the root node. The child nodes are assigned values ​​based on the co-occurrence frequency so that the sum of the values ​​of all child nodes is one. A semantic connection module is used to construct a word sense connection tree for each child node based on the child node of the word sense connection tree as a root node, until a preset number of layers is reached to obtain a semantic connection tree; The semantic connection module includes a data processing unit, which is used to extract words from historical texts and count the occurrence frequencies, generate a word frequency table, calculate the co-occurrence frequencies between words, and generate a co-occurrence frequency matrix; generating a co-occurrence frequency matrix includes: creating a word pair co-occurrence matrix The rows and columns of the word pair co-occurrence matrix are all words that appear in the historical text; set the window size of the word pair, perform co-occurrence calculation, and calculate the co-occurrence of each word. Traverse and find other words within the window range , and in the co-occurrence matrix Update the value of the corresponding position in the , the update logic is: , is the number of updates, For other characters within the window The initial value is zero. Whenever other characters appear within the window range, , the value is increased by one, and after the traversal is completed, the statistical word and Calculate the co-occurrence frequency based on the co-occurrence number to generate a co-occurrence frequency matrix between words. The calculation logic of the co-occurrence frequency is: , For the word and The co-occurrence frequency of In the co-occurrence matrix, all Index of co-occurring words; The keyword detection module is used to perform matching analysis on the semantic connection tree of the document to be analyzed and the keyword to obtain the public opinion keyword set of the document to be analyzed, including calculating the matching degree between the segmentation words in the word sequence of the document to be analyzed and the semantic connection tree of the keyword; the calculation logic of the matching degree of the semantic connection tree includes: obtaining the number of repeated words in the segmentation words in the word sequence and the semantic connection tree of the keyword, and calculating the repetition similarity; calculating the co-occurrence similarity according to the value assigned to the repeated words in the semantic connection tree; calculating the hierarchical similarity according to the hierarchy of the repeated words in the semantic connection tree; calculating the matching degree of the semantic connection tree of the segmentation words and the keyword based on the repetition similarity, co-occurrence similarity and hierarchical similarity.

2. A public opinion detection and analysis system according to claim 1, characterized in that: The semantic connection module also includes: a node construction unit and a node assignment unit; A node construction unit is used to select the corresponding word as the root node according to the descending order of the word frequency table, extract the words connected to the root node from the co-occurrence matrix, and fill the child nodes according to the order of co-occurrence frequency; The node assignment unit is used to assign a value to each child node according to the co-occurrence frequency of the word corresponding to the child node and the root node, and the sum of all child nodes is one.

3. A public opinion detection and analysis system according to claim 1, characterized in that: Assign a value to each child node based on the co-occurrence frequency of the word corresponding to the child node and the root node, and the sum of all child nodes is one. The calculation logic for assigning a value to each child node is: ,in, For the word The assigned value on the child node, The number of child nodes.

4. A public opinion detection and analysis system according to claim 1, characterized in that: The logic for building the semantic connection tree includes: Extracting a set of child nodes of the current node from the semantic connection tree, starting from the first child node, taking the current child node as the root node, creating a new semantic connection tree for the current child node, and repeating to complete the set of child nodes; For each new semantic connection tree, assign a value to each child node based on the co-occurrence frequency of the word corresponding to the child node and its parent node, and the sum of all child nodes is one; Repeat until the preset number of layers is reached to obtain a complete semantic connection tree.

5. A public opinion detection and analysis system according to claim 1, characterized in that: In the keyword detection module, keywords are generated by presetting, and a semantic connection tree of keywords is generated based on the pre-set keywords; When matching and analyzing the semantic connection tree of the document to be analyzed and the keyword, and obtaining the public opinion keyword set of the document to be analyzed, it includes: Perform word segmentation on the document to be analyzed to obtain the word sequence of the document to be analyzed; Mapping the first character of the segmented word in the word sequence to the root node of the semantic connection tree of the keyword, and mapping the remaining characters in the word sequence to the corresponding nodes in the semantic connection tree of the keyword; Based on the matching degree of the semantic connection tree, all words with matching degrees greater than a preset matching degree threshold are input into the keyword candidate set; Calculate the semantic impact factor of each word in the keyword candidate set and select the top word with the highest semantic impact factor. words as the final public opinion keyword set.

6. A public opinion detection and analysis system according to claim 5, characterized in that: The logic of word segmentation processing is: format the document to be analyzed, remove punctuation and paragraphs, obtain the text set of the document to be analyzed, obtain the pinyin of each word, and match it with the pinyin of the root node of the semantic connection tree of the keyword. When a match is found, start from the matched word, obtain the same number of words as the preset number of layers, set them as word segmentation words, and collect all the word segmentation words to obtain the word sequence of the document to be analyzed.

7. A public opinion detection and analysis system according to claim 1, characterized in that: The calculation logic of the repeated similarity is: ,in, is the repeat similarity, is the number of repeated words in the segmented words and the semantic connection tree, is the total number of words in the segmented words; the calculation logic of the co-occurrence similarity is: ,in, is the co-occurrence similarity, The first repeated word in the segmentation word and the semantic connection tree The value assigned to a word in the semantic connection tree of the keyword; the calculation logic of the hierarchical similarity is: ,in, is the hierarchical similarity, The first repeated word in the segmentation word and the semantic connection tree The absolute difference in the number of layers of the word; the calculation logic of the matching degree of the semantic connection tree of the segmented words and the keywords is: ,in, is the matching degree of the semantic connection tree between the segmented words and the keywords, 、 and is the weight coefficient of similarity, both are greater than zero, and .

8. A public opinion detection and analysis system according to claim 7, characterized in that: The calculation logic of the semantic impact factor of each segmented word in the keyword candidate set is: ,in, is the semantic influence factor of each segmented word in the keyword candidate set, The first word in the sequence The frequency of occurrence of the word in the document to be analyzed, is the number of segmented words in the word sequence.

Images