A conflict and dispute event early warning method, system, program product and storage medium

Abstract

The present application provides a conflict and dispute incident early warning method, system, program product and storage medium, and relates to the field of information and communication technology specifically suitable for administrative, commercial, financial, management or supervisory purposes. The first feature data related to the subject information is extracted in the event structure tree using feature extraction rules, so that the key features of the event can be clearly presented. For personnel data, after constructing the personnel structure tree by word segmentation and part-of-speech tagging, the extracted words are subjected to sentiment tendency analysis, and negative sentiment words are marked, so as to facilitate subsequent judgment of the conflict and dispute incidents. The subject information and personnel are clustered according to the first feature data and the second feature data. The personnel carry the marked words during the clustering process, so that the event is closely associated with the relevant personnel, and the deep integration of data is realized. The problem of data confusion and mismatch is solved, so as to judge the real situation and development trend of the conflict and dispute incidents and improve the accuracy of the early warning.

Description

Technical Field

[0001] The present application relates to the field of information and communication technology specifically suitable for administrative, commercial, financial, management or supervisory purposes, and in particular to a conflict and dispute event early warning method, system, program product and storage medium. Background Art

[0002] In the relevant technical field, due to the extremely complex data sources, the data collection methods, storage formats and data quality of each data source are uneven. This leads to the fact that when facing conflicts and disputes, even if there is a large amount of relevant data, it is impossible to effectively extract key information due to the confusion and mismatch of the data, and it is difficult to accurately judge the true situation and development trend of the conflicts and disputes, making the early warning of conflicts and disputes inaccurate. Summary of the invention

[0003] The present application provides a conflict and dispute event early warning method, system, program product and storage medium, which are used to solve data confusion and mismatch problems and improve the accuracy of early warning.

[0004] In the first aspect, the present application provides a conflict and dispute event early warning method, including: extracting data information from a data source, dividing the data information into different types according to a pre-set data storage method and business data relationship, including event data of type event and personnel data of type personnel; the data source includes a demand collection source, a grid member reporting source, a technical patrol and alarm collection source, a superior and department data source, and an offline data source; segmenting the event data and personnel data to obtain words, and tagging the words with parts of speech, and constructing an event structure tree and a personnel structure tree respectively according to the results of the part of speech tagging; extracting subject information from the event structure tree; and applying feature extraction rules to the event structure tree. The tree extracts the first feature data related to the subject information; the subject information is the relevant expression reflecting the event content or topic; the sentiment tendency analysis is performed on the words extracted from the personnel structure tree; the words judged to have negative sentiments are marked, and the second feature data related to the personnel are extracted in the personnel structure tree using the feature extraction rules; the subject information and the personnel are clustered according to the first feature data and the second feature data, and the personnel will carry the marked words in the clustering process; the threshold of the subject information pre-set in the threshold configuration file is read, and when the words carrying the mark exceed the corresponding threshold, the event to which the subject information belongs is judged to be a conflict and dispute event, and the corresponding personnel after clustering are output together.

[0005] By adopting the above technical scheme, data information is extracted from multiple types of data sources, and it is divided into event data and personnel data, so as to realize the preliminary classification and sorting of data and avoid the data being disordered; then the event data is segmented to obtain words and the event structure tree is constructed after part-of-speech tagging, and the subject information is extracted from the event structure tree. This step greatly reduces the interference of irrelevant data and accurately locates the core content of the event. Then, the feature extraction rules are used to extract the first feature data related to the subject information in the event structure tree, so that the key features of the event can be clearly presented. For the personnel data, after the personnel structure tree is constructed by word segmentation and part-of-speech tagging, the sentiment tendency analysis is performed on the extracted words, and the negative sentiment words are marked, so as to facilitate the subsequent judgment of the conflict and dispute events, and then the feature extraction rules are used to extract the second feature data related to the personnel, and the key information in the personnel data is deeply excavated and the initial connection is established with the event. According to the first feature data and the second feature data, the subject information and the personnel are clustered, and the personnel carry the marked words in the clustering process, so that the event is closely associated with the relevant personnel, and the deep integration of data is realized. Finally, the threshold of the subject information in the threshold configuration file is read. When the words carrying the mark exceed the corresponding threshold, the event to which the subject information belongs is accurately determined to be a conflict and dispute event, and the corresponding personnel after clustering processing are output, which solves the problem of data confusion and mismatch, thereby judging the real situation and development trend of the conflict and dispute event and improving the accuracy of the early warning.

[0006] In combination with some embodiments of the first aspect, in some embodiments, the steps of dividing data information into different types are performed based on pre-set data storage methods and business data relationships, specifically including: extracting keywords in the current data information, and dividing the data information into different types according to the types of keywords; the current data information is any data information; after classification, adding a source identification field to the current data information, and the source identification field is assigned a corresponding identification value according to the data source from which it comes; based on the source identification field, the business data relationship is constructed with the current data information as the main node and the data information with flow and association relationships between the data information as the secondary node; adding the classification of the current data information to the data information with flow and association relationships between the data information; and storing different types of data information in the corresponding database.

[0007] By adopting the above technical solution, keywords in data information are extracted, classified according to their types and source identification fields are added, and then business data relationships are constructed. In this process, the classification relationship of the main node is assigned to the secondary node, so that the secondary node adds a classification relationship. The originally isolated or vaguely related data are closely connected. In this way, the data dimensions are richer and the connectivity between data is significantly enhanced. This effectively avoids the one-sidedness of information caused by the lack of data association, makes the basis for early warning judgment more accurate, and improves the accuracy of early warning of conflict and dispute events.

[0008] In combination with some embodiments of the first aspect, in some embodiments, the step of storing different types of data information in corresponding databases specifically includes: for data information with multiple types, randomly selecting a corresponding database for storage, and adding index identifiers tracing back to other corresponding databases in the storage records.

[0009] By adopting the above technical solution, data with multiple types of information can be randomly selected for storage and index identification can be added. Random storage can balance the load of each database to avoid excessive pressure on a single database affecting overall performance. Index identification provides a convenient path for data tracing. When comprehensive data analysis is required, even if the data is stored in a dispersed manner, it can be quickly associated with related data in other databases based on the index.

[0010] In combination with some embodiments of the first aspect, in some embodiments, the step of extracting topic information from the event structure tree specifically includes: extracting topic information from the event structure tree using a topic information extraction model; the training process of the topic information extraction model is: presetting training topic information and a training corpus, the training corpus including training corpus of categories related to the training topic information and training corpus of categories unrelated to the training topic information; traversing the training corpus once, identifying each word with the help of syntactic analysis, and generating corresponding extraction rules for it; continuing to traverse the training corpus, generating statistical data for each extraction rule, and scoring the extraction rules according to the statistical data, wherein the extraction rule is positively correlated with the number of times it appears in the training corpus of categories related to the training topic information and negatively correlated with the number of times it appears in the training corpus of categories unrelated to the training topic information; retaining the extraction rules whose scoring results exceed the scoring threshold.

[0011] By adopting the above technical solution, by presetting the training topic information and the training corpus containing related and unrelated categories, the corpus is traversed to generate extraction rules and the rules are scored and screened according to the number of appearances in different categories of corpus. Rules that frequently appear in related categories but rarely appear in unrelated categories are retained, which makes it more accurate to focus on content related to conflict and dispute events when extracting topic information, and effectively eliminates interference from irrelevant information. It can accurately extract the core theme of the event and improve the accuracy and effectiveness of event information processing in the entire early warning process.

[0012] In combination with some embodiments of the first aspect, in some embodiments, the step of scoring the extraction rules according to statistical data specifically includes: the score of the extraction rule is the number of times it appears in the training corpus of the category related to the training topic information divided by the number of times it appears in the training corpus of the category unrelated to the training topic information; after obtaining all the scoring results, the scoring results are binarized.

[0013] By adopting the above technical solution, the ratio of the number of occurrences in the training corpus of the relevant and irrelevant categories of the training topic information is used as the extraction rule score. This calculation method intuitively reflects the strength of the relevance between the rule and the topic information. After the data is binarized, the score result is further simplified to highlight the usability of the rule.

[0014] In combination with some embodiments of the first aspect, in some embodiments, after extracting data information from a data source and dividing the data information into different types of steps based on a pre-set data storage method and business data relationship, the method also includes: storing data information whose time is lower than a time threshold in a memory database, and storing data information whose time is not lower than the time threshold in a distributed file system.

[0015] By adopting the above technical solution, data information with a time lower than the threshold is stored in the memory database. Due to its fast reading and writing speed, data generated recently and possibly closely related to current conflicts and disputes can be quickly called for analysis, thus speeding up the data processing rhythm. Data with a time not lower than the threshold is stored in the distributed file system, which ensures long-term data storage while not occupying too much memory resources, and the distributed architecture is conducive to large-scale data management.

[0016] In combination with some embodiments of the first aspect, in some embodiments, after the step of storing data information with a time lower than a time threshold in a memory database and storing data information with a time not lower than a time threshold in a distributed file system, the method further includes: establishing a data index for the data information.

[0017] By adopting the above technical solution, a data index is established for the data information. When querying data, there is no need to traverse the entire database. The required data can be quickly located through the index, whether it is event data or personnel data, it can be quickly retrieved and called.

[0018] In a second aspect, the present application provides a conflict and dispute event warning system, which includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and one or more processors call the computer instructions to enable the conflict and dispute event warning system to execute the method described in the first aspect and any possible implementation method of the first aspect.

[0019] In a third aspect, the present application provides a computer program product comprising instructions, which, when executed on a conflict and dispute event early warning system, enables the conflict and dispute event early warning system to execute the method described in the first aspect and any possible implementation of the first aspect.

[0020] In a fourth aspect, the present application provides a computer-readable storage medium comprising instructions, which, when executed on a conflict and dispute event warning system, enables the conflict and dispute event warning system to execute the method described in the first aspect and any possible implementation of the first aspect.

[0021] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0022] 1. Extract data information from multiple data sources, divide it into event data and personnel data, realize the preliminary classification and sorting of data, and avoid data disorder; then segment the event data to obtain words and construct the event structure tree after part-of-speech tagging, and extract the subject information from the event structure tree. This step greatly reduces the interference of irrelevant data and accurately locates the core content of the event. Then use the feature extraction rules to extract the first feature data related to the subject information in the event structure tree, so that the key features of the event can be clearly presented. For personnel data, after segmenting and part-of-speech tagging to construct the personnel structure tree, perform sentiment analysis on the extracted words, mark negative sentiment words, and make subsequent judgments on conflict and dispute events. Then use the feature extraction rules to extract the second feature data related to personnel, deeply explore the key information in the personnel data and establish a preliminary connection with the event. According to the first feature data and the second feature data, the subject information and personnel are clustered. Personnel carry the marked words in the clustering process, so that the event is closely associated with the relevant personnel, and the deep integration of data is realized. Finally, the threshold of the subject information in the threshold configuration file is read. When the words carrying the mark exceed the corresponding threshold, the event to which the subject information belongs is accurately determined to be a conflict and dispute event, and the corresponding personnel after clustering processing are output, which solves the problem of data confusion and mismatch, thereby judging the real situation and development trend of the conflict and dispute event and improving the accuracy of the early warning.

[0023] 2. Extract keywords from data information, classify them by type, add source identification fields, and then build business data relationships. In this process, the classification relationship of the main node is assigned to the secondary node, so that the secondary node adds classification relationships. This allows the originally isolated or vaguely related data to be closely connected. This makes the data dimension richer and the connectivity between data significantly enhanced. This effectively avoids the one-sidedness of information caused by the lack of data association, makes the basis for early warning judgment more accurate, and improves the accuracy of early warning of conflict and dispute events.

[0024] 3. By presetting the training topic information and the training corpus containing related and unrelated categories, the corpus is traversed to generate extraction rules and the rules are scored and screened according to the number of occurrences in different categories of corpus. Rules that frequently appear in related categories but rarely appear in unrelated categories are retained, which makes it more accurate to focus on content related to conflict and dispute events when extracting topic information, and effectively eliminates interference from irrelevant information. It can accurately extract the core theme of the event and improve the accuracy and effectiveness of event information processing in the entire warning process. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a flow chart of a conflict and dispute event early warning method in an embodiment of the present application;

[0026] Figure 2 This is a specific flow chart of step S101 in the embodiment of the present application;

[0027] Figure 3 This is a specific flow chart of step S103 in the embodiment of the present application;

[0028] Figure 4 It is a schematic diagram of an exemplary hardware structure of the conflict and dispute incident early warning system in the embodiment of the present application. DETAILED DESCRIPTION

[0029] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to be used as limitations to the present application. As used in the specification and appended claims of the present application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to also include plural expressions, unless there is a clear indication to the contrary in the context. It should also be understood that the term "and / or" used in the present application refers to and includes any or all possible combinations of one or more listed items.

[0030] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more.

[0031] See also Figure 1 , Figure 1 This is a flow chart of a conflict and dispute event early warning method in an embodiment of the present application;

[0032] S101, extracting data information from a data source, and classifying the data information into different types according to a pre-set data storage method and business data relationship, including event data of event type and personnel data of personnel type; the data source includes demand collection source, grid member reporting source, technical patrol and alarm collection source, superior and department data source, and offline data source;

[0033] Among them, the data source represents the channel for obtaining data, and the business data relationship refers to a logical context formed by the generation, circulation and mutual connection of data in the business process. Through this relationship, the source and evolution process of the data can be clearly traced.

[0034] In some embodiments, after collecting data information from multiple data sources, the storage path and format may be determined according to factors such as data type, generation time, and region based on a pre-set data storage method, and the internal connection between data is analyzed based on the business data relationship. For example, if it is found that the reported data is associated with the appeal data, the relevant data information is divided into different types based on this association, where information related to specific events is classified as event data, and information related to the people involved is classified as personnel data.

[0035] It should be noted that some data is not text data, such as video data. In this case, it is necessary to use video-to-text related technologies to better integrate it into the overall data processing process. First of all, advanced video speech recognition technology will be used to accurately convert the voice information in the video, such as the recorded audio of the on-site personnel conversation, into text content with the help of high-precision speech recognition algorithms. In the conversion process, the voice timbre, intonation, speed and other characteristics will be intelligently analyzed to improve the recognition accuracy and reduce errors caused by environmental noise, accents and other factors.

[0036] For the parts of the video without voice, image recognition combined with natural language generation technology will be used.

[0037] After such video-to-text processing, the non-text data originally in the form of video can be classified and integrated in the form of text together with other text data according to the established data storage method and business data relationship. It can also be successfully involved in a series of analysis operations such as building a grammatical structure tree and extracting feature data, ensuring that all types of data can play their due role in social governance work such as early warning of conflicts and disputes.

[0038] In some specific embodiments, step S101 specifically includes:

[0039] See also Figure 2 , Figure 2This is a specific flow chart of step S101 in the embodiment of the present application;

[0040] S1011, extracting keywords from the current data information, and classifying the data information into different types according to the types of the keywords; the current data information is any data information;

[0041] Among them, keywords refer to words that can reflect the core content, key themes or important features of data information. Current data information refers to any individual data content collected from various data sources.

[0042] In some embodiments, the keyword extraction algorithm in natural language processing technology is first used to analyze the current data information, such as based on word frequency statistics, TF-IDF and other methods, to extract the keywords therein. Then, according to the types of these keywords, such as different types involving people, events, places, time, etc., the data information is divided into different categories.

[0043] In some specific embodiments, a custom keyword dictionary is established, which covers common keywords corresponding to various pre-set topics; the words in the current data information are matched with the keyword dictionary in turn, and the frequency of occurrence of the keywords corresponding to each topic is counted; the type of the data information is determined based on the frequency.

[0044] S1012, after classification, adding a source identification field to the current data information, where the source identification field is assigned a corresponding identification value according to the data source;

[0045] Among them, the source identification field refers to a kind of marking information used to clarify which specific data source the data information is obtained from, and different sources are reflected by assigning corresponding identification values.

[0046] In some embodiments, after the initial classification of data information is completed, this step needs to be performed in order to clearly trace the source of the data, understand the reliability and accuracy of the data, and control the data quality. For each piece of classified current data information, according to the actual data source, the pre-set identification value correspondence table is searched and the corresponding source identification field is added for it.

[0047] S1013. Based on the source identification field, the business data relationship is constructed with the current data information as the primary node and the data information with the flow and association relationship between the data information as the secondary node;

[0048] Among them, the primary node represents the data information currently being processed, and it is the core starting point for building this business data relationship; the secondary nodes are those data information that have a flow and association relationship with the current data information.

[0049] In some embodiments, the current data information with the added source identifier is used as the main node, and then through the data association analysis algorithm, such as rule-based association analysis (setting some association rules such as time sequence, same subject, etc.) or association rule mining algorithm in machine learning (such as Apriori algorithm, etc.), to find the data information with flow and association relationship, and use the found data information as the secondary node. Then, the business data relationship is constructed according to a certain data structure (such as tree structure, graph structure, etc.).

[0050] S1014, adding a classification of current data information for data information having a circulation and association relationship between data information;

[0051] Among them, the classification of the current data information is added for the data information with the circulation and association relationship between the data information. The classification of the current data information here refers to the category to which the data information belongs that has been determined in the previous step (S1011), such as event category, personnel category, etc. Its purpose is to allow the secondary node data information associated with the main node to also carry the classification attributes of the main node, further strengthening the association and classification consistency between the data.

[0052] In some embodiments, along the established business data relationship, find the data information that serves as a secondary node and has a flow and association relationship with the primary node, then check the classification corresponding to the primary node, and add the classification mark to the corresponding attribute position of the secondary node data information associated with it.

[0053] S1015. Store different types of data information into corresponding databases.

[0054] In some embodiments, after completing a series of operations such as previous data classification, adding source identification, building business data relationships, and adding classifications to secondary nodes, this step will be performed in order to make data storage more standardized and orderly, improve data storage efficiency and security, and facilitate subsequent rapid and accurate acquisition of required types of data according to different needs.

[0055] It can be seen that the keywords in the data information are extracted, classified according to their types and source identification fields are added, and then the business data relationship is constructed. In this process, the classification relationship of the main node is assigned to the secondary node, so that the secondary node adds a classification relationship. The originally isolated or vaguely related data are closely connected. In this way, the data dimension is richer and the connectivity between data is significantly enhanced. This effectively avoids the one-sidedness of information caused by the lack of data association, makes the basis for early warning judgment more accurate, and improves the accuracy of early warning of conflict and dispute events.

[0056] In some embodiments, S1015 is specifically as follows: for data information having multiple types, randomly select a corresponding database for storage, and add index identifiers tracing back to other corresponding databases in the storage record.

[0057] In some embodiments, in actual data storage scenarios, since the amount of data is often very large and the types are diverse, if all types of data are strictly stored in their corresponding unique databases according to fixed classification rules, some databases may be overloaded, while other database resources are idle, affecting the performance and efficiency of the entire data storage system. Therefore, for data information with multiple types, a corresponding database is randomly selected for storage. For example, there is a data containing both community environmental event information (belonging to the event category) and staff information (belonging to the personnel category) involved in the environmental event, and it is randomly selected to be stored as a whole in one of the event database or the personnel database. While storing the data, an index identifier that traces back to other corresponding databases will be added to the storage record of this data. For example, if this data is finally stored in the event database, then an index identifier pointing to the staff information related to the event in the personnel database is added to its storage record, so that when the community environmental event needs to be fully analyzed later, it involves querying the relevant personnel situation, and the relevant data in the personnel database can be quickly located by this index identifier, and the cross-database association query of the data can be realized, which provides strong support for more in-depth data analysis and comprehensive judgment of conflict and dispute events.

[0058] It can be seen that for multi-type data information, random databases are selected for storage and index identifiers are added. Random storage can balance the load of each database to avoid excessive pressure on a single database affecting overall performance. Index identifiers provide a convenient path for data tracing. When comprehensive data analysis is required, even if the data is stored in a dispersed manner, related data in other databases can be quickly associated based on the index.

[0059] S102, performing word segmentation on the event data and the personnel data to obtain words, and performing part-of-speech tagging on the words, and constructing an event structure tree and a personnel structure tree respectively according to the part-of-speech tagging results;

[0060] Among them, word segmentation refers to the process of segmenting continuous text data information into independent words according to semantic and grammatical rules. Part-of-speech tagging is used to indicate the grammatical category tagging of segmented words, such as nouns, verbs, adjectives, etc. The event structure tree refers to a tree structure built based on the words in the event data and the grammatical relationship between the words, which can intuitively show the semantic structure of the event.

[0061] In some embodiments, for event data, a professional word segmentation tool is used, such as a word segmentation module in a natural language processing library, to segment the text data into words, and then each word is tagged with a part-of-speech tagging algorithm. For example, for the event data "xx quarreled due to the tight parking space in the community", after word segmentation, words such as "xx", "due to", "community", "parking space", "tense", "occurred", and "quarrel" are obtained, and the part-of-speech tagging results are "noun", "preposition", "noun", "noun", "adjective", "verb", and "noun". Based on these part-of-speech tagging results, with the event subject as the root, such as "xx", an event structure tree is constructed according to semantic logic, and related words are connected with reasonable grammatical relationships. Similarly, word segmentation and part-of-speech tagging operations are also performed on personnel data to construct a personnel structure tree.

[0062] S103, extracting topic information from the event structure tree; and extracting first feature data related to the topic information in the event structure tree using feature extraction rules; the topic information is a relevant expression reflecting the event content or topic;

[0063] Among them, topic information refers to relevant statements that can summarize the core content or main topics of the event, and is the key information mined from the event structure tree.

[0064] In some embodiments, after the event structure tree is constructed, this step is performed in order to extract the key features and themes of the event for subsequent analysis and judgment. The theme information is extracted from the event structure tree using a pre-trained theme information extraction model. For example, for an event structure tree about a community environmental dispute, the extracted theme information may be "community environmental pollution problem". Then, the feature extraction rules are applied, which may be rules set according to specific vocabulary, grammatical structure or semantic relationship, to extract the first feature data related to the theme information in the event structure tree, such as time, location and other related data information.

[0065] In some specific embodiments, step S103 specifically includes:

[0066] See also Figure 3 , Figure 3 This is a specific flow chart of step S103 in the embodiment of the present application;

[0067] S1031, extracting topic information from the event structure tree using a topic information extraction model;

[0068] Among them, the topic information extraction model refers to a model built based on a specific algorithm and architecture, which is used to automatically extract relevant descriptive information that can summarize the core content of the event, key topics, etc. from the event structure tree, a data representation form with a certain grammatical and semantic structure.

[0069] S1032, the training process of the topic information extraction model is: presetting the training topic information and the training corpus, the training corpus including the training corpus of the category related to the training topic information and the training corpus of the category unrelated to the training topic information;

[0070] The preset training topic information refers to the approximate content and scope of the topic information that the expected model can extract from the event structure tree, which is preset before the training of the topic information extraction model, based on business needs and actual application scenarios. It is equivalent to a guiding direction, letting the model know what goal to learn towards. The training corpus refers to a large set of text data used to train the topic information extraction model. These text data cover a variety of content. The training corpus related to the training topic information is the text data that contains the content involved in the preset training topic information; and the training corpus of the irrelevant category refers to the data whose text content has nothing to do with the preset training topic information, which allows the model to better distinguish which content related to the topic information really needs to be extracted through comparative learning.

[0071] S1033, traversing the training corpus, identifying each word with the help of syntactic analysis, and generating corresponding extraction rules for it;

[0072] Among them, extraction rules refer to some criteria and methods formulated for each word based on the results of syntactic analysis to determine whether the word is related to the preset training topic information and how to extract relevant topic information from the text. For example, if a word has a specific syntactic structure (such as the object position after the production in the "because...produced..." structure) and often appears in texts related to the preset training topic information, corresponding extraction rules can be formulated to focus on and extract content related to the topic information containing the word.

[0073] In some embodiments, through a program or algorithm, starting from the first text in the training corpus, each text is processed in turn. When processing each text, a syntactic analysis tool (such as open source software or deep learning model based on dependency syntactic analysis, etc.) is used to analyze the sentences in the text one by one to determine the grammatical relationship between the words in each sentence. For example, for the sentence "xx quarreled with the property because of the noise problem in the community", after syntactic analysis, it can be concluded that "xx" is the subject, "occurred" is the predicate, "quarrel" is the object, and "because of the noise problem in the community" is the adverbial. Then, for each word, according to its grammatical position in the sentence, the collocation with other words, and the frequency of occurrence in the relevant texts in the entire training corpus, etc., the corresponding extraction rules are formulated. For example, for the word "quarrel", if it is found that it frequently appears in similar syntactic structures in many texts related to conflicts and disputes, a rule can be formulated as "when the word 'quarrel' appears in the predicate position and its subject and object involve people or groups, the text where the word is located is likely to be related to the preset training theme information, and the surrounding words can be focused on to extract the theme information".

[0074] S1034, continue to traverse the training corpus to generate statistical data for each extraction rule;

[0075] Among them, continuing to traverse the training corpus means that after completing a traversal to generate extraction rules, all text data in the training corpus are viewed and processed again in a certain order. The purpose is to further count the relevant performance of each rule in the entire training corpus based on the previously generated extraction rules, and collect more comprehensive data for subsequent evaluation and screening of the extraction rules.

[0076] In some embodiments, the traversal program is restarted, starting from the first text in the training corpus, and each text is checked again in turn. In this process, for each extraction rule that has been generated before, the rule is searched in each text to see if it appears. For example, if there is an extraction rule that is "when the word 'dispute' appears in the text and words indicating the location appear before and after it", it is searched in each text to see if this condition is met. Once it is met, the appearance of the rule in this text is recorded. By traversing the entire training corpus again in this way, statistics are calculated on how many times each extraction rule appears in the training corpus of the category related to the training subject information, how many times it appears in the training corpus of the unrelated category, and their distribution in different texts, etc., to provide a basis for subsequent rule scoring and screening.

[0077] S1035, scoring the extraction rules according to the statistical data, wherein the extraction rules are positively correlated with the number of times the training corpus appears in the category related to the training subject information, and negatively correlated with the number of times the training corpus appears in the category unrelated to the training subject information;

[0078] In some embodiments, the number of times the extraction rule appears in the relevant category training corpus and the number of times it appears in the irrelevant category training corpus are calculated according to a pre-set scoring formula. For example, the number of times the extraction rule appears in the relevant category training corpus divided by the number of times it appears in the irrelevant category training corpus is used as the basis for scoring. If the result is greater than 1 and the larger the value, the more representative the rule is in the relevant text and the higher the score; if the result is less than 1, it indicates that the rule may also be more common in irrelevant texts and the score is lower. Through such a scoring method, each extraction rule can be clearly quantitatively evaluated so that high-quality rules can be subsequently screened out for the topic information extraction model.

[0079] In some embodiments, step S1035 specifically includes:

[0080] S10351. The score of the extraction rule is the number of times the rule appears in the training corpus of the category related to the training subject information divided by the number of times the rule appears in the training corpus of the category unrelated to the training subject information;

[0081] In some embodiments, the score of the extraction rule is calculated by dividing the number of times the extraction rule appears in the training corpus of the training subject information-related category by the number of times it appears in the training corpus of the training subject information-unrelated category. Still taking the above example, according to this calculation method, the score of the extraction rule is 30÷10=3. This calculation method is very intuitive. If the score is greater than 1, it means that the rule appears more frequently in the training corpus of the relevant category, and is more likely to be an effective rule that is closely related to the subject information and can help accurately extract the subject information; if the score is less than 1, it means that the rule also appears more frequently in irrelevant texts, and may not be very useful in distinguishing the subject information, and the accuracy is poor. Therefore, through such a score calculation, a preliminary value judgment can be made quickly on each extraction rule.

[0082] S10352. After all the scoring results are obtained, the scoring results are binarized.

[0083] Data binarization refers to converting the scores of the extraction rules calculated previously into a form between 0 and 1 according to certain standards and rules, thereby narrowing the score gap while retaining the features.

[0084] It can be seen that the ratio of the number of occurrences in the training corpus of the relevant and irrelevant categories of the training topic information is used as the extraction rule score. This calculation method intuitively reflects the strength of the correlation between the rule and the topic information. After the data is binarized, the score result is further simplified to highlight the usability of the rule.

[0085] S1036. Retain the extraction rules whose scoring results exceed the scoring threshold.

[0086] In some embodiments, the remaining rules are those that perform well in the training corpus and have a high correlation with the preset training topic information.

[0087] It can be seen that by presetting the training topic information and the training corpus containing relevant and irrelevant categories, the corpus is traversed to generate extraction rules and the rules are scored and screened according to the number of appearances in different categories of corpus. Rules that frequently appear in relevant categories but rarely appear in irrelevant categories are retained, which makes it more accurate to focus on content related to conflict and dispute events when extracting topic information, and effectively eliminates interference from irrelevant information. It can accurately extract the core theme of the event and improve the accuracy and effectiveness of event information processing in the entire warning process.

[0088] S104, performing sentiment analysis on the words extracted from the personnel structure tree; marking the words determined to have negative sentiment, and extracting second feature data related to the personnel in the personnel structure tree using feature extraction rules;

[0089] Among them, sentiment analysis refers to the process of judging and classifying the sentiment attitudes contained in the words extracted from the personnel structure tree, such as positive, negative, neutral, etc. Negative sentiment words refer to words that are judged to have negative sentiment in sentiment analysis.

[0090] It should be noted that the feature extraction rules used in the personnel structure tree and the event structure tree are exactly the same, even though the data of the personnel structure tree and the event structure tree come from different sources.

[0091] However, as long as there is a communicative relationship between the person and the event, the extracted first feature data and the second feature data are the same in some key elements. For example, in the time dimension, whether it is the time when the event occurred or the time when the person participated in the event, it can be accurately extracted and kept consistent through the feature extraction rules; in terms of location, the location where the event occurred and the relevant location information of the person will also be accurately identified and correspond to each other.

[0092] In some embodiments, sentiment analysis is performed on the words extracted from the personnel structure tree, and the sentiment analysis algorithm, such as a dictionary-based sentiment analysis method, is used to match the words with the vocabulary in the sentiment dictionary to determine their sentiment tendency. Words that are determined to have negative sentiment are marked, for example, in the personnel data "Residents are dissatisfied with the services of the community property", "dissatisfaction" is marked as a negative sentiment word.

[0093] In some specific embodiments, a custom sentiment dictionary is constructed, common sentiment words are classified into positive, negative, and neutral, and different sentiment weights are set; words are extracted from the personnel structure tree and matched with the sentiment dictionary, sentiment tendency scores are calculated based on the matching results and weights, the sentiment tendency is determined and negative sentiment words are marked; according to the structural characteristics of the personnel structure tree, a path-based feature extraction method is adopted, such as extracting path information from a personnel node to a specific sentiment word node as the second feature data, which is not limited here.

[0094] S105, clustering the subject information and the personnel according to the first characteristic data and the second characteristic data, and the personnel will carry marked words during the clustering process;

[0095] In some embodiments, after obtaining the first feature data, the second feature data and the marked words, in order to effectively associate and integrate the event with the relevant personnel, the subject information and the personnel are clustered according to the information contained in the first feature data and the second feature data. In the clustering process, the personnel will carry the previously marked negative emotional words, so that the clustering results can better reflect the association between the event and the personnel and the emotional factors of the personnel in the event.

[0096] S106, reading the threshold of the subject information pre-set in the threshold configuration file, when the words carrying the mark exceed the corresponding threshold, determining that the event to which the subject information belongs is a conflict and dispute event, and outputting the corresponding personnel after clustering processing.

[0097] In some embodiments, the threshold value corresponding to the current topic information in the threshold configuration file is read. For example, for the topic information of community dispute events, the threshold value may be set to the number of people carrying negative emotional words reaching a certain proportion or the intensity of negative emotional words reaching a certain level. When the words carrying the mark after clustering processing exceed the corresponding threshold value, the event to which the topic information belongs is determined to be a conflict and dispute event, and the corresponding personnel after clustering processing are output together, so as to handle and respond to the conflict and dispute event in the future, such as arranging relevant personnel for mediation or taking other management measures.

[0098] It can be seen that data information is extracted from multiple types of data sources and divided into event data and personnel data to achieve preliminary classification and sorting of data and avoid data disorder; then the event data is segmented to obtain words and part-of-speech tagging to construct an event structure tree, and the subject information is extracted from the event structure tree. This step greatly reduces the interference of irrelevant data and accurately locates the core content of the event. Then, the feature extraction rules are used to extract the first feature data related to the subject information in the event structure tree, so that the key features of the event can be clearly presented. For personnel data, after the segmentation and part-of-speech tagging to construct the personnel structure tree, the sentiment tendency analysis of the extracted words is carried out, and negative sentiment words are marked, so as to facilitate the subsequent judgment of the conflict and dispute events, and then the feature extraction rules are used to extract the second feature data related to the personnel, deeply explore the key information in the personnel data and establish a preliminary connection with the event. According to the first feature data and the second feature data, the subject information and the personnel are clustered. The personnel carry the tag words in the clustering process, so that the event is closely associated with the relevant personnel, and the deep integration of data is realized. Finally, the threshold of the subject information in the threshold configuration file is read. When the words carrying the mark exceed the corresponding threshold, the event to which the subject information belongs is accurately determined to be a conflict and dispute event, and the corresponding personnel after clustering processing are output, which solves the problem of data confusion and mismatch, thereby judging the real situation and development trend of the conflict and dispute event and improving the accuracy of the early warning.

[0099] In some embodiments, after step S101, the method further includes:

[0100] S107, storing data information whose time is lower than the time threshold in the memory database, and storing data information whose time is not lower than the time threshold in the distributed file system;

[0101] In some embodiments, during actual operation, the system will obtain the time attribute corresponding to each piece of data information (this time attribute can be the time when the data was generated, the time when it was last updated, etc., depending on the specific business definition), and then compare it with the pre-set time threshold. For those data information whose time attributes show that the time is lower than the time threshold, they are stored in the memory database through a special data transmission interface or database operation statement to ensure that they can be quickly obtained when these newer data need to be quickly called for analysis, judgment of conflicts and disputes, etc. For data information whose time is not lower than the time threshold, the corresponding transmission mechanism is also used to store them on each storage node of the distributed file system, which not only ensures the long-term storage of data, but also takes advantage of the large capacity characteristics of the distributed file system to avoid occupying too many memory resources, providing an efficient and reasonable solution for the data storage architecture of the entire system.

[0102] S108: Create a data index for the data information.

[0103] In some embodiments, the system will determine the key dimensions that need to be indexed based on the characteristics of the data information and the business application scenarios, such as the type of event, the identity of the people involved, the time range of the event, etc. Then, the corresponding index creation algorithm and technology are used to create indexes for the data information in the memory database according to the selected key dimensions using its built-in index creation function. For the data in the distributed file system, the distributed index construction tool is also used to build index information on each storage node based on the set key dimensions, and the index information is effectively integrated and associated, so that when performing data queries, no matter where the data is stored in the memory database or the distributed file system, it can be quickly located through the index, achieving efficient data retrieval services and improving the data utilization efficiency of the entire system.

[0104] It can be seen that storing data information with a time lower than the threshold in the memory database can make the data generated recently and closely related to the current conflict and dispute events be quickly called and analyzed due to its fast reading and writing speed, thus speeding up the data processing rhythm. Data with a time not lower than the threshold is stored in the distributed file system, which ensures long-term data storage while not occupying too much memory resources, and the distributed architecture is conducive to large-scale data management.

[0105] It can be seen that by establishing a data index for data information, when querying data, there is no need to traverse the entire database. The required data can be quickly located through the index, whether it is event data or personnel data, it can be quickly retrieved and called.

[0106] The following introduces an exemplary conflict and dispute event warning system 400 provided in an embodiment of the present application. Figure 4 It is a schematic diagram of an exemplary hardware structure of the conflict and dispute event warning system 400 provided in an embodiment of the present application.

[0107] In some embodiments, the conflict and dispute event early warning system 400 is a computer device or the conflict and dispute event early warning system 400 includes a computer device. The computer device includes a processor, a memory and a network interface connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data. The network interface of the computer device is used to communicate with other external terminals or servers through a network connection. In some embodiments, the network interface can be a wired network interface, and in some embodiments, the network interface can also be a wireless network interface. When the computer program is executed by the processor, the method in the embodiment of the present application is implemented.

[0108] Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0109] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0110] As used in the above embodiments, the term "when..." may be interpreted to mean "if..." or "after..." or "in response to determining..." or "in response to detecting...", depending on the context. Similarly, the phrases "upon determining..." or "if (the stated condition or event) is detected" may be interpreted to mean "if determining..." or "in response to determining..." or "upon detecting (the stated condition or event)" or "in response to detecting (the stated condition or event)", depending on the context.

[0111] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk), etc.

[0112] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

Claims

1. A conflict and dispute event early warning method, characterized in that: include: Extracting data information from a data source, extracting keywords in the current data information, and classifying the data information into different types according to the types of the keywords; The current data information is any of the data information; including event data of type event and personnel data of type personnel; the data sources include demand collection sources, grid member reporting sources, technical patrol and alarm collection sources, superior and department data sources, and offline data sources; After classification, a source identification field is added to the current data information, wherein the source identification field is assigned a corresponding identification value according to the data source; Based on the source identification field, the business data relationship is constructed by taking the current data information as the main node and the data information having the flow and association relationship between the data information as the secondary node; Adding the classification of the current data information to the data information having the circulation and association relationship between the data information; Storing different types of data information in corresponding databases; Segmenting the event data and the personnel data to obtain words, and tagging the words with parts of speech, and constructing an event structure tree and a personnel structure tree according to the results of the part of speech tagging; Extracting subject information from the event structure tree; and extracting first feature data related to the subject information from the event structure tree using a feature extraction rule; The subject information is a relevant statement reflecting the content or topic of the event; Perform sentiment analysis on the words extracted from the personnel structure tree; mark the words determined to have negative sentiment, and use the feature extraction rule to extract second feature data related to the personnel in the personnel structure tree; the first feature data and the second feature data include the same elements; Clustering the subject information and the persons according to the first characteristic data and the second characteristic data, and the persons may carry marked words during the clustering process; The threshold of the subject information pre-set in the threshold configuration file is read. When the words carrying the mark exceed the corresponding threshold, the event to which the subject information belongs is determined to be a conflict and dispute event, and the corresponding personnel after the clustering process are output together.

2. The method according to claim 1, characterized in that The step of storing different types of data information in corresponding databases specifically includes: For the data information having multiple types, a corresponding database is randomly selected for storage, and index identifiers tracing back to other corresponding databases are added to the storage record.

3. The method according to claim 1, characterized in that The step of extracting subject information from the event structure tree specifically includes: Extracting the topic information from the event structure tree using a topic information extraction model; The training process of the topic information extraction model is as follows: Preset training topic information and training corpus set, wherein the training corpus set includes training corpus of categories related to the training topic information and training corpus of categories unrelated to the training topic information; Traversing the training corpus once, identifying each word by syntactic analysis, and generating corresponding extraction rules for it; Continue traversing the training corpus to generate statistical data for each of the extraction rules; Scoring the extraction rules according to the statistical data, wherein the extraction rules are positively correlated with the number of times the training corpus appears in a category related to the training subject information, and negatively correlated with the number of times the training corpus appears in a category unrelated to the training subject information; The extraction rules whose scoring results exceed the scoring threshold are retained.

4. The method according to claim 3, characterized in that The step of scoring the extraction rules according to the statistical data specifically includes: The score of the extraction rule is the number of times the training corpus in the category related to the training subject information appears divided by the number of times the training corpus in the category unrelated to the training subject information appears; After obtaining all the scoring results, the scoring results are binarized.

5. The method according to claim 1, characterized in that: After the step of extracting data information from the data source and dividing the data information into different types according to a preset data storage method and business data relationship, the method further includes: The data information whose time is lower than the time threshold is stored in the memory database, and the data information whose time is not lower than the time threshold is stored in the distributed file system.

6. The method according to claim 5, characterized in that After the step of storing the data information whose time is lower than the time threshold in the memory database and storing the data information whose time is not lower than the time threshold in the distributed file system, the method further includes: Create a data index for the data information.

7. A conflict and dispute incident early warning system, characterized in that: The conflict and dispute event warning system includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the conflict and dispute event warning system to execute the method described in any one of claims 1-6.

8. A computer program product comprising instructions, characterized in that When the computer program product is run on a conflict and dispute event early warning system, the conflict and dispute event early warning system is enabled to execute the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium comprising instructions, characterized in that: When the instruction is executed on the conflict and dispute event early warning system, the conflict and dispute event early warning system executes the method as described in any one of claims 1-6.

Images