An event text data processing method, device and electronic device

Abstract

The present invention discloses an event text data processing method, device and electronic device, including: obtaining event text data to be analyzed; using a pre-trained keyword extraction model to extract target keywords from the event text data to be analyzed, where the target keywords include an event subject, an event, and a keyword describing the relationship between the event subject and the event; constructing a graph database with the event subject and the event as data vertices and the relationship between the event subject and the event as a data relationship edge; using a preset graph algorithm to process the graph database to generate a knowledge graph corresponding to the event text data to be analyzed. This method is applied to the analysis and processing of social comprehensive governance data. The obtained event text data is used to extract the event subject, the event, and the relationship between the event subject and the event by using a pre-trained keyword extraction model, and then a graph database is constructed to generate a corresponding knowledge graph, accurately completing the processing of the event text data.

Description

Technical Field

[0001] The present invention relates to the technical field of text data processing, and in particular to a method, apparatus, and electronic device for processing event text data. Background Art

[0002] Generally, when processing text data, it is first necessary to preprocess the text data, and then perform Chinese word segmentation. One of the most important parts of Chinese NLP is word segmentation, and the quality of word segmentation will directly affect the subsequent model training effect; then perform feature processing, also known as word vector encoding, to convert the text data into data that can be recognized by a computer for easy calculation, usually into numerical data; finally, perform machine learning. After the word vectors are encoded, the text data can be converted into numerical data and input into a machine model for calculation and training. When analyzing data using this text data processing method, it is impossible to accurately process event text data, and thus it cannot be applied to the processing and analysis of comprehensive governance data. Summary of the Invention

[0003] Therefore, the technical problem to be solved by the present invention is to overcome the defect that the existing method cannot accurately process event text, and thus provide a method, apparatus, and electronic device for processing event text data.

[0004] According to a first aspect, an embodiment of the present invention discloses a method for processing event text data, including: obtaining the event text data to be analyzed; using a pre-trained keyword extraction model to extract target keywords from the event text data to be analyzed, where the target keywords include an event subject, an event, and keywords describing the relationship between the event subject and the event; constructing a graph database with the event subject and the event as data vertices and the relationship between the event subject and the event as a data relationship edge; using a preset graph algorithm to process the graph database to generate a knowledge graph corresponding to the event text data to be analyzed.

[0005] Optionally, before using the pre-trained keyword extraction model to extract target keywords from the event text data to be analyzed, the method further includes: obtaining text data; performing word segmentation on the text data using a preset word segmentation algorithm and performing target keyword extraction using an initial keyword extraction model; when the extraction result does not meet the requirements, performing expansion and data enhancement operations on the extracted target keywords; training the initial keyword model using the data after expansion and enhancement processing until the target keyword extraction requirements are met.

[0006] Optionally, the method further includes: using a pre-trained dangerous event prediction model to analyze the event text data to be analyzed; when the event text data to be analyzed contains a dangerous event, marking the dangerous event in the knowledge graph.

[0007] Optionally, the method further includes: determining the type of the hazardous event according to the target keyword of the hazardous event by using a clustering algorithm; obtaining a solution from an expert knowledge base according to the type of the hazardous event and pushing the solution to a client.

[0008] According to a second aspect, an embodiment of the present invention further discloses an event text data processing apparatus, including: a first obtaining module, configured to obtain event text data to be analyzed; a first extracting module, configured to perform target keyword extraction on the event text data to be analyzed by using a pre-trained keyword extraction model, where the target keyword includes an event subject, an event, and a keyword describing the relationship between the event subject and the event; a construction module, configured to construct a graph database by using the event subject and the event as data vertices and the relationship between the event subject and the event as a data relationship edge; and a processing module, configured to process the graph database by using a preset graph algorithm to generate a knowledge graph corresponding to the event text data to be analyzed.

[0009] Optionally, the apparatus further includes: a second obtaining module, configured to obtain text data; a second extracting module, configured to perform word segmentation on the text data by using a preset word segmentation algorithm and perform target keyword extraction by using an initial keyword extraction model; an expansion module, configured to perform expansion and data enhancement operations on the extracted target keyword when the extraction result does not meet the requirements; and a training module, configured to train the initial keyword model by using the data after expansion and enhancement processing until the target keyword extraction requirements are met.

[0010] Optionally, the apparatus further includes: an analysis module, configured to analyze the event text data to be analyzed by using a pre-trained hazardous event prediction model; and a marking module, configured to mark the hazardous event in the knowledge graph when the event text data to be analyzed includes a hazardous event.

[0011] Optionally, the apparatus further includes: a determination module, configured to determine the type of the hazardous event according to the target keyword of the hazardous event by using a clustering algorithm; and a pushing module, configured to obtain a solution from an expert knowledge base according to the type of the hazardous event and push the solution to a client.

[0012] According to a third aspect, an embodiment of the present invention further discloses an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; where the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the steps of the event text data processing method according to the first aspect or any optional implementation manner of the first aspect.

[0013] According to a fourth aspect, an embodiment of the present invention also discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the event text data processing method described in the first aspect or any optional embodiment of the first aspect are implemented.

[0014] The technical solution of the present invention has the following advantages:

[0015] The event text data processing method / apparatus provided by the present invention includes: obtaining event text data to be analyzed; using a pre-trained keyword extraction model to extract target keywords from the event text data to be analyzed, where the target keywords include an event subject, an event, and a keyword describing the relationship between the event subject and the event; constructing a graph database with the event subject and the event as data vertices and the relationship between the event subject and the event as a data relationship edge; and using a preset graph algorithm to process the graph database to generate a knowledge graph corresponding to the event text data to be analyzed. This method is applied to the analysis and processing of comprehensive governance data. The obtained event text data is used to extract the event subject, the event, and the relationship between the event subject and the event by using a pre-trained keyword extraction model, and then a graph database is constructed to generate a corresponding knowledge graph, accurately completing the processing of the event text data. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for use in the description of the specific embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0017] Figure 1 It is a flowchart of a specific example of the event text data processing method in an embodiment of the present invention;

[0018] Figure 2 It is a schematic block diagram of a specific example of the event text data processing apparatus in an embodiment of the present invention;

[0019] Figure 3 It is a specific example diagram of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0020] The technical solutions of the present invention will be clearly and completely described below with reference to the drawings. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the protection scope of the present invention.

[0021] In the description of the present invention, it should be noted that the orientation or positional relationship indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation to the present invention. In addition, the terms "first", "second", "third" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance.

[0022] In the description of the present invention, it should be noted that unless otherwise clearly specified and defined, the terms "installation", "connection", "coupling" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, an electrical connection; it can be directly connected, or indirectly connected through an intermediate medium, and can also be the communication inside two elements. It can be a wireless connection or a wired connection. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

[0023] In addition, the technical features involved in different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0024] An embodiment of the present invention discloses an event text data processing method. As Figure 1 shown, the method includes the following steps:

[0025] Step 101, obtaining the event text data to be analyzed.

[0026] Exemplarily, the obtained event text data can be some text data including the data relationships between people and people and the data relationships between people and things. In this specific embodiment, the event text data can include, but is not limited to, the text data in the contradiction and dispute investigation information form, the community public opinion information form, and the grid reported clue information form.

[0027] Step 102, using a pre-trained keyword extraction model to extract target keywords from the event text data to be analyzed, where the target keywords include an event subject, an event, and keywords describing the relationship between the event subject and the event.

[0028] Exemplarily, the pre-trained keyword extraction model extracts keywords from the obtained event text data according to certain rules, and obtains the event subject, event, and the relationship between the two in the corresponding event text. In this specific embodiment, the event subject may be an event person. The pre-trained keyword extraction model is used to extract keywords from the event text data, and the event keywords, event person keywords, and the data relationships between event persons, event persons and events in the text data are obtained.

[0029] Step 103: Construct a graph database with the event subject and event as data vertices and the relationship between the event subject and event as a data relationship edge.

[0030] Exemplarily, in this specific embodiment, the event keywords, event person keywords, and the data relationships between event persons, event persons and events extracted by the keyword extraction model are converted into data vertices and data relationship edges of event persons and events, and a graph database is constructed.

[0031] Step 104: Use a preset graph algorithm to process the graph database to generate a knowledge graph corresponding to the to-be-analyzed event text data.

[0032] Exemplarily, a preset graph algorithm is used to analyze the complex data relationships in the graph database to generate a visual knowledge graph corresponding to the to-be-analyzed event text. The type of this preset graph algorithm is not limited in the embodiments of the present application, and those skilled in the art can comprehensively process and analyze the data in the graph database according to the form of the required knowledge graph.

[0033] The event text data processing method provided by the present invention includes: obtaining to-be-analyzed event text data; using a pre-trained keyword extraction model to extract target keywords from the to-be-analyzed event text data, where the target keywords include an event subject, an event, and a keyword describing the relationship between the event subject and the event; constructing a graph database with the event subject and event as data vertices and the relationship between the event subject and event as a data relationship edge; using a preset graph algorithm to process the graph database to generate a knowledge graph corresponding to the to-be-analyzed event text data. This method is applied to the analysis and processing of social comprehensive governance data. The obtained event text data is used to extract the event subject, event, and the relationship between the event subject and event by using a pre-trained keyword extraction model, and then a graph database is constructed to generate a corresponding knowledge graph, accurately completing the processing of the event text data.

[0034] As an alternative embodiment of the present invention, before using the pre-trained keyword extraction model to extract target keywords from the text data of the event to be analyzed, the method further includes: obtaining text data; performing a word segmentation operation on the text data using a preset word segmentation algorithm and performing a target keyword extraction operation using an initial keyword extraction model; when the extraction result does not meet the requirements, performing an expansion and data enhancement operation on the extracted target keywords; and training the initial keyword model using the data after the expansion and enhancement processing until the target keyword extraction requirements are met.

[0035] Exemplarily, before extracting keywords from the text data of the event book, it is necessary to first obtain the text data, perform a word segmentation operation on the text data using a preset word segmentation algorithm and use the initial keyword extraction model to extract target keywords. In this specific embodiment, the preset word segmentation algorithm may include, but is not limited to, the textrank algorithm, the LDA algorithm model, the EmbedRank algorithm, and the SIFRank model. When the effect of keyword extraction is average, perform an expansion and data enhancement operation on the extracted target keywords. In this specific embodiment, the methods for performing an expansion and data enhancement operation on the extracted target keywords include expanding using some similar words found in practice and manually defined, expanding the keywords using the semantic primitive knowledge base of CNKI, and expanding using synonyms and parts of speech; training the initial keyword model using the data after the expansion and enhancement processing. In this specific embodiment, use DL models such as roberta+crf to train the initial keyword model.

[0036] As an alternative embodiment of the present invention, the method further includes: analyzing the text data of the event to be analyzed using a pre-trained dangerous event prediction model; when the text data of the event to be analyzed contains a dangerous event, marking the dangerous event in the knowledge graph.

[0037] Exemplarily, in this specific embodiment, the method for establishing a pre-trained dangerous event prediction model is: extracting the text features of the contradiction and dispute events and the existing field features from the contradiction and dispute events that have been converted into dangerous cases in the historical contradiction data to form a sample data set, and then establishing an SVM model and training the SVM model to obtain a dangerous event prediction model. When it is determined that the text data of the event to be analyzed contains a dangerous event after being analyzed by the pre-trained dangerous event prediction model, mark the dangerous event in the knowledge graph.

[0038] As an alternative embodiment of the present invention, the method further includes: determining the type of the dangerous event using a clustering algorithm according to the target keywords of the dangerous event; obtaining a solution from an expert knowledge base according to the type of the dangerous event and pushing the solution to the user side.

[0039] Exemplarily, the target keywords of a dangerous event are used to determine the type of the dangerous event by means of a clustering algorithm. In this specific embodiment, word segmentation, word frequency, and a clustering algorithm are used to classify the data of resolved contradictions and disputes. The clustering algorithm is an unsupervised algorithm and does not require pre-tagging. After the data of resolved contradictions and disputes are classified, each category includes several or dozens of resolved contradiction and dispute events, as well as corresponding solutions, which serve as an expert knowledge base. When a new contradiction and dispute event requires a recommended solution, the same clustering algorithm is first used to determine the category, and then the solution obtained from the expert knowledge base is pushed to the client. Considering the basic situation of contradictions and disputes, multi-dimensional factors such as the mediation history of contradictions and disputes, the mediation results of contradictions and disputes, time, and space, a multi-dimensional factor relationship is established to form a knowledge association for the mediation of contradictions and disputes. For newly emerging contradictions and disputes, an AI algorithm is used to associate the optimal mediation solution.

[0040] The embodiment of the present invention also discloses an event text data processing device, as Figure 2 shown. The device includes: a first acquisition module 201 for acquiring the event text data to be analyzed; a first extraction module 202 for extracting target keywords from the event text data to be analyzed by using a pre-trained keyword extraction model, where the target keywords include an event subject, an event, and keywords describing the relationship between the event subject and the event; a construction module 203 for constructing a graph database with the event subject and the event as data vertices and the relationship between the event subject and the event as a data relationship edge; and a processing module 204 for processing the graph database by using a preset graph algorithm to generate a knowledge graph corresponding to the event text data to be analyzed.

[0041] The event text data processing device provided by the present invention includes: a first acquisition module for acquiring the event text data to be analyzed; a first extraction module for extracting target keywords from the event text data to be analyzed by using a pre-trained keyword extraction model, where the target keywords include an event subject, an event, and keywords describing the relationship between the event subject and the event; a construction module for constructing a graph database with the event subject and the event as data vertices and the relationship between the event subject and the event as a data relationship edge; and a processing module for processing the graph database by using a preset graph algorithm to generate a knowledge graph corresponding to the event text data to be analyzed. This device is applied to the analysis and processing of social comprehensive governance data. The acquired event text data is used to extract the event subject, the event, and the relationship between the event subject and the event by using a pre-trained keyword extraction model, and then a graph database is constructed to generate a corresponding knowledge graph, accurately completing the processing of the event text data.

[0042] As an alternative embodiment of the present invention, the device further comprises: a second acquisition module, configured to acquire text data; a second extraction module, configured to perform a word segmentation operation on the text data by using a preset word segmentation algorithm and perform a target keyword extraction operation by using an initial keyword extraction model; an expansion module, configured to perform an expansion and data enhancement operation on the extracted target keyword when the extraction result does not meet the requirements; and a training module, configured to train the initial keyword model by using the data after the expansion and enhancement processing until the target keyword extraction requirements are met.

[0043] As an alternative embodiment of the present invention, the device further comprises: an analysis module, configured to analyze the text data of the event to be analyzed by using a pre-trained dangerous event prediction model; and a marking module, configured to mark the dangerous event in the knowledge graph when the text data of the event to be analyzed contains a dangerous event.

[0044] As an alternative embodiment of the present invention, the device further comprises: a determination module, configured to determine the type of the dangerous event by using a clustering algorithm according to the target keyword of the dangerous event; and a push module, configured to obtain a solution from an expert knowledge base according to the type of the dangerous event and push the solution to a user terminal.

[0045] An embodiment of the present invention further provides an electronic device, as Figure 3 shown. The electronic device may include a processor 401 and a memory 402. The processor 401 and the memory 402 may be connected by a bus or other means. Figure 3 Taking the connection by a bus as an example.

[0046] The processor 401 may be a central processing unit (CPU). The processor 401 may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. chips, or a combination of the above various types of chips.

[0047] The memory 402, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the event text data processing method in the embodiments of the present invention. The processor 401 executes various functional applications and data processing of the processor by running the non-transitory software programs, instructions, and modules stored in the memory 402, that is, implements the event text data processing method in the above method embodiments.

[0048] The memory 402 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created by the processor 401, etc. In addition, the memory 402 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some embodiments, the memory 402 may optionally include a memory remotely disposed relative to the processor 401, and these remote memories can be connected to the processor 401 through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0049] The one or more modules are stored in the memory 402 and, when executed by the processor 401, execute the event text data processing method in the embodiments as Figure 1 shown.

[0050] Specific details of the above electronic device can be understood by referring to the corresponding relevant descriptions and effects in the Figure 1 shown embodiments, and will not be elaborated here.

[0051] Those skilled in the art can understand that to implement all or part of the processes in the above method embodiments, it can be completed by instructing relevant hardware through a computer program. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the above method embodiments. Among them, the storage medium can be a magnetic disk, an optical disc, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD), or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above types of memories.

[0052] Although embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations fall within the scope defined by the appended claims.

Claims

1. A method for processing event text data, characterized in that, it includes: Obtain the event text data to be analyzed; Use a pre-trained keyword extraction model to extract target keywords from the event text data to be analyzed, where the target keywords include the event subject, the event, and keywords describing the relationship between the event subject and the event; Construct a graph database with the event subject and the event as data vertices and the relationship between the event subject and the event as data relationship edges; Use a preset graph algorithm to process the graph database to generate a knowledge graph corresponding to the event text data to be analyzed; Before using the pre-trained keyword extraction model to extract target keywords from the event text data to be analyzed, the method further includes: Obtain text data; Use a preset word segmentation algorithm to perform word segmentation on the text data and use an initial keyword extraction model to perform target keyword extraction operations; When the extraction result does not meet the requirements, perform expansion and data enhancement operations on the extracted target keywords. The expansion and data enhancement operations on the target keywords include using the same-category word polarity expansion of the target keyword, using the semantic primitive knowledge base of HowNet to expand the keyword, and using synonyms and parts of speech to expand; Use the data after expansion and enhancement processing to train the initial keyword extraction model until the target keyword extraction requirements are met; The method further includes: Use a pre-trained dangerous event prediction model to analyze the event text data to be analyzed; When the event text data to be analyzed contains a dangerous event, mark the dangerous event in the knowledge graph; The dangerous event prediction model is established through the following steps: Extract the text features of the contradiction and dispute events and the existing field features from the contradiction and dispute events that have been transformed into dangerous cases in the historical contradiction data to form a sample data set, and then establish an SVM model and train the SVM model to obtain a dangerous event prediction model.

2. The method according to claim 1, characterized in that, the method further includes: According to the target keywords of the dangerous event, use a clustering algorithm to determine the type of the dangerous event; Obtain a solution from the expert knowledge base according to the type of the dangerous event and push the solution to the user side.

3. An event text data processing device, characterized in that, it includes: A first acquisition module for acquiring event text data to be analyzed; A first extraction module for using a pre-trained keyword extraction model to extract target keywords from the event text data to be analyzed, where the target keywords include the event subject, the event, and keywords describing the relationship between the event subject and the event; A construction module for constructing a graph database with the event subject and the event as data vertices and the relationship between the event subject and the event as data relationship edges; A processing module for using a preset graph algorithm to process the graph database to generate a knowledge graph corresponding to the event text data to be analyzed; The device further includes: A second acquisition module for acquiring text data; A second extraction module, configured to perform word segmentation on the text data by using a preset word segmentation algorithm and perform target keyword extraction by using an initial keyword extraction model; An expansion module, configured to perform expansion and data enhancement operations on the extracted target keywords when the extraction result does not meet the requirements. The expansion and data enhancement operations on the target keywords include using the same-category word polarity expansion of the target keyword, expanding the keyword by using the synset knowledge base of HowNet, and expanding by using synonyms and parts of speech; A training module, configured to train the initial keyword extraction model by using the data after expansion and enhancement processing until the target keyword extraction requirement is met; The apparatus further includes: An analysis module, configured to analyze the text data of the event to be analyzed by using a pre-trained dangerous event prediction model; A marking module, configured to mark the dangerous event in the knowledge graph when the text data of the event to be analyzed contains a dangerous event.

4. The apparatus according to claim 3, wherein, the apparatus further includes: A determination module, configured to determine the type of the dangerous event by using a clustering algorithm according to the target keyword of the dangerous event; A push module, configured to obtain a solution from an expert knowledge base according to the type of the dangerous event and push the solution to a user terminal.

5. An electronic device, wherein, it includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the steps of the event text data processing method according to claim 1 or 2.

6. A computer-readable storage medium, on which a computer program is stored, wherein, the computer program, when executed by a processor, implements the steps of the event text data processing method according to claim 1 or 2.

Images