Social news data knowledge graph automatic construction method based on semantic network

Abstract

The invention discloses a semantic information extraction method based on news-oriented social data, which comprises the following steps: establishing a social news data semantic network, performing information extraction on news, and obtaining a news social data ontology; constructing a cue word generation module, and obtaining the cue word information of the news data based on the extracted news social data ontology; according to the prompt word information, processing other news information by adopting a large language model to obtain a corresponding news social data ontology; and establishing a relationship between different news social data ontologies to obtain a map. According to the method disclosed by the invention, complex unstructured text data can be processed, a more accurate information extraction method is provided, unstructured social news data is converted into organized knowledge, and the availability of the data is improved.

Description

Technical Field

[0001] The present invention relates to a method for automatically constructing a social news data knowledge graph based on a semantic network, and belongs to the technical field of artificial intelligence. Background Art

[0002] The automated construction of knowledge graphs can extract, link and organize information from text, data and multimedia resources to build rich semantic networks to provide support for knowledge management and intelligent applications.

[0003] The current automated construction of knowledge graphs is mainly based on semi-automatic and manual annotations, which has many defects.

[0004] First, these methods are affected by the limited and subjective nature of manually annotated data, resulting in incompleteness and inaccuracy of knowledge graphs.

[0005] Second, the construction process is time-consuming and expensive, and is not suitable for the continuous updating and maintenance of large-scale knowledge graphs.

[0006] Finally, it is difficult to track rapidly changing information, such as news events and updates to domain knowledge.

[0007] Therefore, it is necessary to further study the existing automatic construction methods of knowledge graphs to solve the above problems. Summary of the invention

[0008] In order to overcome the above problems, the inventors conducted in-depth research and designed a semantic information extraction method based on news social data, including the following steps:

[0009] S1. Establish a social news data semantic network, extract news information, and obtain news social data ontology;

[0010] S2, constructing a prompt word generation module to obtain prompt word information of news data based on the extracted news social data ontology;

[0011] S3, based on the prompt word information, use the large language model to process other news information and obtain the corresponding news social data ontology;

[0012] S4. Establish the relationship between different news social data ontologies and obtain a graph.

[0013] In a preferred embodiment, the social news data semantic network extracts news into meta information and event information.

[0014] The meta information includes a label of the publishing organization, a search label of the news, and a label of the media type;

[0015] The event information includes a plurality of sub-event information, and each sub-event information includes time information, location information, a core object, a subject, and an object.

[0016] In a preferred embodiment, the process of extracting event information includes the following sub-steps:

[0017] S11, extract event type;

[0018] S12. Construct an extraction framework to automatically extract news information to obtain event information.

[0019] In a preferred embodiment, in S11, a predicate trigger word that matches the event type is found in the sentence, and the event that occurred is determined according to the head and tail positions of the event predicate trigger word, thereby determining the event type.

[0020] In a preferred embodiment, in S11, sequence annotation is used to determine the head and tail positions of the trigger words;

[0021] Set an encoder and a decoder to process the sequence of sequence annotations, where the encoder encodes the input sequence and maps each element to a latent space vector;

[0022] The decoder generates an output sequence of hidden states, with a corresponding result for each position in the sequence.

[0023] In a preferred embodiment, the encoder adopts a BERT model, and the decoder uses a linear decoder.

[0024] In a preferred embodiment, a decoder is provided for each event type to extract event predicate trigger words and thereby obtain the event type.

[0025] In a preferred embodiment, in S12, when the extraction framework is trained, a prompt learning strategy is adopted. During the training process, the event type and the results extracted by the extraction framework are concatenated and used as prompt words.

[0026] The present invention also provides an electronic device, comprising:

[0027] at least one processor; and

[0028] A memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any of the methods described above.

[0029] The present invention also provides a computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute any method described in any one of the above.

[0030] The beneficial effects of the present invention include:

[0031] (1) It can process complex unstructured text data and provide a more accurate information extraction method;

[0032] (2) Better understanding of the meaning of news text content and accurate extraction of key information will help improve the accuracy of information extraction and processing;

[0033] (3) Convert unstructured social news data into organized knowledge, improving the usability of data. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A flow chart of a method for automatically constructing a social news data knowledge graph based on a semantic network according to a preferred embodiment of the present invention is shown. DETAILED DESCRIPTION

[0035] The present invention will be further described in detail below through the accompanying drawings and embodiments. Through these descriptions, the characteristics and advantages of the present invention will become more clear and distinct.

[0036] The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments. Although various aspects of the embodiments are shown in the drawings, the drawings are not necessarily drawn to scale unless otherwise noted.

[0037] The present invention provides a method for automatically constructing a social news data knowledge graph based on a semantic network. Figure 1 As shown, the following steps are included:

[0038] S1. Establish a social news data semantic network, extract news information, and obtain news social data ontology;

[0039] S2, constructing a prompt word generation module to obtain prompt word information of news data based on the extracted news social data ontology;

[0040] S3, based on the prompt word information, use the large language model to process other news information and obtain the corresponding news social data ontology;

[0041] S4. Establish the relationship between different news social data ontologies and obtain a graph.

[0042] The social news data semantic network extracts news into meta-information and event information.

[0043] The meta information includes a publishing agency tag, a news search tag, and a media type tag.

[0044] In the present invention, the publishing agency tag includes personal publishing and organizational publishing, which are used to express the creator and publishing agency of the news.

[0045] The search tag of the news is used to indicate the hot search words of the news;

[0046] The media type tag is used to indicate the type of news media.

[0047] Each piece of news social data is generally published by the media creator through the media platform on a specific date, with the title as the core feature, that is, the meta-information in the news is all structured data.

[0048] According to the present invention, a meta-information semantic network is set up to extract information from structured data in news to obtain meta-information.

[0049] In the present invention, the search tags and media type tags of the news in the meta-information are directly obtained through the news platform. Each piece of social news data is usually created by a media creator and published on a certain date through a specific media. When published, a title will be set, and the meta-information can be directly extracted through the news platform.

[0050] News platforms, such as the Wenhai Big Data Platform, define tags for each media and provide recommended search terms as hot words, as shown in Table 1.

[0051] Table 1 Examples of social news data metadata

[0052]

[0053] According to the present invention, the issuing agency tag in the meta-renewal is obtained through the FOAF framework. FOAF (friend of a friend) is an ontology for describing individuals, organizations and groups. The Person and Organization classes in the framework are used to obtain the creator and issuing agency.

[0054] Preferably, the meta-information semantic network can also extract entity data from tabular data of news.

[0055] The event information includes a plurality of sub-event information, and each sub-event information includes time information, location information, a core object, a subject, and an object.

[0056] Among them, time information is used to describe the time when the sub-event occurs; location information is used to describe the location where the sub-event occurs; the core object is used to describe the entity or element of the event itself, which is the core action of the sub-event, and the core action determines the central meaning of the sub-event; the subject is the initiator of the core action; the object is the receiver of the core action. It can be seen that event information belongs to unstructured data.

[0057] For the unstructured data in the news, an event information semantic network is set up to extract information and obtain event information.

[0058] The information extraction process of the event information semantic network includes the following sub-steps:

[0059] S11, extract event type;

[0060] S12. Construct an extraction framework to automatically extract news information to obtain event information.

[0061] In S11, the event type is the same as defined in the existing dataset, for example

[0062] Qianyan Dataset DUEE1.0 has 65 event types in 8 aspects. The 65 event types include not only common event types in traditional event extraction and evaluation, such as sports events and trading behaviors, but also event types with the characteristics of the times, such as "likes". For each type, different event frameworks are defined and annotated, which is the highest quality general event annotation data in the industry.

[0063] Sentences in news have predicates and trigger words, where predicates are words that act as verbs or adjectives in a sentence, such as "eat", "buy", "happy", etc. Trigger words refer to words related to a predicate in a sentence, which can be verbs, nouns, adjectives, etc., such as "ate", "buyer", "happier", etc.

[0064] In the process of extracting event types, find the predicate trigger words that match the event type in the sentence, and use the head and tail positions of the event predicate trigger words as signs of event occurrence; determine the event type based on the event that occurred.

[0065] In a preferred embodiment, sequence tagging is used to determine the head and tail positions of trigger words.

[0066] More preferably, the head and tail positions of the trigger words are determined using sequence annotation based on BIO and CRF.

[0067] Sequence labeling is a type of method in natural language processing, which is widely used in information extraction, machine translation, question-answering systems, etc.

[0068] BIO is a commonly used tagging scheme used to mark entities in text. CRF (Conditional Random Field) is a sequence labeling algorithm based on probabilistic graphical models. It can overcome the defects of traditional models, has strong modeling and generalization capabilities, and has been widely used in natural language processing, computer vision, bioinformatics and other fields.

[0069] In traditional sequence labeling methods, the starting position and the ending position of an entity are represented by one type of label, which makes it difficult to handle nested structures. The nested structure refers to the situation in which some entities are included in other entities in a text.

[0070] In the present invention, an encoder and a decoder are set to process the sequence labeled sequence, wherein:

[0071] The encoder encodes the input sequence, mapping each element to a latent space vector;

[0072] The decoder generates an output sequence of hidden states, with a corresponding result for each position in the sequence.

[0073] Furthermore, the encoder adopts the BERT model, which is a pre-trained language model based on Transformer and is widely used in natural language processing.

[0074] In the present invention, tasks requiring internal access to a sequence are solved by directly pointing to elements in the sequence, and subsequences can be directly selected from the sequence and their start and end positions can be output, thereby solving the output problem of sequences with variable lengths.

[0075] More preferably, the encoder adopts the BERT-wwm-ext model, which is a Chinese pre-training model based on full-word Mask released by Harbin Institute of Technology iFLYTEK Laboratory.

[0076] The decoder uses a linear decoder, which includes two linear layers and one normalization layer.

[0077] Specifically, according to the output results of the encoder, the starting position and tail position of the content to be extracted are obtained, and the scores of the starting position and tail position of the content to be extracted are calculated respectively through two linear layers. The scores are normalized through the normalization layer, and then the range of the extracted content is determined by the screening threshold.

[0078] Using a linear decoder helps reduce model parameters, making the model easier to converge and speeding up the calculation of the model.

[0079] Furthermore, a decoder is set for each event type to extract the event predicate trigger word and then obtain the event type. This method can not only solve the nesting problem, but also dynamically adjust the threshold according to the distribution of data to balance precision and recall.

[0080] Preferably, during the training of the encoder and decoder, the loss function uses the BCEWithLogitsLoss function. In the task of starting position prediction, it is necessary to predict whether each token is the starting and ending position of the extracted content, so it can be regarded as a binary classification problem. Compared with the sigmoid function, the loss function of the traditional binary classification problem, the BCEWithLogitsLoss function is used to avoid the problem of the sigmoid function's gradient vanishing or exploding when the probability value is close to 0 or 1, making the training more stable.

[0081] In S12, the extraction framework adopts the BERT model. The BERT model is a model based on the Transformer Encoder structure and is widely used in natural language processing.

[0082] Preferably, when training the extraction framework, a prompt-based learning strategy is used, which has many advantages over other algorithms. First, the prompt learning strategy reduces the amount of knowledge that the model needs to learn. Since the prompt text already provides the semantics of the event argument, the model only needs to learn how to match the prompt text with the input text, so as to quickly learn how to extract the event argument; second, the flexibility of the model is improved. Only by replacing different prompt words, different types of event argument extraction can be achieved through a single model; at the same time, the prompt text of the prompt learning strategy can be adjusted, modified and expanded according to new data and tasks, so that the model can be more adaptable to new situations. This adaptive prompt text design can achieve better extraction effects in new fields and tasks.

[0083] In the traditional pre-training + fine-tuning method, the pre-trained model usually learns a large amount of text and language knowledge information by pre-training on large-scale text data. However, these models may not be able to fully realize their potential in downstream tasks, because the input and output forms of downstream tasks may be different from those in the pre-training stage, resulting in the model being unable to accurately understand and process the tasks. Prompt-based Learning effectively solves this problem.

[0084] The training process using the prompt learning strategy generally includes two stages, the first stage is the prompt content selection stage, and the second stage is the model tuning stage. The advantage of using this two-stage extraction is that, first, the event types extracted in the first stage can supplement the framework elements of the second part, enrich the prompt words, and improve the efficiency of the second stage extraction. Secondly, the design of the second stage improves the robustness of event type extraction, allowing the model to extract event types outside the defined distribution.

[0085] In a preferred embodiment, in the prompt content selection stage, the event type and the results extracted by the extraction framework are concatenated as prompt words. This method realizes efficient and standardized generation of prompt word content.

[0086] According to the present invention, the meta information and event information are combined to obtain the news social data ontology.

[0087] According to the present invention, the meta information and event information are combined using the Protege ontology building tool of Stanford University.

[0088] In S2, the prompt words are formed by combining the event type and the news social data ontology.

[0089] In S3, there is no limitation on the specific type of the large language model, and any existing large language model may be used.

[0090] Furthermore, the prompt word information and the news information to be processed are input into the large language model together, and the large language model outputs the corresponding news social data ontology.

[0091] Preferably, the output results of the large language model are also cleaned, the generated fields are screened, and the fields that do not meet the preset values ​​are removed.

[0092] In S4, each piece of news social data itself is regarded as a news node, and the news nodes are associated with each other through shared entities, shared relationships, meta-information connections or social connections to form a graph.

[0093] A map is a graphic that shows the development process and structural relationship of knowledge. It is widely used in the field of library and information. It can visualize the core structure and development history, providing users with practical and valuable reference.

[0094] Preferably, the shared entity can be an entity type such as a person, organization, place, etc. For example, if two news items both mention the same person or organization, we can establish a connection between the two news items through the person or organization entity;

[0095] A shared relationship is a semantic relationship or a link relationship. For example, the connection between two entities can be a relationship type such as "belongs to", "leads", or "influences".

[0096] Meta-information connection refers to the connection between news metadata, such as the connection between release time, release media, etc.

[0097] Social connections refer to the connections between news and social entities, such as people, organizations, places, etc.

[0098] The acquisition of this graph will help improve the ability to process and analyze massive news data; it will also help to achieve more efficient applications such as information retrieval and knowledge reasoning.

[0099] Various embodiments of the method described above in the present invention can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs, which may be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general programmable processor, which may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0100] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0101] In the context of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0102] To provide interaction with a user, the methods and apparatus described herein may be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0103] The methods and apparatus described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communications network). Examples of communications networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0104] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0105] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in the present disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution disclosed in the present invention can be achieved, and this document is not limited here.

[0106] Example

[0107] Example 1

[0108] A simulation experiment was conducted to build a social news data knowledge graph based on a data set that contains more than 1,000 pieces of news in the fields of politics, economy, technology, and sports. The following steps were included:

[0109] S1. Establish a social news data semantic network, extract news information, and obtain news social data ontology;

[0110] S2, constructing a prompt word generation module to obtain prompt word information of news data based on the extracted news social data ontology;

[0111] S3, based on the prompt word information, use the large language model to process other news information and obtain the corresponding news social data ontology;

[0112] S4. Establish the relationship between different news social data ontologies and obtain a graph.

[0113] The information extraction process of the event information semantic network includes the following sub-steps:

[0114] S11, extract event type;

[0115] S12. Construct an extraction framework to automatically extract news information to obtain event information.

[0116] In the process of extracting event types, we find the predicate trigger words that match the event type in the sentence, and use the head and tail positions of the event predicate trigger words as the sign of the event occurrence; according to the event that occurred, we determine the event type. Among them, we use sequence annotation based on BIO and CRF to determine the head and tail positions of the trigger words.

[0117] Set the encoder and decoder to process the sequence of sequence labeling, where:

[0118] The encoder encodes the input sequence, mapping each element to a latent space vector;

[0119] The decoder generates an output sequence of hidden states, with a corresponding result for each position in the sequence.

[0120] Furthermore, the encoder adopts a BERT model, and the decoder uses a linear decoder, which includes two linear layers and one normalization layer.

[0121] In S12, the extraction framework adopts the BERT model, and when training the extraction framework, a prompt learning strategy is adopted, which includes two stages, the first stage is the prompt content selection stage, and the second stage is the model tuning stage. In the prompt content selection stage, the event type and the result extracted by the extraction framework are spliced ​​as the prompt word.

[0122] Comparative Example 1

[0123] The same experiment as in Example 1 was performed, except that a sequence labeling model was used.

[0124] The results of Example 1 were compared with those of Comparative Example 1, and the results were evaluated by the accuracy of entity recognition, the accuracy of relationship extraction, the semantic consistency, and the completeness of the graph.

[0125] Among them, the accuracy of entity recognition (Precision): the proportion of correct entities among the entities recognized by the two methods;

[0126] Recall accuracy of relation extraction: the proportion of correct relations among the relations between entities identified by the two methods;

[0127] Semantic Consistency: The proportion of semantically reasonable and consistent graphs generated by the two methods is compared through expert review or automated scoring system;

[0128] Completeness of the graph: Compare the completeness of the entities and relationships contained in the graphs constructed by the two methods.

[0129] The comparison results are shown in Table 1.

[0130] Table 1

[0131] Indicator Type Comparative Example 1 Example 1 Entity Recognition Accuracy 85％ 95％ Relation Extraction Accuracy 78％ 92％ Semantic consistency 70％ 90％ Graph Completeness 75％ 94％

[0132] As can be seen from Table 1, the method in Example 1 can more accurately identify entities in news texts, more accurately capture the complex relationships between entities, maintain the semantic consistency of the graph, avoid semantic conflicts and logical errors, improve the overall quality of the graph, ensure the integrity of the graph, and reduce information missing.

[0133] In summary, the method in Example 1 can significantly improve the quality of the knowledge graph when constructing the knowledge graph. It not only performs well in entity recognition and relationship extraction, but also outperforms Comparative Example 1 in terms of semantic consistency and graph integrity. These advantages enable the method of the present invention to construct a more valuable and reliable knowledge graph, which is suitable for various complex practical application scenarios, such as information retrieval, intelligent question answering, and knowledge reasoning.

[0134] The present invention has been described above in conjunction with preferred embodiments, but these embodiments are only exemplary and serve only as an illustration. On this basis, the present invention may be subjected to a variety of substitutions and improvements, all of which fall within the scope of protection of the present invention.

Claims

1. A semantic information extraction method based on news social data, characterized in that: The following steps are involved: S1. Establish a social news data semantic network, extract news information, and obtain news social data ontology; S2, constructing a prompt word generation module to obtain prompt word information of news data based on the extracted news social data ontology; S3, based on the prompt word information, use the large language model to process other news information and obtain the corresponding news social data ontology; S4, establish the relationship between different news social data ontologies and obtain a graph; The event information extraction process includes the following sub-steps: S11, extract event type; S12, constructing an extraction framework to automatically extract news information to obtain event information; In S11, a predicate trigger word that matches the event type is found in the sentence, and the event that occurred is determined according to the head and tail positions of the event predicate trigger word, thereby determining the event type.

2. The semantic information extraction method based on news social data according to claim 1 is characterized in that: The social news data semantic network extracts news into meta-information and event information. The meta information includes a label of the publishing organization, a search label of the news, and a label of the media type; The event information includes a plurality of sub-event information, and each sub-event information includes time information, location information, a core object, a subject, and an object.

3. The semantic information extraction method based on news social data according to claim 1 is characterized in that: In S11, sequence tagging is used to determine the head and tail positions of trigger words; Set an encoder and a decoder to process the sequence of sequence annotations, where the encoder encodes the input sequence and maps each element to a latent space vector; The decoder generates an output sequence of hidden states, with a corresponding result for each position in the sequence.

4. The semantic information extraction method based on news social data according to claim 1 is characterized in that: The encoder adopts the BERT model, and the decoder uses a linear decoder.

5. The semantic information extraction method based on news social data according to claim 3 is characterized in that: A decoder is set for each event type to extract the event predicate trigger word and then obtain the event type.

6. The semantic information extraction method based on news social data according to claim 1 is characterized in that: In S12, when the extraction framework is trained, a prompt learning strategy is adopted. During the training process, the event type and the results extracted by the extraction framework are concatenated as prompt words.

7. The semantic information extraction method based on news social data according to claim 3 is characterized in that: The linear decoder includes two linear layers and one normalization layer.

8. The semantic information extraction method based on news social data according to claim 1 is characterized in that: In S4, each piece of news social data itself is regarded as a news node, and the news nodes are associated with each other through shared entities, shared relationships, meta-information connections or social connections to form a graph.

9. An electronic device, comprising: at least one processor; as well as A memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor so that the at least one processor can execute the method of any one of claims 1-8.

10. A computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-8.

Images