A method and computer program for evaluating the quality of event content completeness

Abstract

The present invention provides a method and computer for evaluating the quality of completeness of event content. The method comprises: constructing an event concept tree, wherein the event concept tree is formed by tree nodes formed by inherent attributes of an event; extracting text elements corresponding to the event concept tree from an event content text to be evaluated; performing attention calculation on the text element extraction result of the event content text to be evaluated and the event concept tree to obtain a text completeness semantic information representation; performing attention calculation on the event concept tree and a concept tree instance formed by the event content text to be evaluated to obtain an element structured instance completeness representation; generating a comprehensive completeness information expression according to the text completeness semantic information representation and the element structured instance completeness representation; and evaluating the completeness quality of the event content text to be evaluated based on the comprehensive completeness information expression.

Description

Technical Field

[0001] The present invention relates to the field of public security management, and in particular to a method and a computer for evaluating the quality of event content completeness. Background Art

[0002] The quality assessment of event content completeness is the content integrity check of the event text. The basis for judging whether the content of an event text is complete can be, for example, whether it contains the seven elements and whether the specific content information of each element is complete. The so-called seven elements are the constituent elements of the event content, including who, what, where, what, why, and what. The event content here can be the content of the police situation.

[0003] There are two types of existing evaluation methods: one is rule-based, which extracts elements from the event text, compares the seven element types, and finally gives the final evaluation result based on the degree of missingness. This method mainly relies on manual combing of rules, cannot enumerate all possibilities, and has poor robustness. Another type is based on the logistic regression model, which mainly encodes the content of the police text directly and then obtains a score through the regression function. The disadvantage of this method is that it cannot obtain the structural representation information of the elements in the text content, and the final evaluation effect has low credibility. Summary of the invention

[0004] The purpose of the present invention is to overcome the deficiencies in the prior art and provide an event content completeness quality assessment method and computer with high credibility of assessment effect.

[0005] To achieve the above object, the present invention provides a method for evaluating the quality of event content completeness, comprising:

[0006] Constructing an event concept system tree, wherein the event concept system tree is formed by tree nodes formed by inherent attributes of the event;

[0007] Extracting text elements corresponding to the event concept system tree from the event content text to be evaluated;

[0008] The text element extraction results of the event content text to be evaluated are used for attention calculation with the event concept tree to obtain the text completeness semantic information representation;

[0009] Attention calculation is performed on the event concept tree and the concept tree instance formed by the event content text to be evaluated to obtain the completeness representation of the element structured instance;

[0010] Generate a comprehensive completeness information expression based on the text completeness semantic information representation and the element structured instance completeness representation;

[0011] Based on the comprehensive completeness information expression, the completeness quality of the event content text to be evaluated is evaluated.

[0012] In a preferred embodiment, the constructing of the event concept system tree includes: constructing a corresponding event concept system tree for each event type.

[0013] In a preferred embodiment, the method further comprises:

[0014] Perform text classification operations on the event content text to be evaluated to determine the event type corresponding to the event content text;

[0015] The step of performing attention calculation on the text element extraction result of the event content text to be evaluated and the event concept system tree comprises: performing attention calculation on the text element extraction result of the event content text to be evaluated and the event concept system tree corresponding to the type of the event;

[0016] The attention calculation of the concept tree instance formed by the event concept tree and the event content text to be evaluated includes: attention calculation of the event concept tree corresponding to the type of the event and the concept tree instance formed by the event content text to be evaluated.

[0017] In a preferred manner, the concept system tree instance formed by the event content text to be evaluated is formed based on the text element extraction result of the event content text to be evaluated and the event concept system tree.

[0018] In a preferred embodiment, the step of performing attention calculation on the text element extraction result of the event content text to be evaluated and the event concept system tree comprises:

[0019] Using the time prediction sequence prediction algorithm, the text element extraction results of the event content text to be evaluated are processed to obtain the event text sequence vector representation;

[0020] Using a tree-structured sequence prediction algorithm, the event concept tree is processed to obtain a vector representation of the event concept tree.

[0021] Based on the vector representation of the event text sequence and the vector representation of the event concept tree, attention calculation is performed to obtain the text completeness semantic information representation.

[0022] In a preferred embodiment, the attention calculation of the concept tree instance formed by the event concept tree and the event content text to be evaluated includes:

[0023] Using a tree-structured sequence prediction algorithm, the event concept tree is processed to obtain a vector representation of the event concept tree.

[0024] Using a tree-structured sequence prediction algorithm, the concept tree instance is processed to obtain a vector representation of the concept tree instance;

[0025] Attention calculation is performed based on the vector representation of the event concept tree and the vector representation of the concept tree instance to obtain a completeness representation of the element structured instance.

[0026] In a preferred embodiment, the step of evaluating the completeness quality of the event content text to be evaluated based on the comprehensive completeness information expression includes:

[0027] The comprehensive completeness information expression is used as an activation function input, and a completeness quality score of a continuous distribution within a predetermined interval is output.

[0028] In a preferred embodiment, generating a comprehensive completeness information expression according to the text completeness semantic information representation and the element structured instance completeness representation includes:

[0029] The text completeness semantic information representation and the element structured instance completeness representation are concatenated to obtain a comprehensive completeness information expression.

[0030] In a preferred embodiment, the inherent attributes of the event include:

[0031] The people, things, time, place, events, causes, and / or results involved in an incident.

[0032] Another aspect of the present invention provides a computer, comprising: a processor and a memory, wherein the processor is used to execute the code in the memory to execute the above-mentioned event content completeness quality assessment method.

[0033] The present invention combines the concept tree of the event, extracts the elements in the event content text and structures them to form an instance of the concept tree, and finally conducts completeness information learning from the two dimensions of text and structured knowledge in combination with the concept tree, so as to better obtain the key information in the alarm content, and then match it with the consistency of the concept tree to obtain a completeness quality score with higher credibility.

[0034] Other beneficial effects of the present invention will be further described in the specification. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0036] Figure 1 A flow chart of a method for evaluating the quality of event content completeness provided by an embodiment of the present invention;

[0037] Figure 2 It is a structural diagram of an event concept system tree;

[0038] Figure 3 It is a schematic diagram of an example of a concept tree;

[0039] Figure 4 is a schematic diagram of a model architecture of a method for evaluating event content completeness quality in an embodiment of the present invention;

[0040] Figure 5 The diagram is a diagram illustrating the process of evaluating the completeness quality of event content, taking the event content text as an example of a piece of police content text. DETAILED DESCRIPTION

[0041] In order to facilitate the understanding of the present invention, the present invention will be described more fully below with reference to the relevant drawings. The preferred embodiments of the present invention are given in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present invention more thoroughly understood.

[0042] The embodiment of the present invention proposes an event content completeness quality assessment model, which combines the concept system tree of the event, extracts the elements in the event content text and structures them to form an instance of the concept system tree, and finally learns the completeness information from the two dimensions of text and structured knowledge, and combines the concept system tree to better obtain the key information in the alarm content, and then matches it with the consistency of the concept system tree to obtain a completeness quality score with higher credibility.

[0043] The detailed implementation is described below.

[0044] Figure 1 A flow chart of a method for evaluating the quality of event content completeness provided by an embodiment of the present invention is shown. The method comprises the following steps:

[0045] Step S101: construct an event concept system tree, which is formed by tree nodes formed by inherent attributes of the event.

[0046] The event concept system tree is summarized by the inherent attributes of the event, and the inherent attributes can have a hierarchical subordinate structure, so it corresponds to a tree structure in the data structure. For this reason, the system can be constructed in a tree structure, that is, the event concept system tree. Each node in the tree is an inherent attribute, and the attribute corresponding to the child node is the subordinate attribute of its parent node attribute. For example, the node representing the "reporter" attribute can be a child node of the node representing "person", and the node representing the "reporter name" attribute is further a child node of the "reporter" attribute node.

[0047] Figure 2 The diagram shows a structural diagram of an event concept system tree. Those skilled in the art should be able to understand that this structure is only an example, and in practice, a part of it can be selected or other parts can be added as needed. Figure 2 In the example shown, the event concept system tree mainly includes attribute nodes of people, objects, time, place, event, cause, and result.

[0048] The sub-nodes of the person attribute node include the attribute nodes of the reporter, the event subject, and the suspected initiator of the event. As an example, the nodes of common attributes of a person are expanded below the reporter attribute node, such as name, gender, temporary residence, etc. It should be understood that similar sub-node expansion settings can also be made under the event subject and the suspected initiator attribute nodes.

[0049] The subnodes of the property node of the object include the property nodes of the subject matter, event tools, and other items. The subnodes of the property node of the subject matter include the property nodes of quantity, model, value, etc. It should be understood that similar subnode expansion settings can also be made in the property nodes of event tools and other items.

[0050] The child nodes of the time attribute node include the reporting time, the event occurrence time, and other time attribute nodes. The child nodes of the location attribute node include the reporting location, the event occurrence location, and other location attribute nodes.

[0051] The child nodes of the event attribute node include the attribute nodes of the event nature and the event process. The child nodes of the cause attribute node mainly include the attribute nodes of the event cause. The child nodes of the result attribute node mainly include the nodes of the personnel loss and property loss attributes.

[0052] In order to reflect the attribute characteristics of different types of events, in a preferred manner, a corresponding event concept system tree can be constructed for each event type. The contents of the event concept system tree for different types of events may be different. For example, the inherent attribute categories or fixed attribute names of different types of events are often different, and the number of nodes, node architecture, and node names of the event concept system tree constructed in this way may be different.

[0053] exist Figure 2 The event concept tree in the event concept tree is specially marked as "event type A".

[0054] Step S102: extracting text elements corresponding to the event concept system tree from the event content text to be evaluated.

[0055] By using the event concept tree constructed in step S101 and with the help of the element extraction model, text elements corresponding to the event concept tree can be automatically extracted from the event content text to be evaluated.

[0056] The feature extraction model can adopt the BERT+LSTM+CRF model.

[0057] It should be noted that, if a corresponding event concept tree is constructed for each type of event in step S101, a text classification operation may be performed on the event content text to be evaluated before step S102 to determine the event type corresponding to the event content text. The text classification operation may be implemented with the aid of text classification algorithms such as TextCNN, DCNN, RCNN, BERT, etc. In a preferred embodiment, a text classification algorithm based on a BERT pre-trained model may be used, such as BERT+linear layer+softmax.

[0058] After determining the event type corresponding to the event content text, step S102 of extracting text elements corresponding to the event concept tree from the event content text to be evaluated is specifically implemented as follows: extracting text elements corresponding to the event concept tree of the event type from the event content text to be evaluated.

[0059] For example, when the event content text is "At about 9:00 on December 12, 2020, Zhang San (male, 13812345678, ** Garden, Suzhou Industrial Park) reported:..." After text allocation, it is determined that the event type is type B (such as theft), and then the text elements corresponding to the event concept system tree of type B are extracted from the event content text.

[0060] The results of text feature extraction can be presented in the form of event content text with feature tags. For example, the results of the feature extraction model can be used as markers to mark the positions of various elements in the original event content text, which can better enable the model to learn the key element information in the text.

[0061] The text element extraction results can be mounted on the event concept tree corresponding to the event type to form a concept tree instance. Figure 3 A schematic diagram showing an example of a concept tree.

[0062] Step S103: Perform attention calculation on the text element extraction result of the event content text to be evaluated and the event concept system tree to obtain the text completeness semantic information representation; perform attention calculation on the event concept system tree and the concept system tree instance formed by the event content text to be evaluated to obtain the element structured instance completeness representation.

[0063] Attention calculation can be an algorithm using the Attention mechanism.

[0064] The text element extraction results of the event content text to be evaluated are subjected to attention calculation with the event concept system tree to obtain the text completeness semantic information representation, which can better preserve the lexical order information between the elements in the original text of the event content text.

[0065] Performing attention calculation on the text element extraction result of the event content text to be evaluated and the event concept system tree may include the following steps:

[0066] Step A.1: Use the time prediction sequence prediction algorithm to process the text element extraction results of the event content text to be evaluated to obtain the event text sequence vector representation;

[0067] Step A.2: Using a tree-structured sequence prediction algorithm, the event concept tree is processed to obtain a vector representation of the event concept tree;

[0068] Step A.3: Based on the vector representation of the event text sequence and the vector representation of the event concept tree, attention calculation is performed to obtain the text completeness semantic information representation.

[0069] Taking the Attention algorithm as an example, the specific implementation of the above steps A.1-A.3 is explained in detail.

[0070] The time prediction sequence prediction algorithm in step A.1 can be a Bi-LSTM algorithm, which uses the Bi-LSTM algorithm to process the text element extraction results of the event content text to be evaluated, and obtains the event content text sequence vector represented as [h0, h1, ..., h t , ..., h L ], where L is the length of the text, h t is the vector representation of the tth character, h t =[h Lt ,h Rt ],h Lt is the vector representation obtained by the forward LSTM, h Rt is the vector representation of the backward LSTM, and finally the two are concatenated to obtain the final vector representation h t .

[0071] The tree-structured sequence prediction algorithm in step A.2 can be a Tree-LSTM algorithm. The Tree-LSTM algorithm is used to process the event concept tree to obtain a vector representation h of the event concept tree. C .

[0072] In step A.3, attention calculation is performed on the vector representation of the event text sequence and the vector representation of the event concept tree. Specifically, the following formula can be used:

[0073]

[0074] c=Σ i α i h i

[0075] Where c is the final representation of the complete semantic information of the text, which is expressed in vector form.

[0076] The event concept tree and the concept tree instance formed by the event content text to be evaluated are used for attention calculation to obtain the completeness representation of the element structured instance, which can better express the hierarchical relationship information between elements.

[0077] Performing attention calculation on a concept tree instance formed by an event concept tree and an event content text to be evaluated may include the following steps:

[0078] Step B.1: Using a tree-structured sequence prediction algorithm, the event concept tree is processed to obtain a vector representation of the event concept tree;

[0079] Step B.2: Using a tree-structured sequence prediction algorithm, the concept tree instance is processed to obtain a vector representation of the concept tree instance;

[0080] Step B.3: Perform attention calculation based on the vector representation of the event concept tree and the vector representation of the concept tree instance to obtain the completeness representation of the element structured instance.

[0081] Taking the Attention algorithm as an example, the specific implementation of the above steps B.1-B.3 is explained in detail.

[0082] The tree-structured sequence prediction algorithm in step B.1 can be a Tree-LSTM algorithm. The Tree-LSTM algorithm is used to process the event concept tree to obtain a vector representation h of the event concept tree. C .

[0083] The tree-structured sequence prediction algorithm in step B.2 can be a Tree-LSTM algorithm. The Tree-LSTM algorithm is used to process the concept tree instance to obtain a factor structured instance vector represented as [h0, h1, ..., h t , ..., h K ], where K is the number of feature instances.

[0084] In step B.3, the Attention algorithm can be used to obtain the complete semantic information representation c′ of the element structured instance using the following formula, where vector information is used.

[0085]

[0086] c′=Σ i α i h i

[0087] It should be noted that step B.1 can be implemented in the same way as the above step A.2, so one of the two steps can be omitted, and the process of obtaining the vector representation of the event concept tree only needs to be performed once.

[0088] Step S104: Generate a comprehensive completeness information expression based on the text completeness semantic information representation and the element structured instance completeness representation.

[0089] Here, the method of generating the comprehensive completeness information expression may specifically be to concatenate the text completeness semantic information representation and the element structured instance completeness representation.

[0090] Step S105: Based on the comprehensive completeness information expression, the completeness quality of the event content text to be evaluated is evaluated.

[0091] Here, an activation function such as a sigmoid function can be used to take the comprehensive completeness information expression obtained in step S104 as input, and output a continuously distributed completeness quality score within a predetermined interval, for example, a score between 0 and 1, where a higher score represents a higher completeness of the current alert content text.

[0092] Figure 4 The schematic diagram of the model architecture of the event content completeness quality assessment method in the embodiment of the present invention is shown, wherein the left side of the model processes the event content text with the help of the Bi-LSTM algorithm and the event concept tree with the help of the Tree-LSTM algorithm in the middle, and the text completeness semantic information representation is obtained by Attention calculation. The right side of the model processes the concept tree instance with the help of the Tree-LSTM algorithm and the event concept tree with the help of the Tree-LSTM algorithm in the middle, and the element structured instance completeness representation is obtained by Attention calculation.

[0093] Figure 5 The example of taking the event content text as a warning content text and the event concept system tree as a seven-element concept tree is shown to illustrate the process of evaluating the completeness quality of the event content in an embodiment of the present invention.

[0094] The event content text completeness quality assessment method proposed in the embodiment of the present invention combines the event concept system tree, extracts the elements in the text and structures them into a concept tree instance, and finally learns the completeness information from the two dimensions of text and structured knowledge, combined with the concept tree, so as to better obtain the key information in the alarm content, and then match it with the consistency of the concept tree, and finally obtain a completeness quality score with higher credibility.

[0095] Another embodiment of the present invention further provides a computer, comprising: a processor and a memory, wherein the processor is used to execute the code in the memory to perform the steps in the event content completeness quality assessment method in the above embodiment.

[0096] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable systems. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server, data center, etc. that includes one or more available media integrated. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

[0097] The various illustrative logic units and circuits described in the embodiments of the present application can be implemented or operated by a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic system, a discrete gate or transistor logic, a discrete hardware component, or the design of any combination of the above functions. The general-purpose processor can be a microprocessor, and optionally, the general-purpose processor can also be any traditional processor, controller, microcontroller or state machine. The processor can also be implemented by a combination of a computing system, such as a digital signal processor and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a digital signal processor core, or any other similar configuration to implement.

[0098] The steps of the method or algorithm described in the embodiments of the present application can be directly embedded in hardware, a software unit executed by a processor, or a combination of the two. The software unit can be stored in an RA memory, a flash memory, an RO memory, an EPRO memory, an EEPRO memory, a register, a hard disk, a removable disk, a CD RO or any other form of storage medium in the art. Exemplarily, the storage medium can be connected to the processor so that the processor can read information from the storage medium and can write information to the storage medium. Optionally, the storage medium can also be integrated into the processor. The processor and the storage medium can be arranged in an ASIC, and the ASIC can be arranged in a terminal. Optionally, the processor and the storage medium can also be arranged in different components in the terminal. These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are performed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0099] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for evaluating the quality of event content completeness, characterized in that: include: Constructing an event concept system tree, wherein the event concept system tree is formed by tree nodes formed by inherent attributes of the event; Extracting text elements corresponding to the event concept system tree from the event content text to be evaluated; The text element extraction results of the event content text to be evaluated are used for attention calculation with the event concept tree to obtain the text completeness semantic information representation; Attention calculation is performed on the event concept tree and the concept tree instance formed by the event content text to be evaluated to obtain the completeness representation of the element structured instance; Generate a comprehensive completeness information expression based on the text completeness semantic information representation and the element structured instance completeness representation; Based on the comprehensive completeness information expression, evaluating the completeness quality of the event content text to be evaluated; The method further includes: Perform text classification operations on the event content text to be evaluated to determine the event type corresponding to the event content text; The step of performing attention calculation on the text element extraction result of the event content text to be evaluated and the event concept system tree comprises: performing attention calculation on the text element extraction result of the event content text to be evaluated and the event concept system tree corresponding to the type of the event; The step of performing attention calculation on the concept tree instance formed by the event concept tree and the event content text to be evaluated comprises: performing attention calculation on the event concept tree corresponding to the event type and the concept tree instance formed by the event content text to be evaluated; The step of evaluating the completeness quality of the event content text to be evaluated based on the comprehensive completeness information expression includes: Using the comprehensive completeness information expression as an activation function input, and outputting a continuous distributed completeness quality score within a predetermined interval; Generating a comprehensive completeness information expression according to the text completeness semantic information representation and the element structured instance completeness representation includes: The text completeness semantic information representation and the element structured instance completeness representation are concatenated to obtain a comprehensive completeness information expression.

2. The method according to claim 1, characterized in that The constructing of the event concept system tree includes: constructing a corresponding event concept system tree for each event type.

3. The method according to claim 1, characterized in that The concept system tree instance formed by the event content text to be evaluated is formed based on the text element extraction result of the event content text to be evaluated and the event concept system tree.

4. The method according to claim 1, characterized in that The step of performing attention calculation on the text element extraction result of the event content text to be evaluated and the event concept system tree includes: Using the time prediction sequence prediction algorithm, the text element extraction results of the event content text to be evaluated are processed to obtain the event text sequence vector representation; Using a tree-structured sequence prediction algorithm, the event concept tree is processed to obtain a vector representation of the event concept tree. Based on the vector representation of the event text sequence and the vector representation of the event concept tree, attention calculation is performed to obtain the text completeness semantic information representation.

5. The method according to claim 1, characterized in that The attention calculation of the concept tree instance formed by the event concept tree and the event content text to be evaluated includes: Using a tree-structured sequence prediction algorithm, the event concept tree is processed to obtain a vector representation of the event concept tree. Using a tree-structured sequence prediction algorithm, the concept tree instance is processed to obtain a vector representation of the concept tree instance; Attention calculation is performed based on the vector representation of the event concept tree and the vector representation of the concept tree instance to obtain a completeness representation of the element structured instance.

6. The method according to claim 1, characterized in that The inherent properties of the event include: The people, things, time, place, events, causes, and / or results involved in an incident.

7. A computer, characterized in that: include: A processor and a memory, wherein the processor is used to execute the code in the memory to execute the event content completeness quality assessment method as described in any one of claims 1 to 6.

Images