Machine integration method of multiple expert opinions in government information projects based on knowledge fusion

Abstract

Peer review is a project evaluation model commonly used in various professional fields. Multiple experts participating in the same project will score relevant indicators according to their own opinions during the review, and then annotate relevant text opinions. The manually integrated expert group opinions are affected by multiple human factors to a certain extent, and have low timeliness and a large review workload, making it difficult to adapt to the working mode of multi-project online review. This achievement proposes to integrate opinions at the text layer based on knowledge fusion and support the integration of text layer opinions with cognitive layer knowledge processing, which effectively improves the efficiency of manual integration and the degree of knowledge coverage. Supported by the knowledge supplement and knowledge representation methods of creating domain knowledge concept trees and project knowledge ontology, expert opinion knowledge is obtained through SAO structured knowledge acquisition and customized unstructured knowledge acquisition methods, and knowledge association and knowledge fusion are completed on the basis of project knowledge ontology, fully realizing the machine integration of expert group opinions of multi-expert review opinions.

Description

1. Technical field

[0001] Computer science and technology (artificial intelligence, knowledge engineering), project management, knowledge management (e-government), information science, text processing technology 2. Background technology

[0002] (1) Project Management

[0003] Compilation of relevant standards and regulations for e-government project management; interpretation of project management processes and project evaluation indicators; analysis of short texts of expert opinions and compilation of templates for expert group opinions, etc.

[0004] (2) Knowledge management technology

[0005] ① Establish a project knowledge concept tree: The project knowledge concept tree is actually a hierarchical knowledge representation method with standard concepts for review objectives. The role of the project knowledge concept tree here is to support hierarchical semantic recognition of expert review opinion entities, support dependency syntactic analysis of expert opinion texts after structured processing, and realize semantic mapping based on the concept tree, thereby realizing semantic recognition of entities in expert opinion texts at different conceptual levels.

[0006] ② Establish the knowledge ontology of government information project approval: The knowledge ontology of government information project can relatively completely represent the background knowledge of the project, and is also the semantic specification of knowledge fusion, which supports the realization of the knowledge fusion technology in this achievement. The concepts and relationships between the concepts of the knowledge ontology of government information project approval come from the relevant standards and regulations of government information project management.

[0007] ③ Opinion mining technology: Opinion mining is an effective means of acquiring knowledge from expert opinion texts. The difficulty of short text opinion mining lies in the fact that the expression of opinion elements in short texts is diverse or non-explicit, which is even more difficult to understand in the Chinese context. Automated short text understanding requires additional knowledge that can help the machine fully explore the connection between words in short texts, such as semantic relevance.

[0008] ④ Knowledge fusion technology: From the perspective of knowledge management, expert opinions are a knowledge source based on project knowledge, and the formation of comprehensive opinions of the expert group is a multi-source knowledge fusion problem based on project knowledge. Knowledge fusion can analyze and extract multi-source heterogeneous data in real time, accurately and intelligently, and realize the disclosure, sharing, association and discovery of knowledge units through scientific calculations for application goals, thereby obtaining new knowledge or new solutions after fusion.

[0009] (3) Text processing technology

[0010] ① Conventional text processing technology: including the compilation of the project evaluation index scoring table, the use of the concept vocabulary of the training project based on the standards and management specifications of the government information project, supplemented by the basic vocabulary consisting of synonyms and near synonyms. The open source tool Hanlp is used to segment the expert opinion text into sentences, words, part-of-speech tags, remove meaningless stop words, use dependency syntax analysis to identify the sentence SAO (subject-verb-object) structure, split the opinion sentences with multiple subjects or multiple objects, and refine the complex sentences containing multiple evaluation objects into multiple simple sentences with SAO structures containing only unique evaluation objects.

[0011] ② Text integration technology based on knowledge fusion: Text integration methods based on information fusion are bound to have the characteristics of uncertainty in knowledge gain due to the lack of domain knowledge norms. On the basis of clear project knowledge boundaries and standardized knowledge expression, the knowledge fusion method can achieve the expression of text integration supported by the knowledge fusion results of the cognitive layer. Compared with the information fusion-based method, it not only has better knowledge gain from the perspective of expanding the knowledge acquisition space, but also has better results in targeted and profound semantic integration results.

[0012] (4) Sentiment Analysis Technology

[0013] A deep learning model is used to judge the sentiment tendency of expert opinion sentences, and four classifiers, RNN, LSTM, BIRNN, and BILSTM, are used to compare sentiment classification to obtain the optimal opinion set with semantic determination and sentiment tendency determination. 3. Summary of the invention

[0014] This achievement uses a hierarchical semantic recognition method to obtain the basic knowledge elements in individual expert opinions, uses domain knowledge ontology to perform semantic standardization and achieve knowledge expansion, and further uses knowledge fusion to support the technical method of expert opinion integration to achieve the generation of expert group opinions. The implementation process of the achievement includes the following main processing links, which are described as follows:

[0015] (1) Knowledge Supplementation and Knowledge Representation

[0016] In order to solve the problem of knowledge acquisition and positioning at the micro level, the project knowledge concept tree is introduced to hierarchically express the semantic hierarchy between project concept knowledge and concepts; at the macro level, the domain knowledge ontology is introduced to represent project knowledge, supporting knowledge expansion and association, knowledge transformation to achieve implicit knowledge discovery; thus completing the generation of comprehensive opinions based on multi-source knowledge fusion. The knowledge that constitutes the concept tree and ontology comes from national standards and relevant management policies for government information projects.

[0017] (2) Knowledge acquisition based on hierarchical semantic recognition

[0018] Expert opinion knowledge acquisition aims to obtain experts' suggestions for adjusting project elements and their emotional attitudes, etc., to provide a basis for knowledge fusion. This strategy mines and acquires knowledge at the lexical level (word segmentation and part-of-speech tagging), the syntactic level (named entity recognition and grammatical analysis), and the semantic level (semantic recognition).

[0019] Since the entities in expert opinions contain rich conceptual semantics, the entity semantics recognition using named entity recognition technology can provide a domain-specific entity vocabulary without a domain dictionary. Based on the short text features of the evaluation opinions, SAO (subject-action-object) extraction is performed on the expert opinions based on grammatical analysis to achieve the acquisition of structured knowledge, and then the concept tree is used to support hierarchical semantic recognition, the evaluation features of expert opinions are mined, and the deep learning model is used to build a sentiment classifier to obtain sentiment polarity. This section specifically includes the following contents:

[0020] ① Named Entity Recognition

[0021] By integrating the four named entity recognition methods of HMM, CRF, BILSTM, and BILSTM-CRF through the voting method, we can obtain better results in the extraction of government function entities. Here, we use the expert evaluation opinion text as data, first use the open source tool Hanlp to preliminarily segment the expert opinion text, segment words, tag parts of speech, remove stop words, and then use the segmentation feature X i , part of speech feature POS i Incorporate named entity annotation to improve the performance of the integrated model. Form a domain entity vocabulary while extracting.

[0022] ②Intrinsic semantic analysis and structured knowledge acquisition

[0023] Dependency syntactic analysis can be used to obtain the dependency grammar in a sentence and reveal the semantic dependency relationship between words in a sentence. SAO is a triple structure extracted from a text corpus. While containing a large amount of information, it can also effectively maintain the intrinsic correlation between information. Using dependency syntactic analysis, opinion sentences with multiple subjects or multiple objects are split up, and fine-grained SAO structured knowledge is extracted, which is used as a basic semantic unit to represent expert opinions. Subject S and object O mainly represent the object of evaluation, which is generally a noun, gerund, etc. The subject or object composed of these words or phrases is closely related to the semantics of the project concept.

[0024] ③ Evaluation feature hierarchical semantic recognition

[0025] The evaluation opinions after SAO structure need to be hierarchically semantically recognized for the subject and object evaluation features to obtain highly targeted fine-grained evaluation features. Due to the differences in concept expression among different experts, the concept semantic mapping based on the concept tree is comprehensively realized by using the precise matching and fuzzy matching methods based on the maximum similarity of the text to obtain better hierarchical semantic recognition effects. The method is as follows:

[0026] 1) Exact matching method

[0027] For SAO structure text SAO i , its subject S i and object O i Respectively with the concept c in the concept set C of the concept tree j Perform a two-way maximum match and take the union of the two successfully matched concept sets to obtain the set R i If R i Not empty, compare R i The hierarchical depth of each concept in , the concept with the largest depth is taken as the semantic recognition result of the sentence; if R i If it is empty, fuzzy matching is performed.

[0028] 2) Fuzzy matching method

[0029] In the process of structuring the opinion text, in order not to destroy the integrity of the expert opinion, modifying words such as adjectives, adverbs, prepositions, conjunctions, etc. are retained. However, in the fuzzy matching using text similarity calculation, it is necessary to use word segmentation and remove modifying words through part of speech screening to improve accuracy, and retain the main word set S i '={s i1 ,s i2 , ..., s im} and O i '={o i1 , o i2 , ..., o in}(m, n are the number of words).

[0030] Therefore, concept c in concept set C j Perform word segmentation to get c j ={c j1 , c j2 , ..., c jt}(t is the number of words). Similarity Sim(S i ',c j )、Sim(O i ',c j ) is calculated as shown in formula (1)(2).

[0031]

[0032]

[0033] Where sim(s ik , c jr ) is the cosine similarity calculation of word2vec word vectors. Through fine-grained word segmentation, the average similarity is calculated in the form of Cartesian product to obtain a more accurate result. Loop through the concept set C and calculate Sim(S i ',c j )、Sim(O i ',c j ), take the maximum similarity greater than the threshold, and its corresponding concept is the SAO i The semantics of the identified concepts.

[0034] ④ Definition of unstructured knowledge acquisition

[0035] Due to the irregularities in the language writing in short texts, the grammatical analysis of some opinions is not good, which inevitably leads to errors in the SAO structured knowledge extraction and the inability to identify the intrinsic semantic relationships in the text. The three main problems in the opinions are summarized and relevant rules are customized to relocate the knowledge, as shown in Table 1.

[0036] Table 1 Semantic analysis of non-SAO structured knowledge of expert opinions

[0037]

[0038] Through the above opinion mining, the knowledge elements in expert opinions can be effectively obtained.

[0039] (3) Integration of expert opinions based on knowledge fusion

[0040] By using the hierarchical semantic knowledge obtained from expert review opinions, the knowledge fusion at the micro and macro levels can effectively reduce this degree of discreteness. Finally, the generation of the specific text structure model of the expert comprehensive opinion is completed with the help of syntactic rules and logical rules. Prior to this, the scores of each indicator and the total score of the project must be calculated according to the sum and average method determined by the project evaluation rules to obtain the review conclusion.

[0041] ① Micro-level knowledge integration

[0042] 1) Generation of comprehensive opinions of parent concepts based on fuzzy theory

[0043] On the basis of opinion mining of hierarchical semantic recognition, a knowledge fusion method based on fuzzy theory is proposed to generate comprehensive comments on parent concepts. Based on fuzzy set theory, the review opinions are converted from "word-vector-word" to generate comprehensive evaluation sentences with high-level semantics. The process mainly includes three steps: a) Fuzzification of sub-concept opinions. The positive tendency probability calculated in sentiment analysis is used to represent the sentiment membership of the opinion under the sub-concept. b) Fuzzy fusion. The sentiment membership corresponding to the sub-concept under the parent concept is fused by the mean method to obtain the comprehensive sentiment membership of the parent concept. c) Clarification. The comprehensive sentiment membership is restored to the comprehensive opinion text.

[0044] 2) Knowledge fusion based on SAO structure alignment

[0045] SAO structural alignment is performed on expert opinions with the same concept node and the same sentiment tendency to achieve knowledge fusion. In the fusion, the hierarchical semantic relationship between the sub-concept and the parent concept is used to retain the underlying concept as much as possible to adjust the sentence to achieve targeted knowledge fusion. This fusion method conforms to the syntactic and logical rules in the text structure model to a certain extent. Before using this method, it is necessary to use word2vec to vectorize the words and use the cosine value as the basis for judging the structural alignment.

[0046] ② Macro-level knowledge integration

[0047] Domain knowledge ontology can break through the boundaries of the concept tree, and complete the semantic association of domain knowledge semantics involved in some expert opinions that cannot be mapped by the concept tree through ontology, and then discover implicit knowledge to support macro-level knowledge fusion. Knowledge association supported by domain knowledge ontology can effectively expand the knowledge fusion space and improve the understandability of fusion results.

[0048] By using custom reasoning rules to perform knowledge reasoning, we take the relationship between the expert review opinions of this link and the project's government affairs functions as an example, and design the reasoning rules as shown in formula (3).

[0049]

[0050] Among them, a is the expert review opinion variable of a certain link. The project element associated with the evaluation opinion a is obtained through the ADJUSTMENT_BELONG_TO relationship, and the functional entity to which the project element belongs is obtained through the FUNC_DUTY relationship.

[0051] Finally, the relationship between the evaluation opinion a and the government functions associated with the project elements is expressed as Represents all government affairs sub-functions under this relationship.

[0052] The relationships obtained by knowledge reasoning are integrated into the macro-level semantics of the project background to make the project knowledge explicit. Fuzzy theory is used to generate evaluation opinions from the perspective of the government functions involved in the project.

[0053] (4) Knowledge fusion test

[0054] In order to specifically complete the validity test of the integration results of this article, a mixed test is carried out using the text content information calculation method and the knowledge unit measurement comparison before and after the integration, which has a double verification effect.

[0055] ①Text content verification based on information calculation

[0056] The Shannon information calculation formula (4) is used to calculate the amount of information contained in the text, where x i is the effective word in the text. The comparison of the calculation results of this formula can illustrate the changes in the effective information content before and after text integration.

[0057]

[0058] ② Knowledge fusion result verification based on knowledge unit measurement

[0059] The calculation formula (5) can be used to calculate the ratio of the number of knowledge units of the comprehensive opinion after text integration to the number of knowledge units before integration. The number of knowledge units obtained by the concept tree semantic mapping and ontology knowledge change of the structured knowledge of the expert group before integration is m. Similarly, the number of knowledge units corresponding to the structured knowledge extraction of the comprehensive opinion after integration is n, and the number of project knowledge units n′ is obtained through ontology knowledge association and reasoning (formula (3)). It can be seen that the value of ContactRatio is greater than 1, which means that the comprehensive opinion after integration has richer project knowledge units than the opinion before integration.

[0060] 4. Description of the drawings

[0061] Figure 1 : Expert Approval Opinion Form for Government Information Project: shows the data source of this achievement;

[0062] Figure 2 :Project domain knowledge concept tree diagram: shows the e-government domain concept tree (part) created by this achievement;

[0063] Figure 3 : Government information project approval knowledge ontology diagram: shows the government information project approval knowledge ontology created by this achievement;

[0064] Figure 4 :Expert opinion acquisition process based on hierarchical semantic recognition Figure: shows the process of knowledge acquisition in this achievement;

[0065] Figure 5 : Expert opinion integration strategy based on knowledge fusion Figure: shows the expert opinion integration strategy in this achievement;

[0066] Figure 6 : Example of knowledge fusion based on SAO alignment: shows the knowledge fusion method in this achievement. 5. Specific implementation methods

[0067] The implementation of this achievement is to realize a machine integration method for multi-expert review opinions of a government informationization project. With the support of the basic corpus of expert review opinions of 214 provincial government informationization projects from 2017 to 2018, 1,211 opinions were tested for actual operation effects. The practical results show that this method can greatly improve the objectivity and generation efficiency of expert group opinions through specific knowledge representation, language and text processing technology, hierarchical semantic recognition of expert review opinions and automatic generation of expert group comprehensive opinions based on knowledge fusion. The above process designed in this achievement can theoretically support the integration of project review opinions in different fields, and its framework has a certain degree of universality.

[0068] The domain ontology of this achievement is stored in Neo4j based on the Cypher language; the experiment is based on the Python language and is implemented in a Windows environment using the pyhanlp package provided by the open source tool Hanlp, Google's open source deep learning framework Tensorflow and its advanced API - keras.

Claims

1. A machine integration method for multi-expert opinions on e-government projects based on knowledge fusion, characterized in that: include: S1. Use the standards and management specifications of government information projects to establish the knowledge ontology of e-government project approval, and fully represent the project background knowledge; establish a project knowledge concept tree to perform semantic recognition of entities in expert opinion texts at different conceptual levels; S2. Based on the knowledge acquisition of short texts of individual expert evaluation opinions, hierarchical semantic recognition of the acquired concepts is performed with the support of the project knowledge concept tree, and the sentiment classifier constructed by the deep learning model is used to obtain the sentiment polarity of the expert opinion sentences; S3. Generate comprehensive evaluation opinion knowledge based on parent concepts by using the knowledge fusion method of fuzzy theory, and complete the knowledge fusion of multiple individual expert evaluation opinion sentences by using the knowledge fusion method of SAO structure alignment of expert opinions with the same concept node and the same sentiment tendency, so as to form comprehensive knowledge of expert group opinions at the text level; S4: Based on the comprehensive knowledge of opinions obtained after processing in S3, the project approval knowledge ontology is used for knowledge association reasoning to obtain richer project knowledge units. Secondly, the semantics of domain knowledge involved in some expert opinions that cannot be mapped by the concept tree are semantically associated through the ontology, and implicit knowledge is discovered to support knowledge fusion at the macro level, and finally the expert group opinion text is generated; S5. Conduct a double validity test on the results of expert opinion integration, that is, conduct a mixed test by comparing the text content information calculation method with the knowledge unit measurement method before and after integration.

2. The method for machine integration of multiple expert opinions for e-government projects based on knowledge fusion according to claim 1, characterized in that: For the subject and object of the SAO structured knowledge extracted from the individual expert review opinion sentences, the method of precise matching and fuzzy matching based on the maximum similarity of text is adopted to comprehensively realize the concept semantic mapping based on the project knowledge concept tree and complete the hierarchical semantic recognition; for the acquisition of unstructured knowledge in the expert opinion short text, when the calibrated dependency relationship is too complex or wrong, the original sentence is segmented and the part of speech is screened, and nouns and gerunds are extracted to locate the concept candidate words; verbs and adjectives are extracted to locate the modifying candidate words, indicating the processing of the internal semantic relationship; When there are two core dependency relationships, but the original sentence is a compound sentence containing a subject clause or an object clause, the SAO structured knowledge of the subject clause or the object clause is used as the subject or object of the compound sentence to complete the SAO structured knowledge of the compound sentence; when the dependency relationship is incorrectly marked and the noun modifier is marked as a parallel object instead of the attributive relationship of the predicate, the objects of the two are merged into one object to form a new SAO structured knowledge, and its internal semantic inclusion relationship is reflected in the knowledge, and the semantic recognition method of SAO structured knowledge is further used to complete semantic recognition.

3. The method for machine integration of multiple expert opinions for e-government projects based on knowledge fusion according to claim 2 is characterized by: The specific method of integrating multiple expert opinions includes: step ① on the basis of opinion mining of hierarchical semantic recognition, the comprehensive evaluation opinions of the parent concept based on fuzzy theory are realized through the three steps of fuzzification, fuzzy fusion and clarification of sub-concept opinions, that is, the review opinions are converted from "word-vector-word" based on fuzzy set theory to obtain a comprehensive evaluation statement with high-level semantics; step ② the expert opinions of the same concept node and the same sentiment tendency are aligned in SAO structure to complete knowledge fusion, and the hierarchical semantic relationship between sub-concepts and parent concepts is used in the fusion to complete knowledge fusion in a targeted manner.

Images