Innovation performance evaluation method and device, computer equipment and readable storage medium
By constructing an innovation performance knowledge graph and a scoring result interpretation model, the problem of insufficient data fusion and adaptive capabilities in traditional evaluation methods is solved, and causal reasoning and interpretable decision support for multi-source heterogeneous data are realized.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- GUANGZHOU POWER SUPPLY BUREAU GUANGDONG POWER GRID CO LTD
- Filing Date
- 2026-02-11
- Publication Date
- 2026-06-05
Smart Images
Figure CN122155491A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the fields of innovation governance and artificial intelligence technology, and in particular to an innovation performance evaluation method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Technology
[0002] With the in-depth implementation of the innovation-driven development strategy, innovation performance evaluation has become a key indicator for measuring the innovation capabilities of enterprises and research institutions.
[0003] Traditional innovation performance evaluation mainly includes subjective experience-driven multi-indicator evaluation methods and data-driven multi-criteria decision-making methods. Subjective experience-driven multi-indicator evaluation methods are typically based on the Balanced Scorecard or Data Envelopment Analysis (DEA), constructing a multi-dimensional indicator system to evaluate innovation performance. Data-driven multi-criteria decision-making methods, on the other hand, are data-driven at their core, encompassing typical MCDM models such as the Analytic Hierarchy Process (AHP), TOPSIS, and fuzzy comprehensive evaluation, used for modeling and ranking multi-dimensional performance indicators in a big data environment. However, traditional methods have significant limitations: the semantic relationships between evaluation indicators are not modeled, resulting in a relatively fragmented structure and difficulty in integrating data from heterogeneous sources; the scoring process has weak interpretability, relying mainly on correlation analysis and failing to support causal path reasoning and verification; and the model weight settings lack adaptability, making it difficult to cope with dynamic adjustments to evaluation strategies or updates to the indicator system.
[0004] Therefore, how to construct an innovative performance evaluation method that can integrate multi-source heterogeneous data, support causal reasoning, and has adaptive capabilities is an urgent problem to be solved. Summary of the Invention
[0005] Therefore, it is necessary to provide an innovative performance evaluation method, device, computer equipment, computer-readable storage medium, and computer program product that can integrate multi-source heterogeneous data, support causal reasoning, and have adaptive capabilities to address the above-mentioned technical problems.
[0006] Firstly, this application provides a method for evaluating innovation performance, including:
[0007] Obtain multi-source innovation performance data;
[0008] Extract multiple entities and the relationships between them from multi-source innovation performance data; construct an innovation performance knowledge graph based on the multiple entities and their relationships.
[0009] Obtain the innovation performance indicators to be evaluated; calculate multiple similarities between the innovation performance indicators to be evaluated and multiple nodes of the innovation performance knowledge graph; based on the multiple similarities, determine at least one anchor point from the multiple graph nodes;
[0010] Based on at least one anchor point, search the innovation performance knowledge graph to obtain at least one evaluation path; based on at least one evaluation path, obtain at least one score result for the innovation performance indicator to be evaluated.
[0011] Based on at least one evaluation path, construct a scoring result interpretation model; input at least one scoring result into the scoring result interpretation model to generate a visual evaluation result report of the innovation performance indicator to be evaluated.
[0012] In one embodiment, acquiring innovation performance data includes:
[0013] Collect initial innovation performance data from multiple heterogeneous data sources;
[0014] The initial innovation performance data is subjected to semantic normalization, format conversion, and outlier detection and processing operations in sequence to obtain the innovation performance data.
[0015] In one embodiment, determining at least one anchor point from multiple graph nodes based on multiple similarities includes:
[0016] Based on the average similarity and standard deviation of multiple similarities;
[0017] Calculate the dynamic similarity threshold based on the average similarity and the standard deviation of similarity;
[0018] If the similarity and the dynamic similarity threshold satisfy the first preset relationship, the graph node corresponding to the similarity will be used as the anchor point.
[0019] In one embodiment, a scoring result interpretation model is constructed based on at least one evaluation path, including:
[0020] Transform at least one evaluation path into a feature vector;
[0021] The feature vectors are input into the model to be trained to obtain the scoring result interpretation model.
[0022] In one embodiment, at least one scoring result is input into a scoring result interpretation model to generate a visual evaluation result report of the innovation performance indicator to be evaluated, including:
[0023] Input at least one rating result into the rating result interpretation model to obtain the indicator contribution ranking and causal origination diagram of at least one rating result;
[0024] Based on the ranking of indicator contributions and the causal origination diagram, a visual evaluation report of the innovation performance indicators to be evaluated is generated.
[0025] In one embodiment, the method further includes:
[0026] In the event of changes in the innovation performance indicators to be evaluated, update the structure of the innovation performance knowledge graph and calculate the graph edit distance;
[0027] When the graph edit distance and the preset threshold are in a second preset relationship, update the parameters of the scoring result interpretation model.
[0028] Secondly, this application also provides an innovation performance evaluation device, comprising:
[0029] The acquisition module is used to acquire multi-source innovation performance data;
[0030] The extraction module is used to extract multiple entities and the relationships between them from multi-source innovation performance data; and to construct an innovation performance knowledge graph based on the multiple entities and the relationships between them.
[0031] The calculation module is used to obtain the innovation performance indicators to be evaluated; calculate multiple similarities between the innovation performance indicators to be evaluated and multiple nodes of the innovation performance knowledge graph; and determine at least one anchor point from the multiple nodes based on the multiple similarities.
[0032] The scoring module is used to search for at least one evaluation path in the innovation performance knowledge graph based on at least one anchor point; and to obtain at least one scoring result for the innovation performance indicator to be evaluated based on at least one evaluation path.
[0033] The explanation module is used to construct a scoring result explanation model based on at least one evaluation path; input at least one scoring result into the scoring result explanation model to generate a visual evaluation result report of the innovation performance indicator to be evaluated.
[0034] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:
[0035] Obtain multi-source innovation performance data;
[0036] Extract multiple entities and the relationships between them from multi-source innovation performance data; construct an innovation performance knowledge graph based on the multiple entities and their relationships.
[0037] Obtain the innovation performance indicators to be evaluated; calculate multiple similarities between the innovation performance indicators to be evaluated and multiple nodes of the innovation performance knowledge graph; based on the multiple similarities, determine at least one anchor point from the multiple graph nodes;
[0038] Based on at least one anchor point, search the innovation performance knowledge graph to obtain at least one evaluation path; based on at least one evaluation path, obtain at least one score result for the innovation performance indicator to be evaluated.
[0039] Based on at least one evaluation path, construct a scoring result interpretation model; input at least one scoring result into the scoring result interpretation model to generate a visual evaluation result report of the innovation performance indicator to be evaluated.
[0040] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:
[0041] Obtain multi-source innovation performance data;
[0042] Extract multiple entities and the relationships between them from multi-source innovation performance data; construct an innovation performance knowledge graph based on the multiple entities and their relationships.
[0043] Obtain the innovation performance indicators to be evaluated; calculate multiple similarities between the innovation performance indicators to be evaluated and multiple nodes of the innovation performance knowledge graph; based on the multiple similarities, determine at least one anchor point from the multiple graph nodes;
[0044] Based on at least one anchor point, search the innovation performance knowledge graph to obtain at least one evaluation path; based on at least one evaluation path, obtain at least one score result for the innovation performance indicator to be evaluated.
[0045] Based on at least one evaluation path, construct a scoring result interpretation model; input at least one scoring result into the scoring result interpretation model to generate a visual evaluation result report of the innovation performance indicator to be evaluated.
[0046] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:
[0047] Obtain multi-source innovation performance data;
[0048] Extract multiple entities and the relationships between them from multi-source innovation performance data; construct an innovation performance knowledge graph based on the multiple entities and their relationships.
[0049] Obtain the innovation performance indicators to be evaluated; calculate multiple similarities between the innovation performance indicators to be evaluated and multiple nodes of the innovation performance knowledge graph; based on the multiple similarities, determine at least one anchor point from the multiple graph nodes;
[0050] Based on at least one anchor point, search the innovation performance knowledge graph to obtain at least one evaluation path; based on at least one evaluation path, obtain at least one score result for the innovation performance indicator to be evaluated.
[0051] Based on at least one evaluation path, construct a scoring result interpretation model; input at least one scoring result into the scoring result interpretation model to generate a visual evaluation result report of the innovation performance indicator to be evaluated.
[0052] The aforementioned innovation performance evaluation methods, devices, computer equipment, computer-readable storage media, and computer program products first acquire multi-source innovation performance data; extract multiple entities and relationships between these entities from the multi-source innovation performance data; and construct an innovation performance knowledge graph based on these entities and their relationships. In the process of constructing the innovation performance knowledge graph, structured extraction and association modeling of various types of entities, such as organizations, tasks, indicators, and results in innovation activities, and their causal relationships are performed, effectively solving the problems of semantic fragmentation of indicators and difficulty in integrating multi-source data in existing technologies. Secondly, the innovation performance indicators to be evaluated are acquired; multiple similarities between the innovation performance indicators to be evaluated and multiple nodes in the innovation performance knowledge graph are calculated; based on the multiple similarities, at least one anchor point is determined from the multiple graph nodes; based on the at least one anchor point, at least one evaluation path is obtained by searching the innovation performance knowledge graph; and based on the at least one evaluation path, at least one score result for the innovation performance indicator to be evaluated is obtained. The above process, through the path search and relationship reasoning mechanism supported by the innovation performance knowledge graph, can effectively construct a logically closed-loop indicator link, enhancing the transparency and consistency of the evaluation system. Finally, by constructing a scoring result interpretation model based on at least one evaluation path, and inputting at least one scoring result into the scoring result interpretation model, a visualized evaluation result report of the innovative performance indicator to be evaluated is generated. This allows the introduction of interpretability learning algorithms to conduct local and global key factor analysis for each scoring decision of the scoring result interpretation model, providing managers with decision-making basis, indicator attribution, and strategy optimization suggestions. This effectively avoids the cognitive obstacles caused by "black box" models, and ultimately constructs an innovative performance evaluation system that integrates multi-source heterogeneous data, supports causal reasoning, and has adaptive capabilities. Attached Figure Description
[0053] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0054] Figure 1 This is a diagram illustrating the application environment of an innovative performance evaluation method in one embodiment.
[0055] Figure 2 This is a flowchart illustrating an innovative performance evaluation method in one embodiment;
[0056] Figure 3 This is a flowchart illustrating an innovative performance evaluation method in another embodiment;
[0057] Figure 4 This is a structural block diagram of an innovative performance evaluation device in one embodiment;
[0058] Figure 5 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation
[0059] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0060] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.
[0061] The innovative performance evaluation method provided in this application can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or placed on a cloud or other network server. Specifically, terminal 102 or server 104 executes an innovation performance evaluation method, which includes: acquiring multi-source innovation performance data; extracting multiple entities and relationships between them from the multi-source innovation performance data; constructing an innovation performance knowledge graph based on the multiple entities and their relationships; acquiring innovation performance indicators to be evaluated; calculating multiple similarities between the innovation performance indicators to be evaluated and multiple nodes in the innovation performance knowledge graph; determining at least one anchor point from the multiple graph nodes based on the multiple similarities; searching for at least one evaluation path in the innovation performance knowledge graph based on the at least one anchor point; obtaining at least one rating result for the innovation performance indicator to be evaluated based on the at least one evaluation path; constructing a rating result interpretation model based on the at least one evaluation path; and inputting the at least one rating result into the rating result interpretation model to generate a visual evaluation result report of the innovation performance indicator to be evaluated.
[0062] Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, drones, low-altitude aircraft, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, and projection equipment. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted displays. Head-mounted displays can be virtual reality (VR) devices, augmented reality (AR) devices, and smart glasses. Server 104 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.
[0063] In one exemplary embodiment, such as Figure 2 As shown, an innovative performance evaluation method is provided, which is then applied to... Figure 1 Taking terminal 102 as an example, the explanation includes the following steps 202 to 210. Wherein:
[0064] Step 202: Obtain multi-source innovation performance data.
[0065] Among them, multi-source innovation performance data refers to multi-dimensional data collected from multiple heterogeneous data sources and preprocessed, including but not limited to project data, personnel data, organizational data, and achievement data.
[0066] Optionally, multi-dimensional data collected from multiple heterogeneous data sources can be preprocessed to obtain multi-source innovation performance data.
[0067] Step 204: Extract multiple entities and the relationships between them from the multi-source innovation performance data; construct an innovation performance knowledge graph based on the multiple entities and the relationships between them.
[0068] In this context, an entity refers to something that exists independently and does not depend on other things. It is the carrier of attributes and the subject of change, including people, organizations, achievements, projects, and indicators. The relationships between multiple entities can be semantic or causal. The innovation performance knowledge graph covers all elements of innovation activities, including a set of graph nodes and a set of edges. Graph nodes represent entities, and edges represent semantic or causal relationships between entities, used to clarify the semantic hierarchy and causal structure between different indicators.
[0069] For example, named entity recognition (NER) and relation extraction techniques are used to extract entities and their relationships from multi-source innovation performance data, and a set of triples is constructed based on this. The set of triples is denoted as:
[0070]
[0071] in, and It is a physical entity and entity ; It is a physical entity and entity The relationship between them.
[0072] Specifically, a named entity recognition model based on the pre-trained language model (BERT) is adopted. A loss function is used to guide the parameter updates of the BERT model during the fine-tuning phase, enabling it to learn to accurately identify and classify predefined entity types from text. The loss function is:
[0073]
[0074] in, For the sample size, For sequence length, For real labels, The input vector; These are the model parameters.
[0075] Specifically, relation extraction employs a sequence-to-sequence model based on an attention mechanism. Attention weights allow the model to selectively focus on the most relevant parts of the input sequence at each processing step, rather than treating all input information equally. The formula for calculating the attention weights is as follows:
[0076]
[0077] in, To query the dot product of the vector and the key vector.
[0078] Optionally, the set of triples is transformed into an innovation performance knowledge graph by learning node representations using the RotatE graph representation learning algorithm. The innovation performance knowledge graph can be denoted as:
[0079]
[0080] in, It is a set of nodes in the graph, including people, organizations, achievements, projects, and indicators; This is an edge set, representing the semantic or causal relationships between nodes. Additionally, each node... Configured as an attribute vector, i.e. The attributes include the affiliated unit, time, number of times participated, and amount of output.
[0081] Specifically, each entity is represented as a complex vector; each relation is represented as a rotation vector with a magnitude of 1 for triples. Tail entity The vector should be the head entity. The vector through the relation This is obtained from the rotation represented. Based on this, a scoring function can be defined. The lower the score obtained by the scoring function, the higher the probability that the triplet is true. The scoring function is:
[0082]
[0083] in, , It represents the element-wise product of two vectors (Hadamard product).
[0084] Furthermore, based on the true triples selected by the scoring function, the tail entity is randomly replaced. or head entity A batch of fake triples is constructed; a loss function is used for optimization, aiming to maximize the difference in scores between real and fake triples; through this series of training processes, vector representations of all entities and all relations are obtained. These vectors are optimized, semantically rich, and dense representations in complex space, ultimately yielding an innovation performance knowledge graph. The loss function is:
[0085]
[0086] The negative sampling probability is:
[0087]
[0088] Optionally, the innovation performance knowledge graph can be stored in a graph database management system (Neo4j database) that supports the Cypher query language.
[0089] Step 206: Obtain the innovation performance indicators to be evaluated; calculate multiple similarities between the innovation performance indicators to be evaluated and multiple nodes of the innovation performance knowledge graph; based on the multiple similarities, determine at least one anchor point from the multiple graph nodes.
[0090] For example, the Sentence-BERT model is a neural network model specifically designed to generate high-quality sentence embeddings, an improvement on BERT. Using Sentence-BERT, the innovation performance indicators to be evaluated can be... Encode as a vector Calculate the innovation performance indicators to be evaluated With multiple graph nodes of the innovation performance knowledge graph The similarity between them is calculated using the following formula:
[0091]
[0092] Among them, the innovation performance indicators to be evaluated are , These are graph nodes.
[0093] For example, after semantic matching of the innovation performance indicators to be evaluated, the similarity between the innovation performance indicators to be evaluated and the graph nodes is calculated. When the similarity meets the preset conditions, the corresponding graph node is determined as the optimal matching node, and the optimal matching node is the anchor point.
[0094] Step 208: Based on at least one anchor point, search the innovation performance knowledge graph to obtain at least one evaluation path; based on at least one evaluation path, obtain at least one score result for the innovation performance indicator to be evaluated.
[0095] For example, using improved The algorithm improves the search path of the innovation performance knowledge graph based on at least one anchor point. The algorithm is a heuristic algorithm that can find the shortest path from a node to an anchor point in an innovation performance knowledge graph. It guides the search direction by calculating a cost function, which employs a heuristic function. This is used to estimate the minimum cost from the current node to the target node. The cost function needs to comprehensively consider semantic similarity, causal relationship strength, and time decay factors. Specifically, the cost function is:
[0096]
[0097] in, From the starting node to the current node The actual cost, For the current node The estimated cost to reach the target point.
[0098] For example, based on the evaluation path, the scoring results of the innovation performance indicators to be evaluated are calculated using a path scoring function, which is:
[0099]
[0100] in, To evaluate path length, For the time difference, , and These are the weighting coefficients. Causal strength calculation is based on conditional probability, specifically:
[0101]
[0102] Step 210: Based on at least one evaluation path, construct a scoring result interpretation model; input at least one scoring result into the scoring result interpretation model to generate a visual evaluation result report of the innovation performance indicators to be evaluated.
[0103] Among them, the rating result interpretation model uses an interpretability learning algorithm to interpret the rating results.
[0104] For example, after converting the evaluation path into feature vectors, these feature vectors are then input into models such as gradient boosting trees or multilayer perceptrons (MLPs) for joint training. The system integrates local interpretation methods such as game theory-based machine learning model interpretability methods (SHAP, SHapley Additive exPlanations) to obtain a scoring result interpretation model. Based on the scoring result interpretation model, a ranking of indicator contributions and a causal origination graph are provided for each evaluation result. Based on the ranking of indicator contributions and the causal origination graph, a template filling technique is used to automatically generate a visual evaluation result report.
[0105] The aforementioned innovation performance evaluation method first involves acquiring multi-source innovation performance data; extracting multiple entities and relationships between them from this data; and constructing an innovation performance knowledge graph based on these entities and their relationships. In constructing this knowledge graph, a structured extraction and association modeling of various entities, including organizations, tasks, indicators, and results in innovation activities, as well as their causal relationships, is performed, effectively solving the problems of semantic fragmentation of indicators and difficulty in integrating multi-source data in existing technologies. Secondly, the method involves acquiring the innovation performance indicators to be evaluated; calculating multiple similarities between the indicators and multiple nodes in the knowledge graph; determining at least one anchor point from these similarities; searching the innovation performance knowledge graph based on this anchor point to obtain at least one evaluation path; and obtaining at least one score for the innovation performance indicator to be evaluated based on this evaluation path. This process, through the path search and relationship reasoning mechanism supported by the innovation performance knowledge graph, effectively constructs a logically closed-loop indicator chain, enhancing the transparency and consistency of the evaluation system. Finally, by constructing a scoring result interpretation model based on at least one evaluation path, and inputting at least one scoring result into the scoring result interpretation model, a visualized evaluation result report of the innovative performance indicator to be evaluated is generated. This allows the introduction of interpretability learning algorithms to conduct local and global key factor analysis for each scoring decision of the scoring result interpretation model, providing managers with decision-making basis, indicator attribution, and strategy optimization suggestions. This effectively avoids the cognitive obstacles caused by "black box" models, and ultimately constructs an innovative performance evaluation system that integrates multi-source heterogeneous data, supports causal reasoning, and has adaptive capabilities.
[0106] In an exemplary embodiment, obtaining innovation performance data includes: collecting initial innovation performance data from multiple heterogeneous data sources; and sequentially performing semantic normalization, format conversion, and outlier detection and processing operations on the initial innovation performance data to obtain innovation performance data.
[0107] The data sources include project management systems, financial databases, and research output repositories. Initial innovation performance data includes structured, semi-structured, and unstructured data.
[0108] For example, since initial innovation performance data covers multiple dimensions of information such as organization, projects, personnel and results, and the degree of data structuring varies, a unified data interface standard is required for semantic normalization and format conversion.
[0109] For example, outlier detection is performed on the data after semantic normalization and format conversion. For detected outliers, the mean of the same category of data is used for replacement. Specifically, this is done using... Criteria for identifying outliers:
[0110]
[0111] in, For data points, The mean of the data. The standard deviation is denoted as .
[0112] In this embodiment, by performing a series of preprocessing operations on the initial innovation performance data collected from multiple heterogeneous data sources, including semantic normalization, format conversion, and outlier detection and processing, the accuracy and reliability of innovation performance evaluation can be effectively improved, providing a reliable data foundation for the subsequent construction of an innovation performance knowledge graph.
[0113] In one embodiment, determining at least one anchor point from multiple graph nodes based on multiple similarities includes: calculating a dynamic similarity threshold based on the average similarity and standard deviation of the multiple similarities; and using the graph node corresponding to the similarity as the anchor point if the similarity and the dynamic similarity threshold satisfy a first preset relationship.
[0114] The anchor point can be denoted as
[0115] For example, the dynamic similarity threshold is:
[0116]
[0117] The average similarity is The standard deviation of similarity is , It is a regulating factor.
[0118] Optionally, when the similarity exceeds a dynamic similarity threshold, the graph node corresponding to the similarity is used as an anchor point. That is, it satisfies... .
[0119] In this embodiment, by introducing the average similarity and the standard deviation of similarity, and setting an adjustment factor, the graph node that best matches the innovation performance indicator to be evaluated is dynamically selected as the anchor point. This enables the threshold to be adaptively adjusted according to the dispersion of the actual data, thereby improving the accuracy and robustness of the anchor point selection and ensuring a high correlation between the selected anchor point and the indicator to be evaluated. This lays a solid foundation for subsequent anchor point-based path search and scoring calculation.
[0120] In one embodiment, a scoring result interpretation model is constructed based on at least one evaluation path, including: converting at least one evaluation path into a feature vector; inputting the feature vector into a model to be trained for training, thereby obtaining the scoring result interpretation model.
[0121] For example, at least one evaluation path is transformed into a feature vector, and the feature vector is:
[0122]
[0123] in, For index features, Features of knowledge graphs.
[0124] For example, feature vectors are input into a Gradient Boosting Decision Tree (XGBoost) model. The objective function measures the difference between the model's predictions and the true values, and the regularization term of the objective function controls model complexity and prevents overfitting. The objective function of the model in this case is:
[0125]
[0126] The regular expression term is:
[0127]
[0128] In this embodiment, by converting the evaluation path into a feature vector and inputting it into a training model such as Gradient Boosting Decision Tree (XGBoost), a scoring result interpretation model can be effectively constructed. In the process of constructing the scoring result interpretation model, the feature vector can comprehensively reflect the various attributes of the evaluation path, and the choice of the XGBoost model can ensure the accuracy and generalization of the scoring result interpretation model, ultimately resulting in a stable and reliable scoring result interpretation model.
[0129] In one embodiment, at least one scoring result is input into a scoring result interpretation model to generate a visual evaluation result report of the innovation performance indicator to be evaluated, including: inputting at least one scoring result into the scoring result interpretation model to obtain the indicator contribution ranking and causal origination diagram of at least one scoring result; and generating a visual evaluation result report of the innovation performance indicator to be evaluated based on the indicator contribution ranking and causal origination diagram.
[0130] For example, the ranking of key indicator contributions is based on the absolute value of the SHAP value, and the critical path is obtained based on the key indicators. The calculation of the SHAP value is based on cooperative game theory, and features... The SHAP value is:
[0131]
[0132] in, The SHAP value quantifies the influence of each feature on the final score using cooperative game theory. The larger the absolute value, the more significant the contribution of the indicator to the decision result. For feature set, It is a feature subset.
[0133] For example, the critical path is visualized using a force-directed algorithm to generate a causal origin graph. Specifically, the formula for calculating the forces between nodes is:
[0134]
[0135] in, and For node position, For the desired distance, It is the elastic coefficient.
[0136] In this embodiment, by inputting the scoring results into the scoring result interpretation model, the contribution ranking of each indicator can be calculated based on the SHAP value, and a causal origination diagram can be generated through the force-oriented algorithm. This can help managers quickly locate key influencing factors and ensure the interpretability and decision support value of the evaluation results.
[0137] In one embodiment, the method further includes: updating the structure of the innovation performance knowledge graph and calculating the graph edit distance when the innovation performance indicator to be evaluated changes; and updating the parameters of the scoring result interpretation model when the graph edit distance and a preset threshold are in a second preset relationship.
[0138] Among them, graph edit distance is used to detect changes in the innovation performance knowledge graph. The second preset relationship is that the graph edit distance is greater than a preset threshold.
[0139] For example, when the external environment, management strategies, or indicator definitions are adjusted, the graph edit distance is used to detect whether knowledge has changed through graph comparison analysis and strategy awareness mechanisms. The formula for calculating the graph edit distance is:
[0140]
[0141] in, The cost of graph operations (adding or deleting nodes or edges). For weights.
[0142] For example, when the graph edit distance is greater than a preset threshold, the scoring result interpretation model is updated, and the parameters of the scoring result interpretation model are updated using incremental learning:
[0143]
[0144] in, For learning rate, This refers to the data in the current batch.
[0145] In this embodiment, by updating the structure of the innovation performance knowledge graph and calculating the graph edit distance when the innovation performance indicators to be evaluated change, and updating the parameters of the scoring result interpretation model when the graph edit distance and the preset threshold are in a preset relationship, it is possible to achieve adaptive path adjustment and model retraining in response to changes in organizational strategy or data structure evolution, which significantly improves the flexibility and stability of the scoring result interpretation model in actual deployment.
[0146] Next reference Figure 3 The following is a specific embodiment of an innovative performance evaluation method of this application:
[0147] First, innovation performance data is collected from multiple heterogeneous data sources and preprocessed. Then, multiple entities and relationships between these entities are extracted from the preprocessed innovation performance data to construct a set of triples. Finally, graph construction technology is used to transform this set of triples into an innovation performance knowledge graph. Graph nodes represent entities, and edges represent semantic or causal relationships between entities.
[0148] Then, semantic matching is performed on the innovation performance indicators to be evaluated, the similarity between the innovation performance indicators to be evaluated and the nodes of the knowledge graph is calculated, and the optimal matching node is determined as the anchor point. Subsequently, path search is performed in the innovation performance knowledge graph based on the anchor point to generate the evaluation path; the scoring results of the indicators to be evaluated are obtained according to the evaluation path.
[0149] Secondly, based on the evaluation path, a scoring result interpretation model is constructed. Graph edit distance is used to detect whether there have been changes in the external environment, management strategies, or indicator definitions. If changes are detected, the scoring result interpretation model is updated; if no changes are detected, the scoring result interpretation model is maintained.
[0150] Finally, the output of the scoring result interpretation model is received, the results are visualized, and a visual evaluation result report is generated.
[0151] In this embodiment, the process of generating a visualized evaluation result report supports various types of structured and unstructured data input, making it suitable for multiple scenarios. This not only enhances the intelligence level of the performance evaluation process and reduces human bias, but also improves the efficiency and fairness of the evaluation work. Therefore, the innovative performance evaluation method of this application can be widely used in practical applications for tasks such as scientific research management, technological achievement evaluation, and performance analysis of innovative projects, possessing significant potential for technological promotion and socio-economic value.
[0152] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.
[0153] Based on the same inventive concept, this application also provides an innovation performance evaluation device for implementing the innovation performance evaluation method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more embodiments of the innovation performance evaluation device provided below can be found in the limitations of the innovation performance evaluation method described above, and will not be repeated here.
[0154] In one exemplary embodiment, such as Figure 4 As shown, an innovation performance evaluation device 400 is provided, including: an acquisition module 402, an extraction module 404, a calculation module 406, a scoring module 408, and an interpretation module 410, wherein:
[0155] Module 402 is used to acquire multi-source innovation performance data.
[0156] Extraction module 404 is used to extract multiple entities and the relationships between multiple entities from multi-source innovation performance data; and to construct an innovation performance knowledge graph based on the multiple entities and the relationships between the multiple entities.
[0157] The calculation module 406 is used to obtain the innovation performance indicators to be evaluated; calculate multiple similarities between the innovation performance indicators to be evaluated and multiple nodes of the innovation performance knowledge graph; and determine at least one anchor point from the multiple nodes based on the multiple similarities.
[0158] The scoring module 408 is used to search for at least one evaluation path in the innovation performance knowledge graph based on at least one anchor point, and to obtain at least one scoring result of the innovation performance indicator to be evaluated based on at least one evaluation path.
[0159] The explanation module 410 is used to construct a scoring result explanation model based on at least one evaluation path; input at least one scoring result into the scoring result explanation model to generate a visual evaluation result report of the innovation performance indicator to be evaluated.
[0160] In one embodiment, the acquisition module is further configured to collect initial innovation performance data from multi-source heterogeneous data sources; and sequentially perform semantic normalization, format conversion, and outlier detection and processing operations on the initial innovation performance data to obtain innovation performance data.
[0161] In one embodiment, the calculation module further includes: calculating a dynamic similarity threshold based on the average similarity and standard deviation of multiple similarities; and using the graph node corresponding to the similarity as an anchor point when the similarity and the dynamic similarity threshold satisfy a first preset relationship.
[0162] In one embodiment, the scoring module further includes converting at least one evaluation path into a feature vector; inputting the feature vector into the model to be trained for training, thereby obtaining a scoring result interpretation model.
[0163] In one embodiment, the interpretation module further includes constructing a scoring result interpretation model based on at least one evaluation path; inputting at least one scoring result into the scoring result interpretation model to generate a visual evaluation result report of the innovation performance indicator to be evaluated.
[0164] In one embodiment, the innovation performance evaluation device further includes an update module, which is used to update the structure of the innovation performance knowledge graph and calculate the graph edit distance when the innovation performance indicators to be evaluated change; and to update the parameters of the scoring result interpretation model when the graph edit distance and a preset threshold are in a second preset relationship.
[0165] Each module in the aforementioned innovation performance evaluation device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.
[0166] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements an innovative performance evaluation method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.
[0167] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0168] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:
[0169] Obtain multi-source innovation performance data;
[0170] Extract multiple entities and the relationships between them from multi-source innovation performance data; construct an innovation performance knowledge graph based on the multiple entities and their relationships.
[0171] Obtain the innovation performance indicators to be evaluated; calculate multiple similarities between the innovation performance indicators to be evaluated and multiple nodes of the innovation performance knowledge graph; based on the multiple similarities, determine at least one anchor point from the multiple graph nodes;
[0172] Based on at least one anchor point, search the innovation performance knowledge graph to obtain at least one evaluation path; based on at least one evaluation path, obtain at least one score result for the innovation performance indicator to be evaluated.
[0173] Based on at least one evaluation path, construct a scoring result interpretation model; input at least one scoring result into the scoring result interpretation model to generate a visual evaluation result report of the innovation performance indicator to be evaluated.
[0174] In one embodiment, when the processor executes the computer program, it also performs the following steps: collecting initial innovation performance data from multiple heterogeneous data sources; and sequentially performing semantic normalization, format conversion, and outlier detection and processing operations on the initial innovation performance data to obtain innovation performance data.
[0175] In one embodiment, when the processor executes the computer program, it further implements the following steps: based on the average similarity and standard deviation of multiple similarities; calculates a dynamic similarity threshold based on the average similarity and standard deviation of similarity; and, if the similarity and the dynamic similarity threshold satisfy a first preset relationship, uses the graph node corresponding to the similarity as an anchor point.
[0176] In one embodiment, when the processor executes the computer program, it further performs the following steps: converting at least one evaluation path into a feature vector; inputting the feature vector into the model to be trained for training, thereby obtaining a scoring result interpretation model.
[0177] In one embodiment, when the processor executes the computer program, it further performs the following steps: inputting at least one rating result into the rating result interpretation model to obtain the indicator contribution ranking and causal origination diagram of at least one rating result; and generating a visual evaluation result report of the innovation performance indicator to be evaluated based on the indicator contribution ranking and causal origination diagram.
[0178] In one embodiment, when the processor executes the computer program, it further performs the following steps: when the innovation performance indicator to be evaluated changes, updating the structure of the innovation performance knowledge graph and calculating the graph edit distance; when the graph edit distance and a preset threshold are in a second preset relationship, updating the parameters of the scoring result interpretation model.
[0179] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:
[0180] Obtain multi-source innovation performance data;
[0181] Extract multiple entities and the relationships between them from multi-source innovation performance data; construct an innovation performance knowledge graph based on the multiple entities and their relationships.
[0182] Obtain the innovation performance indicators to be evaluated; calculate multiple similarities between the innovation performance indicators to be evaluated and multiple nodes of the innovation performance knowledge graph; based on the multiple similarities, determine at least one anchor point from the multiple graph nodes;
[0183] Based on at least one anchor point, search the innovation performance knowledge graph to obtain at least one evaluation path; based on at least one evaluation path, obtain at least one score result for the innovation performance indicator to be evaluated.
[0184] Based on at least one evaluation path, construct a scoring result interpretation model; input at least one scoring result into the scoring result interpretation model to generate a visual evaluation result report of the innovation performance indicator to be evaluated.
[0185] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: collecting initial innovation performance data from multiple heterogeneous data sources; and sequentially performing semantic normalization, format conversion, and outlier detection and processing operations on the initial innovation performance data to obtain innovation performance data.
[0186] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: In one embodiment, when the processor executes the computer program, it further performs the following steps: Based on the average similarity and standard deviation of multiple similarities; calculate a dynamic similarity threshold based on the average similarity and standard deviation of similarity; and, if the similarity and the dynamic similarity threshold satisfy a first preset relationship, use the graph node corresponding to the similarity as an anchor point.
[0187] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: converting at least one evaluation path into a feature vector; inputting the feature vector into the model to be trained for training, and obtaining a scoring result interpretation model.
[0188] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: inputting at least one rating result into the rating result interpretation model to obtain the indicator contribution ranking and causal origination diagram of at least one rating result; and generating a visual evaluation result report of the innovation performance indicator to be evaluated based on the indicator contribution ranking and causal origination diagram.
[0189] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: updating the structure of the innovation performance knowledge graph and calculating the graph edit distance when the innovation performance indicator to be evaluated changes; and updating the parameters of the scoring result interpretation model when the graph edit distance and a preset threshold are in a second preset relationship.
[0190] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:
[0191] Obtain multi-source innovation performance data;
[0192] Extract multiple entities and the relationships between them from multi-source innovation performance data; construct an innovation performance knowledge graph based on the multiple entities and their relationships.
[0193] Obtain the innovation performance indicators to be evaluated; calculate multiple similarities between the innovation performance indicators to be evaluated and multiple nodes of the innovation performance knowledge graph; based on the multiple similarities, determine at least one anchor point from the multiple graph nodes;
[0194] Based on at least one anchor point, search the innovation performance knowledge graph to obtain at least one evaluation path; based on at least one evaluation path, obtain at least one score result for the innovation performance indicator to be evaluated.
[0195] Based on at least one evaluation path, construct a scoring result interpretation model; input at least one scoring result into the scoring result interpretation model to generate a visual evaluation result report of the innovation performance indicator to be evaluated.
[0196] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: collecting initial innovation performance data from multiple heterogeneous data sources; and sequentially performing semantic normalization, format conversion, and outlier detection and processing operations on the initial innovation performance data to obtain innovation performance data.
[0197] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: based on the average similarity and standard deviation of multiple similarities; calculates a dynamic similarity threshold based on the average similarity and standard deviation of similarity; and, if the similarity and the dynamic similarity threshold satisfy a first preset relationship, uses the graph node corresponding to the similarity as an anchor point.
[0198] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: converting at least one evaluation path into a feature vector; inputting the feature vector into the model to be trained for training, and obtaining a scoring result interpretation model.
[0199] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: inputting at least one rating result into the rating result interpretation model to obtain the indicator contribution ranking and causal origination diagram of at least one rating result; and generating a visual evaluation result report of the innovation performance indicator to be evaluated based on the indicator contribution ranking and causal origination diagram.
[0200] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: updating the structure of the innovation performance knowledge graph and calculating the graph edit distance when the innovation performance indicator to be evaluated changes; and updating the parameters of the scoring result interpretation model when the graph edit distance and a preset threshold are in a second preset relationship.
[0201] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.
[0202] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.
[0203] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.
[0204] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.
Claims
1. An innovative performance evaluation method, characterized in that, The method includes: Obtain multi-source innovation performance data; Extract multiple entities and the relationships between them from the multi-source innovation performance data; construct an innovation performance knowledge graph based on the multiple entities and the relationships between them; Obtain the innovation performance indicator to be evaluated; calculate multiple similarities between the innovation performance indicator to be evaluated and multiple graph nodes of the innovation performance knowledge graph; based on the multiple similarities, determine at least one anchor point from the multiple graph nodes; Based on the at least one anchor point, at least one evaluation path is obtained in the innovation performance knowledge graph search path; based on the at least one evaluation path, at least one score result of the innovation performance indicator to be evaluated is obtained; Based on the at least one evaluation path, a scoring result interpretation model is constructed; the at least one scoring result is input into the scoring result interpretation model to generate a visual evaluation result report of the innovation performance indicator to be evaluated.
2. The method according to claim 1, characterized in that, The acquisition of multi-source innovation performance data includes: Collect initial innovation performance data from multiple heterogeneous data sources; The initial innovation performance data is sequentially subjected to semantic normalization, format conversion, and outlier detection and processing operations to obtain multi-source innovation performance data.
3. The method according to claim 1, characterized in that, The step of determining at least one anchor point from the plurality of map nodes based on the plurality of similarities includes: Based on the average similarity and standard deviation of the multiple similarities; Based on the average similarity and the standard deviation of similarity, the dynamic similarity threshold is calculated; If the similarity and the dynamic similarity threshold satisfy a first preset relationship, the graph node corresponding to the similarity is used as an anchor point.
4. The method according to claim 1, characterized in that, The construction of a scoring result interpretation model based on the at least one evaluation path includes: Transform the at least one evaluation path into a feature vector; The feature vectors are input into the model to be trained to obtain the scoring result interpretation model.
5. The method according to claim 1, characterized in that, The step of inputting the at least one scoring result into the scoring result interpretation model to generate a visual evaluation result report of the innovation performance indicator to be evaluated includes: Input the at least one rating result into the rating result interpretation model to obtain the indicator contribution ranking and causal origination diagram of the at least one rating result; Based on the contribution ranking of the indicators and the causal origination diagram, a visual evaluation result report of the innovation performance indicators to be evaluated is generated.
6. The method according to claim 1, characterized in that, The method further includes: In the event of changes in the innovation performance indicators to be evaluated, the structure of the innovation performance knowledge graph is updated and the graph edit distance is calculated; When the graph editing distance and the preset threshold are in a second preset relationship, the parameters of the scoring result interpretation model are updated.
7. An innovative performance evaluation device, characterized in that, The device includes: The acquisition module is used to acquire multi-source innovation performance data; An extraction module is used to extract multiple entities and the relationships between the multiple entities from the multi-source innovation performance data; and to construct an innovation performance knowledge graph based on the multiple entities and the relationships between the multiple entities. The calculation module is used to obtain the innovation performance indicators to be evaluated; calculate multiple similarities between the innovation performance indicators to be evaluated and multiple graph nodes of the innovation performance knowledge graph; and determine at least one anchor point from the multiple graph nodes based on the multiple similarities. The scoring module is used to obtain at least one evaluation path based on the at least one anchor point in the innovation performance knowledge graph search path; and to obtain at least one scoring result of the innovation performance indicator to be evaluated based on the at least one evaluation path. An interpretation module is used to construct a scoring result interpretation model based on the at least one evaluation path; input the at least one scoring result into the scoring result interpretation model to generate a visual evaluation result report of the innovation performance indicator to be evaluated.
8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.
10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.