Evaluation index selection method and system based on evaluation key point understanding
By using a method based on understanding the key evaluation points, the system automatically analyzes the compatibility of the evaluation indicator system with the data, generates evaluation indicator schemes, solves the problems of low efficiency and omissions in manual generation, and achieves efficient and low-cost generation of evaluation indicators.
Patent Information
- Application Number
- CN202410958036.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-17
- Publication Date
- 2024-11-12
AI Technical Summary
In existing technologies, manually generated evaluation index schemes are inefficient and prone to omissions, and cannot efficiently and adaptively generate evaluation index schemes.
By using a method based on understanding the key evaluation points, the key evaluation points are decomposed, and evaluation index schemes are generated by using demand matching and text similarity calculation. The suitability of the evaluation index system and data is automatically analyzed to generate the final evaluation index scheme.
It reduces the workload of manual labor, improves the efficiency of generating evaluation indicator schemes, reduces the possibility of omissions, and lowers the cost of system updates.
Smart Images

Figure CN118941106A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of data evaluation and analysis, and in particular to a method and system for selecting evaluation indicators based on understanding of evaluation key points. Background Art
[0002] In order to evaluate the ability, effect, efficiency, etc. of evaluation objects such as things and activities, understand the advantages and disadvantages of the evaluation objects, and find the direction of improvement of the evaluation objects, so as to optimize and improve the ability, effect, and efficiency of the evaluation objects. At present, the evaluation index system for evaluation objects such as things and activities is manually generated based on the evaluation points and evaluation data. Although the manual combing method can accurately understand the evaluation points, the manual review of the evaluation index system and evaluation data elements of previous years is burdensome, time-consuming, and inefficient for complex evaluation points and evaluation data elements. It is also easy to miss indicators due to negligence. It is necessary to find a new method to adaptively generate evaluation index schemes based on evaluation points.
[0003] At present, the similarity calculation method for simple texts is relatively mature, and the evaluation requirements for the same type of things, activities and other evaluation objects are relatively stable. For evaluation data with greater uncertainty, it can also be screened by the type of evaluation object, and then the screened evaluation data elements can be extracted to accurately screen the evaluation indicators. Therefore, we should continue to accumulate evaluation needs and evaluation indicator systems over the years, form a demand library and an evaluation indicator system library respectively, and add new evaluation needs and evaluation indicators to the corresponding library, continuously enrich and improve the demand library and evaluation indicator system library, and study a set of knowledge based on the demand library and evaluation indicator system library, analyze the evaluation points, and adaptively generate evaluation indicator solutions. Summary of the invention
[0004] The purpose of the present invention is to provide an evaluation indicator selection method and system based on the understanding of evaluation key points to address the problems of low work efficiency and easy omission of indicator items in the process of generating an evaluation indicator scheme due to manual review of the evaluation data supporting the calculation of indicators based on evaluation key points, sorting and selecting the evaluation indicators to generate the evaluation indicator scheme, etc., to analyze the compatibility of the evaluation key points with the evaluation indicator system and the indicators supported by the current evaluation data, and comprehensively generate a computable evaluation indicator scheme that meets the evaluation key points.
[0005] The present invention discloses a method for selecting an evaluation index based on understanding of evaluation key points, which comprises:
[0006] Step 1: Read and parse the evaluation key points data, extract all evaluation objects in the evaluation key points and the evaluation requirements corresponding to each evaluation object, decompose the evaluation key points according to the evaluation objects, and form multiple evaluation sub-key points;
[0007] Step 2: For each evaluation sub-point, use the demand matching method to preliminarily select the indicator system, compare it with the data support provided by the evaluation data, generate an evaluation indicator sub-scheme, and integrate all evaluation indicator sub-schemes to obtain the final adapted evaluation indicator scheme.
[0008] Furthermore, the step 2 comprises:
[0009] Step 21: Evaluate the sub-point requirements extraction;
[0010] Step 22: Generate labels for the evaluation indicator system;
[0011] Step 23: Calculate the compatibility of the evaluation sub-points with the evaluation indicator system;
[0012] Step 24: Generate evaluation indicator sub-scheme;
[0013] Step 25: Check whether the current evaluation sub-point is the last one. If not, repeat steps 21 to 25. If the current evaluation sub-point is the last one, output all evaluation indicator sub-schemes. Combine all evaluation indicator sub-schemes to generate an evaluation indicator scheme that meets the evaluation point requirements.
[0014] Furthermore, the step 21 comprises:
[0015] Select the evaluation sub-point, read the evaluation requirements of the evaluation sub-point, compare with the accumulated requirements library, extract the evaluation requirements, add them to the evaluation objects to generate an evaluation requirements list, and mark all requirement matching values in the list as 0.
[0016] Further, the step 22 comprises:
[0017] For all constructed evaluation indicator systems, select secondary indicators and evaluation objects applicable to the indicator system as the evaluation indicator system label list, and mark all label matching values in the list as 0.
[0018] Further, the step 23 comprises:
[0019] For the evaluation objects of the evaluation sub-points and the evaluation objects used in the evaluation index system, text similarity calculation is performed, the text similarity calculation results are recorded, the threshold value Y1 is set, and the evaluation index systems with similarities higher than the threshold value Y1 are screened out, and a pre-screened evaluation index system list is generated. The evaluation object label matching values used by all the evaluation index systems in the pre-screened evaluation index system list are marked as 1, and the evaluation object matching values in the evaluation sub-point requirement list are marked as 1.
[0020] Furthermore, the step 23 further includes:
[0021] Select an evaluation indicator system in order from the pre-screening evaluation indicator system list, then select a label with a matching value of 0 in the label list of the evaluation indicator system in order, compare the label with the requirements with a matching value of 0 in the evaluation sub-point requirement list one by one, calculate the text similarity, set a threshold value Y2, select the highest similarity and compare it with the threshold value Y2, record the highest similarity that exceeds the threshold value Y2, and mark the matching value of the evaluation indicator system label corresponding to the highest similarity and the corresponding evaluation sub-point requirement as 1;
[0022] After traversing and executing the matching calculation of all evaluation indicator systems and their labels in the pre-screened evaluation indicator system list with the evaluation sub-key point requirements, a matching calculation result matrix list is generated. Each matrix in the list is a matching calculation result matrix of the label of an evaluation indicator system and the evaluation sub-key point requirements. The columns of the matrix represent the evaluation indicator system label dimensions, and the rows represent the evaluation sub-key point requirement dimensions. The values of the matrix are the highest similarity values recorded in the matching calculation that exceed the threshold value Y2. If there is no match, it is set to 0.
[0023] Furthermore, for the matrix list of matching calculation results, each matrix in the list is selected in turn, and based on the evaluation sub-point requirement dimension, all similarity values of the same requirement row are added and then multiplied by the weight, and all weighted values are added to obtain the adaptability calculation result of the evaluation index system. According to the adaptability calculation result, the evaluation index system is arranged from large to small, and the evaluation index system corresponding to the maximum value is selected as the applicable evaluation index system candidate. When the maximum value corresponds to multiple evaluation index systems, all corresponding evaluation index systems are used as applicable evaluation index system candidates to generate a list of applicable evaluation index system candidates.
[0024] Further, the step 24 comprises:
[0025] According to the evaluation object type of the evaluation sub-point, query the corresponding evaluation data, extract the evaluation data elements, select the candidate evaluation indicator system in the candidate list of applicable evaluation indicator systems in turn, and mark the computability of all underlying evaluation indicators as 0; traverse the data source required for the calculation of each underlying indicator in the candidate evaluation indicator system, and compare it with the list of evaluation data elements. If the list of evaluation data elements contains the data source required for the calculation of the underlying indicator, mark the computability of the underlying indicator as 1, and record the number of underlying indicators with computability of 1 in all candidate evaluation indicator systems; after the comparison and calculation of all candidate evaluation indicator systems are completed, arrange the candidate evaluation indicator systems from large to small according to the number of underlying indicators with computability of 1, select the evaluation indicator system corresponding to the maximum value, add all underlying indicators with computability of 1 and their associated upper indicators to the indicator list, associate them with the evaluation sub-points, and generate an evaluation indicator sub-scheme.
[0026] The present invention also discloses an evaluation index selection system based on understanding of evaluation key points, which implements the above-mentioned evaluation index selection method based on understanding of evaluation key points, and comprises:
[0027] The evaluation key point decomposition module is used to read and parse the evaluation key point data, extract all the evaluation objects in the evaluation key points and the evaluation requirements corresponding to each evaluation object, and decompose the evaluation key points according to the evaluation objects to form multiple evaluation sub-points;
[0028] The evaluation indicator scheme generation module is used to preliminarily select the indicator system for each evaluation sub-point using the demand matching method, compare it with the data support provided by the evaluation data, generate an evaluation indicator sub-scheme, and integrate all evaluation indicator sub-schemes to obtain the final adapted evaluation indicator scheme.
[0029] Due to the adoption of the above technical solution, the present invention has the following advantages:
[0030] 1. It can reduce the manual burden, improve the efficiency of generating evaluation indicator schemes, and reduce the possibility of missing items in evaluation indicators. The present invention analyzes and decomposes the evaluation points, extracts the evaluation point requirements, and performs adaptability calculation analysis with the evaluation indicator system labels to obtain the most suitable evaluation indicator system, and then generates the final evaluation indicator scheme by analyzing the support of the relevant evaluation data for the evaluation indicator calculation. Compared with manual analysis, the present invention is an automated process, and there is no need to manually check the evaluation indicator system data and evaluation data, which greatly reduces the manual burden and improves the efficiency of generating evaluation indicator schemes. The algorithm automatically associates and identifies the evaluation indicators and evaluation data, which can reduce the possibility of missing items in the evaluation indicators due to human errors to a certain extent.
[0031] 2. It can reduce system cost and is suitable for promotion. The present invention mainly updates the software algorithm and does not need to involve hardware and other contents, which can reduce the update cost of the system. The algorithm flow is fixed, and the parameter column can be directly added to the existing algorithm to increase the number of loop processing. It is easy to modify and suitable for promotion. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.
[0033] Figure 1 The present invention is a flowchart of a method for selecting an evaluation index based on understanding of evaluation key points according to an embodiment of the present invention. DETAILED DESCRIPTION
[0034] The present invention is further described in conjunction with the accompanying drawings and embodiments, and the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field should fall within the scope of protection of the embodiments of the present invention.
[0035] See also Figure 1 The present invention provides an embodiment of an evaluation index selection method based on understanding of evaluation key points, which comprises the following steps:
[0036] Step 1: Analyze the key points of the evaluation. Read the key point data and parse it, extract all the evaluation objects in the key points and the evaluation requirements corresponding to each evaluation object, and decompose the key points according to the evaluation objects to form multiple sub-key points of the evaluation.
[0037] Step 2: Extract evaluation sub-point requirements. Select the evaluation sub-point, read the evaluation requirements of the evaluation sub-point, compare with the accumulated requirements library, extract the evaluation requirements, add the evaluation objects to generate the evaluation requirements list, and mark all the requirement matching values in the list as 0.
[0038] Step 3: Generate labels for the evaluation indicator system. For all constructed evaluation indicator systems, select secondary indicators and evaluation objects applicable to the indicator system as the evaluation indicator system label list, and mark all label matching values in the list as 0.
[0039] Step 4: Compatibility calculation between evaluation sub-points and evaluation index system. First, use the semantic similarity method to calculate the text similarity between the evaluation object of the evaluation sub-point and the evaluation object used by the evaluation index system, record the text similarity calculation results, set the threshold value Y1 based on manual experience, screen out the evaluation index systems with similarity higher than the Y1 threshold, generate a pre-screened evaluation index system list, mark the evaluation object label matching value used by all evaluation index systems in the pre-screened evaluation index system list as 1, and mark the evaluation object matching value in the evaluation sub-point requirement list as 1.
[0040] Step 5: Select the evaluation indicator system in the pre-screened evaluation indicator system list in order, and then select the label with a matching value of 0 in the label list of the evaluation indicator system in the order of the label list, compare the label with the requirements with a matching value of 0 in the evaluation sub-point requirement list one by one, calculate the text similarity using the semantic similarity method, set the threshold value Y2 according to manual experience, select the highest similarity and compare it with the Y2 threshold, record this text similarity if it exceeds the threshold, and mark the matching value of the evaluation indicator system label and the corresponding evaluation sub-point requirement as 1. After traversing and executing the matching calculation of all evaluation indicator systems and their labels and evaluation sub-point requirements in the pre-screened evaluation indicator system list, a matching calculation result matrix list is generated. Each matrix in the list is a matching calculation result matrix of the label of an evaluation indicator system and the evaluation sub-point requirement. The columns of the matrix represent the evaluation indicator system label dimension, the rows represent the evaluation sub-point requirement dimension, and the values of the matrix are the highest similarity values recorded in the matching calculation that exceed the Y2 threshold, and are set to 0 if they do not match.
[0041] Step 6: For the list of matrixes with the results of the adaptation calculation, select each matrix in the list at a time, and take the evaluation sub-point requirement dimension as the benchmark, add all the similarity values of the same requirement row and multiply them by the weight. The weight is obtained by the average weight method. Add all the weighted values to obtain the adaptability calculation result of the evaluation index system. Arrange the evaluation index systems from large to small according to the adaptability calculation results, and select the evaluation index system corresponding to the maximum value as the candidate for the applicable evaluation index system. When the maximum value corresponds to multiple evaluation index systems, all corresponding evaluation index systems are used as candidates for the applicable evaluation index system to generate a list of candidates for the applicable evaluation index system.
[0042] Step 7: Generate evaluation indicator sub-schemes. Query the corresponding evaluation data according to the evaluation object type of the evaluation sub-point, and extract the evaluation data elements. Select the candidate evaluation indicator systems in the candidate list of applicable evaluation indicator systems in turn, and mark the computability of all underlying evaluation indicators as 0. Traverse the data sources required for the calculation of each underlying indicator in the candidate evaluation indicator system, and compare them with the list of evaluation data elements. If the list of evaluation data elements contains the data sources required for the calculation of the underlying indicators, mark the computability of the underlying indicator as 1, and record the number of underlying indicators with computability of 1 in all candidate evaluation indicator systems. After the comparison and calculation of all candidate evaluation indicator systems are completed, arrange the candidate evaluation indicator systems from large to small according to the number of underlying indicators with computability of 1, select the evaluation indicator system corresponding to the maximum value, add all underlying indicators with computability of 1 and their associated upper-level indicators to the indicator list, associate them with the evaluation sub-points, and generate an evaluation indicator sub-scheme.
[0043] Step 8: Check whether the current evaluation sub-point is the last one. If not, repeat steps 2 to 8. If it is the last evaluation sub-point, output all evaluation indicator sub-schemes.
[0044] Step 9: Combine all evaluation indicator sub-schemes to generate an evaluation indicator scheme that meets the evaluation requirements.
[0045] The present invention may have various modifications and variations. For example, in addition to using semantic similarity, the method for calculating the matching degree between the evaluation index system label and the evaluation sub-point requirement may also use methods such as edit distance, cosine similarity, and Jaccard similarity. When setting weights, in addition to the average weight in the example of the present invention, a custom weight method may also be used.
[0046] The present invention also provides an embodiment of an evaluation indicator selection system based on understanding of evaluation key points, which implements the evaluation indicator selection method based on understanding of evaluation key points described in the above embodiment, and includes:
[0047] The evaluation key point decomposition module is used to read and parse the evaluation key point data, extract all the evaluation objects in the evaluation key points and the evaluation requirements corresponding to each evaluation object, and decompose the evaluation key points according to the evaluation objects to form multiple evaluation sub-points;
[0048] The evaluation indicator scheme generation module is used to preliminarily select the indicator system for each evaluation sub-point using the demand matching method, compare it with the data support provided by the evaluation data, generate an evaluation indicator sub-scheme, and integrate all evaluation indicator sub-schemes to obtain the final adapted evaluation indicator scheme.
[0049] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.
Claims
1. A method for selecting evaluation indicators based on understanding of evaluation key points, characterized in that: include: Step 1: Read and parse the evaluation key points data, extract all evaluation objects in the evaluation key points and the evaluation requirements corresponding to each evaluation object, decompose the evaluation key points according to the evaluation objects, and form multiple evaluation sub-key points; Step 2: For each evaluation sub-point, use the demand matching method to preliminarily select the indicator system, compare it with the data support provided by the evaluation data, generate an evaluation indicator sub-scheme, and integrate all evaluation indicator sub-schemes to obtain the final adapted evaluation indicator scheme.
2. The method for selecting evaluation indicators based on understanding of evaluation key points according to claim 1, characterized in that: The step 2 comprises: Step 21: Evaluate the sub-point requirements extraction; Step 22: Generate labels for the evaluation index system; Step 23: Calculate the compatibility of the evaluation sub-points with the evaluation indicator system; Step 24: Generate evaluation indicator sub-scheme; Step 25: Check whether the current evaluation sub-point is the last one. If not, repeat steps 21 to 25. If the current evaluation sub-point is the last one, output all evaluation indicator sub-schemes. Combine all evaluation indicator sub-schemes to generate an evaluation indicator scheme that meets the evaluation point requirements.
3. The method for selecting evaluation indicators based on understanding of evaluation key points according to claim 2, characterized in that: The step 21 comprises: Select the evaluation sub-point, read the evaluation requirements of the evaluation sub-point, compare with the accumulated requirements library, extract the evaluation requirements, add them to the evaluation objects to generate an evaluation requirements list, and mark all requirement matching values in the list as 0.
4. The method for selecting evaluation indicators based on understanding of evaluation key points according to claim 2, characterized in that: The step 22 comprises: For all constructed evaluation indicator systems, select secondary indicators and evaluation objects applicable to the indicator system as the evaluation indicator system label list, and mark all label matching values in the list as 0.
5. The method for selecting evaluation indicators based on understanding of evaluation key points according to claim 2, characterized in that: The step 23 comprises: For the evaluation objects of the evaluation sub-points and the evaluation objects used in the evaluation index system, the semantic similarity method is used to calculate the text similarity, the text similarity calculation results are recorded, the threshold value Y1 is set, and the evaluation index systems with similarities higher than the threshold value Y1 are screened out, and a pre-screened evaluation index system list is generated. The evaluation object label matching values used by all the evaluation index systems in the pre-screened evaluation index system list are marked as 1, and the evaluation object matching values in the evaluation sub-point requirement list are marked as 1.
6. The method for selecting evaluation indicators based on understanding of evaluation key points according to claim 2, characterized in that: The step 23 further comprises: Select an evaluation indicator system in order from the pre-screening evaluation indicator system list, then select a label with a matching value of 0 in the label list of the evaluation indicator system in order, compare the label with the requirements with a matching value of 0 in the evaluation sub-point requirement list one by one, calculate the text similarity, set a threshold value Y2, select the highest similarity and compare it with the threshold value Y2, record the highest similarity that exceeds the threshold value Y2, and mark the matching value of the evaluation indicator system label corresponding to the highest similarity and the corresponding evaluation sub-point requirement as 1; After traversing and executing the matching calculation of all evaluation indicator systems and their labels in the pre-screened evaluation indicator system list with the evaluation sub-key point requirements, a matching calculation result matrix list is generated. Each matrix in the list is a matching calculation result matrix of the label of an evaluation indicator system and the evaluation sub-key point requirements. The columns of the matrix represent the evaluation indicator system label dimensions, and the rows represent the evaluation sub-key point requirement dimensions. The values of the matrix are the highest similarity values recorded in the matching calculation that exceed the threshold value Y2. If there is no match, it is set to 0.
7. The method for selecting evaluation indicators based on understanding of evaluation key points according to claim 6, characterized in that: For the matrix list of matching calculation results, select each matrix in the list in turn, take the evaluation sub-point requirement dimension as the benchmark, add all similarity values of the same requirement row and multiply them by the weight, add all weighted values, and get the adaptability calculation result of the evaluation index system. Arrange the evaluation index systems from large to small according to the adaptability calculation results, and select the evaluation index system corresponding to the maximum value as the candidate for the applicable evaluation index system. When the maximum value corresponds to multiple evaluation index systems, all corresponding evaluation index systems are used as candidates for the applicable evaluation index system to generate a candidate list of applicable evaluation index systems.
8. The method for selecting evaluation indicators based on understanding of evaluation key points according to claim 2, characterized in that: The step 24 comprises: According to the evaluation object type of the evaluation sub-point, query the corresponding evaluation data, extract the evaluation data elements, select the candidate evaluation indicator system in the candidate list of applicable evaluation indicator systems in turn, and mark the computability of all underlying evaluation indicators as 0; traverse the data source required for the calculation of each underlying indicator in the candidate evaluation indicator system, and compare it with the list of evaluation data elements. If the list of evaluation data elements contains the data source required for the calculation of the underlying indicator, mark the computability of the underlying indicator as 1, and record the number of underlying indicators with computability of 1 in all candidate evaluation indicator systems; after the comparison and calculation of all candidate evaluation indicator systems are completed, arrange the candidate evaluation indicator systems from large to small according to the number of underlying indicators with computability of 1, select the evaluation indicator system corresponding to the maximum value, add all underlying indicators with computability of 1 and their associated upper indicators to the indicator list, associate them with the evaluation sub-points, and generate an evaluation indicator sub-scheme.
9. An evaluation indicator selection system based on understanding of evaluation key points, implementing the evaluation indicator selection method based on understanding of evaluation key points according to any one of claims 1 to 8, characterized in that: include: The evaluation key point decomposition module is used to read and parse the evaluation key point data, extract all the evaluation objects in the evaluation key points and the evaluation requirements corresponding to each evaluation object, and decompose the evaluation key points according to the evaluation objects to form multiple evaluation sub-points; The evaluation indicator scheme generation module is used to preliminarily select the indicator system for each evaluation sub-point using the demand matching method, compare it with the data support provided by the evaluation data, generate an evaluation indicator sub-scheme, and integrate all evaluation indicator sub-schemes to obtain the final adapted evaluation indicator scheme.
