Evaluation method, device, equipment and medium based on type evaluation system
Through the evaluation method based on the type evaluation system, the problem of inefficiency in traditional type inspection is solved. The evaluation scores are calculated through data screening, grouping and feature extraction, and combined with expert evaluation, the evaluation accuracy and comprehensiveness of the type evaluation system are improved.
Patent Information
- Application Number
- CN202210027000.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-01-11
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2042-01-11
AI Technical Summary
Due to irregular operations and complex work processes in traditional type inspection work, it leads to inefficiency, overdue tasks and unclear division of responsibilities, and lacks information-based inspection and evaluation methods.
The evaluation method based on the type evaluation system is adopted to obtain and filter the data to be evaluated, feature data are extracted in groups, match degree and evaluation scores are calculated, and target evaluation results are output in combination with expert evaluation, so as to improve the accuracy and comprehensiveness of the evaluation.
The comprehensive and accurate evaluation of evaluation data is achieved, the evaluation accuracy and efficiency of the type evaluation system are improved, and the comprehensiveness of the automatic system evaluation and expert manual evaluation are comprehensively considered.
Smart Images

Figure CN114372087B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of inspection and evaluation technology, and in particular to an evaluation method, device, computer equipment and computer storage medium based on a type evaluation system. Background Art
[0002] Inspection and testing work mainly includes type inspection, supervisory inspection, and commissioned inspection. Due to the non-standard operation and complex workflow of traditional type evaluation work, its working mode has always remained in the relatively primitive state of manual transfer and manual certification. This working method has caused a series of problems such as low work efficiency, serious task overdue, and unclear division of responsibilities. How to provide an information-based type evaluation method for inspection work is an urgent problem to be solved. Summary of the invention
[0003] Based on this, it is necessary to propose an evaluation method, device computer equipment and computer storage medium based on the type evaluation system to address the above problems.
[0004] In the present invention, an evaluation method based on a type evaluation system is provided, the method comprising:
[0005] Acquire the data to be evaluated input by the user through the type evaluation system, wherein the data to be evaluated is the inspection data of whether the measuring equipment to be evaluated of the user to be evaluated meets the measurement standard, and the data to be evaluated includes multiple pieces of data;
[0006] Screening each piece of data to be evaluated according to a preset data format to determine whether each piece of data to be evaluated meets the format requirements corresponding to each piece of data to be evaluated;
[0007] When the data to be evaluated meets the format requirements, the data to be evaluated are grouped;
[0008] For each data group, extract at least one feature data corresponding to the data to be evaluated in each data group;
[0009] Determine the standard data corresponding to each data group and the first weight coefficient corresponding to each data group, and under each data group, calculate the matching degree between the standard data and the characteristic data, wherein the matching degree is used to identify whether the characteristic data meets the requirements of the standard data; determine the evaluation sub-score corresponding to each data group according to the matching degree;
[0010] Calculate the first evaluation score corresponding to the data to be evaluated according to the first weight coefficient and the evaluation sub-score corresponding to each data group;
[0011] Obtaining a second evaluation score input by the expert through the model evaluation system;
[0012] A second weight coefficient is determined according to the user to be evaluated and the expert, and a target evaluation score is calculated according to the second weight coefficient and the first evaluation score and the second evaluation score; and an evaluation result corresponding to the target evaluation score is output.
[0013] Optionally, the method further includes:
[0014] Obtain historical evaluation data of the user to be evaluated corresponding to the data to be evaluated in the evaluation system, and determine the confidence level corresponding to the user to be evaluated based on the historical evaluation data;
[0015] Determine the standard data requirements corresponding to each data group according to the confidence level, wherein different confidence levels may correspond to different standard data requirements;
[0016] The step of calculating the matching degree between the standard data and the characteristic data for each data group, wherein the matching degree is used to identify whether the characteristic data meets the requirements of the standard data, and determining the evaluation sub-score corresponding to each data group according to the matching degree, further includes:
[0017] According to the requirements of different standard data determined by the confidence level, it is determined whether the feature data meets the requirements of the standard data and the degree of matching is determined.
[0018] Optionally, the step of grouping the data to be evaluated further includes:
[0019] Performing a first grouping process on the data to be evaluated to obtain a plurality of first data groups, wherein the data to be evaluated in the plurality of first data groups obtained by the first grouping process do not overlap;
[0020] Using a first data processing algorithm, performing a first screening operation on the data to be evaluated in each first data group according to the importance value;
[0021] Performing a second grouping process on the data to be evaluated in the first data group after the first screening, so as to obtain a plurality of second data groups under each first data group, wherein the data to be evaluated in the plurality of second data groups obtained by the second grouping process may overlap;
[0022] A second screening operation is performed on the data to be evaluated in the second data group according to the data characteristic values.
[0023] Optionally, the step of performing a second grouping process on the data to be evaluated in the first data group after the first screening to obtain a plurality of second data groups under each first data group further includes:
[0024] Performing a combination operation on the plurality of pieces of data to be evaluated in the first data group after the first screening, wherein each combined data packet obtained by the combination operation contains one or more pieces of data to be evaluated;
[0025] Performing a second grouping process on the combined data under the first data group to obtain a plurality of second data groups;
[0026] The step of performing a second screening operation on the data to be evaluated in the second data group according to the data characteristic value also includes:
[0027] The combined features corresponding to each combined data are calculated as data feature values, and a second screening operation is performed on the combined data in the second data group according to the data feature values.
[0028] Optionally, the step of calculating the degree of match between the standard data and the characteristic data in each data group further includes:
[0029] Perform prediction analysis on each combined feature to obtain the predicted value corresponding to the combined feature; calculate the predicted residual value based on the combined feature and the predicted value;
[0030] Calculate the negative entropy corresponding to the predicted residual value, and calculate the contribution value corresponding to the combined feature based on the calculated negative entropy;
[0031] The similarity between the standard data and the combined data is calculated, and the matching degree between the standard data and the feature data is calculated based on the contribution value and similarity of the combined features.
[0032] Optionally, the first evaluation score and the second evaluation score each include a plurality of evaluation sub-scores;
[0033] The step of determining a second weight coefficient and calculating a target evaluation score according to the second weight coefficient and the first evaluation score and the second evaluation score also includes:
[0034] Performing cross-fusion processing on multiple evaluation sub-scores included in the first evaluation score to obtain a first target score after the cross-fusion processing;
[0035] Performing cross-fusion processing on multiple evaluation sub-scores included in the second evaluation score to obtain a second target score after the cross-fusion processing;
[0036] The first target score and the second target score are cross-fused according to the second weight coefficient to obtain the target evaluation score.
[0037] Optionally, the step of calculating the first evaluation score corresponding to the data to be evaluated according to the first weight coefficient and the evaluation sub-score corresponding to each data group further includes:
[0038] Constructing an evaluation sub-model corresponding to each data group, and performing superposition processing on the sub-model corresponding to each data group according to a first weight coefficient and a preset superposition algorithm to construct a first evaluation model;
[0039] Based on the first evaluation model, taking the evaluation sub-score corresponding to each data group as input, obtaining the first evaluation score;
[0040] The step of obtaining the second evaluation score input by the expert through the model evaluation system also includes:
[0041] Calculate the second evaluation score according to the preset second evaluation model;
[0042] The step of determining a second weight coefficient according to the user to be evaluated and the expert, and calculating a target evaluation score according to the second weight coefficient and the first evaluation score and the second evaluation score, further includes:
[0043] According to a preset superposition algorithm, based on a second weight coefficient, the first evaluation model and the second evaluation model are superimposed to construct a target evaluation model;
[0044] The target evaluation score is calculated according to the target evaluation model using the first evaluation score and the second evaluation score as input.
[0045] In the present invention, an evaluation device based on a type evaluation system is provided, the device comprising:
[0046] A data input module is used to obtain the data to be evaluated input by the user through the type evaluation system, wherein the data to be evaluated is the inspection data of whether the measuring equipment to be evaluated of the user to be evaluated meets the measurement standard, and the data to be evaluated includes multiple pieces of data;
[0047] The data initial screening module is used to screen each piece of data to be evaluated according to a preset data format to determine whether each piece of data to be evaluated meets the format requirements corresponding to each piece of data to be evaluated;
[0048] A data grouping module, used for grouping the data to be evaluated when the data to be evaluated meets the format requirements;
[0049] A feature extraction module, used to extract at least one feature data corresponding to the data to be evaluated in each data group, for each data group;
[0050] A first evaluation score calculation module is used to determine the standard data corresponding to each data group and the first weight coefficient corresponding to each data group, and under each data group, calculate the matching degree between the standard data and the feature data, the matching degree is used to identify whether the feature data meets the requirements of the standard data; determine the evaluation sub-score corresponding to each data group according to the matching degree; calculate the first evaluation score corresponding to the data to be evaluated according to the first weight coefficient and the evaluation sub-score corresponding to each data group;
[0051] A second evaluation score calculation module, used to obtain a second evaluation score input by an expert through the model evaluation system;
[0052] The third evaluation score calculation module is used to determine the second weight coefficient according to the user to be evaluated and the expert, calculate the target evaluation score according to the second weight coefficient and the first evaluation score and the second evaluation score; and output the evaluation result corresponding to the target evaluation score.
[0053] In the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium is used to store a computer program, and the computer program is used to execute the evaluation method based on the type evaluation system as described above.
[0054] In the present invention, a computer device is provided, the computer device comprises a memory and a processor, the memory has executable code, when the executable code runs on the processor to implement the evaluation method based on the type evaluation system as described above.
[0055] The embodiments of the present invention have the following beneficial effects:
[0056] The evaluation method, device, computer equipment and computer storage medium based on the type evaluation system are adopted to obtain the data to be evaluated input by the user through the type evaluation system, wherein the data to be evaluated is the inspection data of whether the metering equipment to be evaluated of the user to be evaluated meets the metering standard, and the data to be evaluated includes multiple data; each data to be evaluated is screened according to a preset data format to determine whether each data to be evaluated meets the format requirements corresponding to each data to be evaluated; when the data to be evaluated meets the format requirements, the data to be evaluated are grouped; for each data group, at least one feature data corresponding to the data to be evaluated in each data group is extracted; and the standard corresponding to each data group is determined. The first weight coefficient corresponding to the standard data and each data group is used to calculate the matching degree between the standard data and the feature data under each data group, and the matching degree is used to identify whether the feature data meets the requirements of the standard data; the evaluation sub-score corresponding to each data group is determined according to the matching degree; the first evaluation score corresponding to the data to be evaluated is calculated according to the first weight coefficient and the evaluation sub-score corresponding to each data group; the second evaluation score input by the expert through the model evaluation system is obtained; the second weight coefficient is determined according to the user to be evaluated and the expert, and the target evaluation score is calculated according to the second weight coefficient and the first evaluation score and the second evaluation score; the evaluation result corresponding to the target evaluation score is output. That is to say, after adopting the evaluation method, device computer equipment and computer storage medium based on the model evaluation system provided by the present invention, the evaluation score corresponding to the data to be evaluated can be comprehensively and accurately calculated for screening, grouping, feature extraction and evaluation score calculation of the data to be evaluated, so as to improve the accuracy of the evaluation of the model evaluation system. In addition, the comprehensiveness of the automatic evaluation of the system and the manual evaluation of the experts can be comprehensively considered to further improve the accuracy of the evaluation of the model evaluation system. BRIEF DESCRIPTION OF THE DRAWINGS
[0057] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0058] in:
[0059] Figure 1 A schematic diagram of a flow chart of an evaluation method based on a type evaluation system in an embodiment;
[0060] Figure 2 is a schematic structural diagram of an evaluation device based on a type evaluation system in one embodiment;
[0061] Figure 3 The structure diagram of a computer device for running the above-mentioned evaluation method based on the type evaluation system in one embodiment is shown. DETAILED DESCRIPTION
[0062] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0063] In this embodiment, a type evaluation system is provided, which can evaluate whether the metrological equipment meets the inspection requirements. The user submits data (data to be evaluated) through the type evaluation system, and then the type evaluation system automatically and experts manually evaluate whether the metrological equipment meets the inspection requirements. After the evaluation of the submitted data to be evaluated is qualified, the user is notified to submit a sample of the metrological equipment to be evaluated, and then the sample is further inspected, thereby completing the type evaluation of the metrological equipment.
[0064] Furthermore, in this embodiment, an evaluation method based on the above-mentioned model evaluation system is provided, which can evaluate whether the data to be evaluated submitted by the user meets the inspection requirements and output the evaluation results.
[0065] For details, see Figure 1 , a flow chart of the above-mentioned evaluation method based on the type evaluation system is given. Among them, the evaluation method based on the type evaluation system includes: Figure 1 Steps S101-S108 are shown.
[0066] Step S101: obtaining the data to be evaluated input by the user through the type evaluation system, wherein the data to be evaluated is the inspection data of whether the measuring equipment to be evaluated of the user to be evaluated meets the measurement standard, and the data to be evaluated includes multiple pieces of data.
[0067] The user submits the data of the metering equipment to be evaluated through the type evaluation system, so as to evaluate whether the metering equipment meets the inspection requirements through the type evaluation system. Among them, the data to be evaluated submitted by the user to the type evaluation system is submitted separately for each item according to the requirements for data submission in the type evaluation system. The data here can be formatted data including multiple attribute items defined by the type evaluation system, or can be submitted as attachments, etc. Therefore, the data to be evaluated here includes multiple data to be evaluated.
[0068] Step S102: Screen each piece of data to be evaluated according to a preset data format to determine whether each piece of data to be evaluated meets the format requirements corresponding to each piece of data to be evaluated.
[0069] Although the user has entered multiple pieces of data to be evaluated, not every piece of data to be evaluated can enter the next step of the evaluation process. Here, each piece of data to be evaluated needs to be screened first to determine the data to be evaluated that enters the next step of the evaluation process. In the event that some necessary data to be evaluated is missing or has problems, the user needs to be reminded to resubmit the data to be evaluated.
[0070] Specifically, each piece of data to be evaluated has a corresponding data format requirement, which is defined by the model evaluation system. When the user inputs the data to be evaluated, each piece of data to be evaluated is screened according to the data format requirement defined by the model evaluation system to determine whether each model evaluation system meets the corresponding data format requirement.
[0071] If the requirements are met, step S103 is executed; if the requirements are not met, the data is returned to the user to resubmit the data to be evaluated.
[0072] Step S103: when the data to be evaluated meets the format requirement, the data to be evaluated are grouped.
[0073] When evaluating the data to be evaluated, it is first necessary to group the data to be evaluated, and then evaluate based on the grouped data to improve the effectiveness and accuracy of the evaluation of the data to be evaluated.
[0074] In a specific embodiment, multiple pieces of data to be evaluated are grouped according to preset grouping requirements to obtain multiple pieces of data to be evaluated corresponding to multiple data groups, wherein each data group contains at least one piece of data to be evaluated, and there is at least one common characteristic or similarity between the multiple pieces of data to be evaluated contained in each data group.
[0075] Furthermore, how to perform grouping is explained in detail.
[0076] First, a first grouping process is performed on a plurality of pieces of data to be evaluated to obtain a plurality of first data groups, wherein the data to be evaluated in different first data groups do not overlap. Here, the first grouping process is a preliminary grouping process of the data to be evaluated to divide the data to be evaluated into a plurality of large groups, and then further refinement processing is performed on the data under the large groups.
[0077] Then, further screening is performed on the data in each first data group. According to the first data processing algorithm, the first screening operation is performed on the data to be evaluated in each first data group according to the importance value; for example, here the importance value of each data to be evaluated can be calculated according to the gradient boosting tree algorithm, and then the first screening operation is performed on the multiple data in each first data group. According to the random sample, the importance value corresponding to each data to be evaluated can be calculated, and then the first screening operation is performed according to whether the calculated importance value is greater than or equal to the preset importance value threshold.
[0078] The data to be evaluated in the first data group after the first screening is subjected to a second grouping process. The second grouping process here is a further subdivision grouping process, which is more refined than the first grouping process, and can perform more refined feature grouping on the data to be evaluated, so that multiple second data groups under each first data group can be obtained, that is, the second data group is obtained by further grouping the data to be evaluated in each first data group, and will not be processed across multiple first data groups. Among them, there may be overlaps between the data to be evaluated in the multiple second data groups obtained by the second grouping process. It should be noted that in the step, during the second grouping process, the data to be evaluated in each first data group will be combined, and the combined data after the combination may contain one or more data to be evaluated. Then, the combined data is grouped. The second grouping process here is to group according to whether a certain feature is contained in the combined feature to obtain a combined feature with the same feature, that is, in the second data group obtained by the second grouping process, each second data group contains multiple combined data obtained according to the data to be evaluated.
[0079] Then, a second screening operation is performed on all the combined data obtained, wherein the second screening operation is performed by calculating the data feature values corresponding to each combined data, wherein the combined data is input into a preset screening model (wherein the model is a neural network model), and then the corresponding data feature values are obtained, and then the combined data screened in the second screening operation is determined based on whether the data feature values meet the preset feature value screening conditions.
[0080] In a specific embodiment, the combined features of the combined data may be extracted by first extracting features corresponding to multiple data to be evaluated in the combined data, and then performing cross-fusion processing on the extracted features. In a specific example, when the combined data includes A1, A2, and A3, the corresponding extracted features are B1, B2, and B3, then the combined feature B is:
[0081] B=contact(f1(B1,B2,B3),f2(B1,B2,B3),f3(B1,B2,B3)),
[0082] Among them, contact is a fusion function, such as a splicing function, an addition function, etc., f1, f2, and f3 are preset feature cross-fusion algorithms, and, in one embodiment, the feature cross-fusion algorithm may not be limited to the three cross-fusion algorithms of f1, f2, and f3 mentioned above, but may also include fewer or more cross-fusion algorithms.
[0083] It should be noted that the second screening operation here can be to calculate the importance value of each combination data according to the gradient boosting tree algorithm, and then perform the second screening operation on each combination data. Among them, the importance value corresponding to each combination data can be calculated based on a random sample, and then the second screening operation is performed based on whether the calculated importance value is greater than or equal to a preset importance value threshold.
[0084] Step S104: for each data group, extract at least one feature data corresponding to the data to be evaluated in each data group.
[0085] After the data to be evaluated is grouped, further evaluation processing operations can be performed.
[0086] Here, it is necessary to further extract corresponding feature data for each data group (the data group here may include a first data group and a second data group), and then perform further evaluation and processing operations based on the extracted feature data.
[0087] For each piece of data to be evaluated, feature data of the data to be evaluated is extracted according to a preset feature extraction algorithm. Here, the feature extraction algorithm may be a feature extraction algorithm defined for each data group, and the feature extraction algorithm corresponding to each data group is different, so as to accurately extract feature data corresponding to each piece of data to be evaluated.
[0088] Furthermore, in the case of the second data group, the characteristic data corresponding to each combination of data may be the aforementioned extracted combination feature (ie, the aforementioned data characteristic value).
[0089] Step S105: Determine the standard data corresponding to each data group and the first weight coefficient corresponding to each data group. Under each data group, calculate the matching degree between the standard data and the characteristic data, and the matching degree is used to identify whether the characteristic data meets the requirements of the standard data; determine the evaluation sub-score corresponding to each data group according to the matching degree.
[0090] In this embodiment, the data in different data groups contribute to the evaluation of the results to different degrees, and have different degrees of influence on whether the evaluation results are qualified. Therefore, for each data group, a different weight coefficient (first weight coefficient) is configured. This first weight coefficient is used to identify the degree of influence of the data in different data groups on the evaluation results.
[0091] Furthermore, different users may also correspond to different standard data requirements. For example, for users of different systems, different types, and different historical data, the corresponding standard data requirements are different; the different standard data referred to here does not mean that the standard data is different, but that the corresponding requirements are different, for example, the confidence corresponding to the standard data is different. Specifically, the historical evaluation data of the user to be evaluated corresponding to the data to be evaluated in the evaluation system is obtained, and the confidence corresponding to the user to be evaluated is determined based on the historical evaluation data; then the requirements of the standard data corresponding to each data group are determined based on the confidence, where different confidences can correspond to different standard data requirements. After calculating and determining the confidence, the matching degree between the standard data and the feature data under each data group can be further calculated. The matching degree here is used to identify whether the feature data meets the requirements of the standard data; that is, this matching degree is also related to the confidence, that is, according to the requirements of different standard data determined by the confidence, it is determined whether the feature data meets the requirements of the standard data and the matching degree is determined. In other words, for different users, the calculation of the corresponding matching degree is also different. The formula for calculating the matching degree is consistent, but the calculated matching degree can be further corrected according to the confidence level to obtain the final matching degree that matches the user.
[0092] Furthermore, after determining the degree of match, the evaluation sub-score corresponding to each data group can be determined based on the degree of match. The evaluation sub-score is the score corresponding to whether the data in the corresponding data group meets the requirements of the standard data, that is, the score obtained according to the requirements of the model evaluation system.
[0093] In a specific embodiment, a prediction analysis is performed for each combined feature to obtain a prediction value corresponding to the combined feature; the prediction value here refers to the prediction value corresponding to feature extraction or result evaluation, and then the prediction residual value is calculated based on the combined feature and the prediction value, and the prediction residual value indicates the loss value between the combined feature and the prediction value. Then, according to a preset entropy function, the negative entropy corresponding to the prediction residual value is calculated, and further, the contribution value corresponding to the combined feature is calculated based on the calculated negative entropy; the contribution value here refers to the degree of influence of the combined feature on the result.
[0094] The similarity between the standard data and the combined data is calculated, and the matching degree between the standard data and the feature data is calculated based on the contribution value and similarity of the combined feature. Here, the contribution value can correct the similarity to obtain the final matching degree.
[0095] Step S106: Calculate the first evaluation score corresponding to the data to be evaluated according to the first weight coefficient and the evaluation sub-score corresponding to each data group.
[0096] Different data groups have different first weight coefficients. Based on the first weight coefficients corresponding to different data groups and the corresponding calculated evaluation sub-scores, the first evaluation score corresponding to the data to be evaluated can be further calculated according to a preset formula. The first evaluation score is the evaluation score automatically calculated by the model evaluation system according to the data to be evaluated input by the user, and the user evaluates whether the data to be evaluated input by the user meets the corresponding requirements.
[0097] In this embodiment, only when the first evaluation score meets the preset first evaluation threshold value, will the subsequent other evaluation processes be entered. This is to screen the type evaluation through the system to reduce the labor cost of the subsequent process to improve efficiency. For example, it is determined whether the first evaluation score is greater than or equal to the preset first evaluation threshold value. If so, it means that the currently submitted data to be evaluated meets the requirements of the type evaluation system, and the subsequent evaluation process can be carried out; if not, it means that the currently submitted data to be evaluated does not meet the requirements of the type evaluation system, and it is directly returned to the user for resubmission.
[0098] Step S107: Obtain the second evaluation score input by the expert through the model evaluation system.
[0099] Furthermore, not only is it necessary for the type evaluation system to automatically calculate whether the corresponding data to be evaluated meets the corresponding requirements, but in order to ensure the accuracy of the evaluation, it is also necessary for experts to intervene and give the corresponding second evaluation score. In a specific embodiment, the second evaluation score can be the second evaluation score obtained by the expert after evaluating the data to be evaluated; in a specific embodiment, the second evaluation score can also be the second evaluation score given by the expert after further testing the metering equipment to be evaluated corresponding to the data to be evaluated, so as to conduct a more comprehensive evaluation from another perspective.
[0100] In a specific embodiment, the second evaluation score is obtained based on the evaluation score obtained by performing a sample evaluation on the metrological device to be evaluated. That is, when the first evaluation score meets the preset first evaluation threshold, the prototype evaluation process corresponding to the metrological device to be evaluated is started, wherein the prototype evaluation process includes at least one evaluation sub-process, and each evaluation sub-process corresponds to an evaluation sub-laboratory, which tests the prototype of the metrological device to be evaluated, provides corresponding test data, and then evaluates whether the prototype meets the requirements of the evaluation sub-process based on the test data.
[0101] In each evaluation sub-process, in order to accurately correspond to the equipment information of the prototype, it is necessary to use a scanning device connected to the type evaluation system to scan the information set on the measuring equipment to be evaluated (for example, a QR code) to obtain the equipment information of the prototype of the measuring equipment to be evaluated, so that the test data can accurately correspond to the measuring equipment to be evaluated.
[0102] In each evaluation sub-process, test data provided by the corresponding evaluation sub-laboratory is obtained, wherein the test data is data provided by the evaluation sub-laboratory after testing the metrological equipment to be evaluated; and the evaluation sub-score corresponding to the evaluation sub-process is determined based on the test data.
[0103] Under each evaluation sub-process, after receiving the prototype of the metrological equipment to be evaluated, the staff of the corresponding evaluation sub-laboratory needs to test the prototype of the metrological equipment to be evaluated according to the corresponding process and standards, and input the corresponding data into the type evaluation system as the test data under the evaluation sub-process. Then, it can be determined whether the metrological equipment to be evaluated has passed the evaluation under the evaluation sub-process based on the corresponding test data.
[0104] It should be noted that, in this embodiment, after obtaining the test data, it is first necessary to judge whether the test data meets the preset requirements. If not, it means that the metering equipment to be evaluated may not have passed the corresponding evaluation sub-process; in this case, it is necessary to trigger a manual inspection process to judge whether the metering equipment to be evaluated has passed the evaluation sub-process. Specifically, the test data is manually judged whether it meets the preset requirements, and the manual judgment result is determined and input into the type evaluation system. If the manual judgment result is not satisfied, it means that the evaluation sub-process has not passed, and the evaluation sub-score corresponding to the evaluation sub-process is set to 0; if the manual judgment result is satisfied, it means that the current evaluation sub-process can enter the next process to continue further judgment on whether it passes the evaluation, and specifically execute the step of determining the evaluation sub-score corresponding to the evaluation sub-process according to the test data.
[0105] According to the test data, the evaluation sub-score of the metering equipment to be evaluated under the evaluation sub-process needs to be further determined, which can be obtained through a neural network model.
[0106] In other embodiments, the evaluation sub-score of the metering equipment to be evaluated in the evaluation sub-process may also be calculated through the following process.
[0107] First, the test data is divided into first test data and second test data; wherein the first test data is basic data, for example, basic information of the equipment, and the second test data is test data for testing the metering equipment to be evaluated. Then, the vector features of the first experimental data are obtained, and the vector features contain multiple features at multiple scales, the number of features is N, and the multiple scales are determined according to the corresponding evaluation sub-process. Then, the first feature subset of the vector features is obtained, for example, repeated data is screened, or some features are obtained from it according to a preset rule as the first feature subset, which can reduce the amount of data for subsequent data processing and reduce the amount of calculation. Moreover, the comprehensiveness of the first feature subset can be guaranteed by the screening criteria here to ensure the accuracy of data processing. Then, the multiple features contained in the vector features are fused to obtain the first feature after fusion.
[0108] In a specific implementation, the vector feature is input into a preset feature fusion model, the number of layers of the feature fusion model is the same as the number of features contained in the vector feature; at the i-th layer, the i-th feature is fused with other features, wherein the fusion process is performed according to the coefficient corresponding to the i-th feature, i=1,...,N, N is the number of layers of the feature fusion model; the fused features obtained at each layer are spliced according to the coefficient to obtain the first feature. At the i-th layer, the i-th feature is used as the main feature, and the weight of the main feature is the coefficient calculated according to the i-th feature, and then the weights of other features are determined according to the determined coefficient, and then the N features are fused according to the determined weights to obtain the fused features of each layer respectively.
[0109] Then, for each layer of fused features, further splicing processing is required. The splicing processing here is not a simple splicing processing, but a splicing processing based on the coefficients corresponding to the main features of each layer. The larger the coefficient value, the higher the proportion of the corresponding fused feature in the first feature after splicing.
[0110] For the second test data, it is necessary to extract the time domain features and frequency domain features of the second test data according to a preset transformation method. In order to avoid excessive data processing volume, it is necessary to sample the acquired time domain features and frequency domain features according to a preset sampling interval to obtain a second feature subset. The sampling interval can be obtained based on the second test data, and the second test data is input into a preset sampling interval model to obtain the corresponding sampling interval, wherein the sampling interval can be determined based on parameters such as data integrity, data volume, and data continuity of the second test data, thereby ensuring that the data after sampling can characterize the characteristics of the second test data as comprehensively as possible.
[0111] Then, because the first feature represents the basic information of the measuring equipment to be evaluated and part of the basic information in the test data, here, for the processing of the second feature subset, it is necessary to process the first feature. Specifically, the second feature itself is feature processed according to the first feature to obtain the second feature. Among them, the feature processing here can be to determine at least two key features in the second feature subset according to the first feature, and then fuse the determined key features to obtain the second feature; so as to obtain a feature that can better characterize the second test data.
[0112] Further, according to the preset evaluation score calculation method, the evaluation score of the first feature and the evaluation score of the second feature are calculated, and the evaluation sub-score corresponding to the evaluation sub-process is calculated. The evaluation score of the first feature and the evaluation score of the second feature can be calculated by a preset evaluation score calculation model, wherein the model can be a deep learning model or any other model, which is not limited here. Further, the final evaluation sub-score is determined by the evaluation score of the first feature and the evaluation score of the second feature.
[0113] Different evaluation sub-processes have different degrees of influence on the final result. Therefore, each evaluation sub-process corresponds to a weight coefficient (first weight coefficient). This weight coefficient is determined and is determined by the importance of the evaluation sub-process in the entire evaluation process. Then, for each evaluation sub-process, if the evaluation sub-score corresponding to a certain evaluation sub-process is too low, its proportion in the entire evaluation process needs to be increased to avoid the poor result of a certain evaluation sub-process being hidden by other results. Specifically, the first weight coefficient is corrected according to the evaluation sub-score, wherein, when the evaluation sub-score is too low (for example, when the evaluation sub-score is lower than the preset value), the correction function or correction coefficient corresponding to the evaluation sub-score is determined, and then the first weight coefficient is corrected based on the correction function and the correction coefficient, and the first weight coefficient after correction is greater than the first weight coefficient before correction.
[0114] Specifically, in one embodiment, if the evaluation sub-score is less than a preset value, a correction function corresponding to the evaluation sub-score is determined, wherein the score of the evaluation sub-score corresponds to the correction function; the first weight coefficient is corrected according to the correction function to obtain the first weight coefficient after correction; wherein the first weight coefficient after correction is greater than the first weight coefficient before correction.
[0115] After obtaining the evaluation sub-score of each evaluation sub-process, the corresponding second evaluation score can be calculated, wherein the second evaluation score is calculated according to the corrected first weight coefficient and the evaluation sub-score corresponding to each evaluation sub-process according to a preset second evaluation score calculation function.
[0116] Step S108: Determine a second weight coefficient, calculate a target evaluation score according to the second weight coefficient and the first evaluation score and the second evaluation score; and output an evaluation result corresponding to the target evaluation score.
[0117] The first evaluation score obtained by the model evaluation system and the second evaluation score given by the expert respectively correspond to the contribution value corresponding to the final target evaluation score, and thus correspond to the corresponding second weight coefficient. Then, according to the determined second weight coefficient and the first evaluation score and the second evaluation score, the corresponding target evaluation score is calculated and output according to the preset calculation formula.
[0118] Furthermore, in one embodiment, both the first evaluation score and the second evaluation score include multiple evaluation sub-scores; in the process of calculating the target evaluation score, it is necessary to further perform cross-fusion processing on the multiple evaluation sub-scores included in the first evaluation score to obtain the first target score after the cross-fusion processing; and, perform cross-fusion processing on the multiple evaluation sub-scores included in the second evaluation score to obtain the second target score after the cross-fusion processing. Then, the first target score and the second target score are cross-fused according to the second weight coefficient to obtain the target evaluation score.
[0119] In another embodiment, the calculation of each evaluation score may also be performed based on a preset model, and does not conflict with the aforementioned calculation process, except that the implementation process may be based on the model.
[0120] Specifically, when calculating the first evaluation score, firstly, for each data group, an evaluation sub-model corresponding to each data group is constructed to output the evaluation sub-score corresponding to each data group; then, according to the first weight coefficient and the preset superposition algorithm, the sub-model corresponding to each data group is superimposed to construct the first evaluation model; and based on the first evaluation model, the evaluation sub-score corresponding to each data group is used as input to obtain the first evaluation score. In addition, the second evaluation score can also be obtained based on the preset second evaluation model.
[0121] Furthermore, when calculating the target evaluation score, the first evaluation model and the second evaluation model are superimposed on each other according to a preset superposition algorithm based on the second weight coefficient to construct a target evaluation model; the first evaluation score and the second evaluation score are used as input, and the target evaluation score is calculated according to the target evaluation model.
[0122] In another embodiment, an evaluation device based on a type evaluation system is also provided. Figure 2 The evaluation device based on the type evaluation system includes:
[0123] The data input module 101 is used to obtain the data to be evaluated input by the user through the type evaluation system, wherein the data to be evaluated is the inspection data of whether the measuring equipment to be evaluated of the user to be evaluated meets the measurement standard, and the data to be evaluated includes multiple pieces of data;
[0124] The data initial screening module 102 is used to screen each piece of data to be evaluated according to a preset data format to determine whether each piece of data to be evaluated meets the format requirements corresponding to each piece of data to be evaluated;
[0125] The data grouping module 103 is used to group the data to be evaluated when the data to be evaluated meets the format requirements;
[0126] The feature extraction module 104 is used to extract at least one feature data corresponding to the data to be evaluated in each data group respectively;
[0127] The first evaluation score calculation module 105 is used to determine the standard data corresponding to each data group and the first weight coefficient corresponding to each data group, and for each data group, calculate the matching degree between the standard data and the feature data, the matching degree is used to identify whether the feature data meets the requirements of the standard data; determine the evaluation sub-score corresponding to each data group according to the matching degree; calculate the first evaluation score corresponding to the data to be evaluated according to the first weight coefficient and the evaluation sub-score corresponding to each data group;
[0128] A second evaluation score calculation module 106, used to obtain a second evaluation score input by the expert through the model evaluation system;
[0129] The third evaluation score calculation module 107 is used to determine the second weight coefficient according to the user to be evaluated and the expert, calculate the target evaluation score according to the second weight coefficient and the first evaluation score and the second evaluation score; and output the evaluation result corresponding to the target evaluation score.
[0130] The evaluation method, device, computer equipment and computer storage medium based on the type evaluation system are adopted to obtain the data to be evaluated input by the user through the type evaluation system, wherein the data to be evaluated is the inspection data of whether the metering equipment to be evaluated of the user to be evaluated meets the metering standard, and the data to be evaluated includes multiple data; each data to be evaluated is screened according to a preset data format to determine whether each data to be evaluated meets the format requirements corresponding to each data to be evaluated; when the data to be evaluated meets the format requirements, the data to be evaluated are grouped; for each data group, at least one feature data corresponding to the data to be evaluated in each data group is extracted; and the standard corresponding to each data group is determined. The first weight coefficient corresponding to the standard data and each data group is used to calculate the matching degree between the standard data and the feature data under each data group, and the matching degree is used to identify whether the feature data meets the requirements of the standard data; the evaluation sub-score corresponding to each data group is determined according to the matching degree; the first evaluation score corresponding to the data to be evaluated is calculated according to the first weight coefficient and the evaluation sub-score corresponding to each data group; the second evaluation score input by the expert through the model evaluation system is obtained; the second weight coefficient is determined according to the user to be evaluated and the expert, and the target evaluation score is calculated according to the second weight coefficient and the first evaluation score and the second evaluation score; the evaluation result corresponding to the target evaluation score is output. That is to say, after adopting the evaluation method, device computer equipment and computer storage medium based on the model evaluation system provided by the present invention, the evaluation score corresponding to the data to be evaluated can be comprehensively and accurately calculated for screening, grouping, feature extraction and evaluation score calculation of the data to be evaluated, so as to improve the accuracy of the evaluation of the model evaluation system. In addition, the comprehensiveness of the automatic evaluation of the system and the manual evaluation of the experts can be comprehensively considered to further improve the accuracy of the evaluation of the model evaluation system.
[0131] Figure 3 FIG. 1 shows an internal structure diagram of a computer device that implements the above-mentioned evaluation method based on the model evaluation system in an embodiment. The computer device can be a terminal or a server. Figure 3 As shown, the computer device includes a processor, a memory and a network interface connected via a system bus. The memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program. When the computer program is executed by the processor, the processor can implement the above method. The internal memory may also store a computer program. When the computer program is executed by the processor, the processor can implement the above method. Those skilled in the art can understand that Figure 3The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0132] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
[0133] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0134] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims. Please enter the specific implementation content part.
Claims
1. An evaluation method based on a type evaluation system, characterized in that: The method comprises: Acquire the data to be evaluated input by the user through the type evaluation system, wherein the data to be evaluated is the inspection data of whether the measuring equipment to be evaluated of the user to be evaluated meets the measurement standard, and the data to be evaluated includes multiple pieces of data; Screening each piece of data to be evaluated according to a preset data format to determine whether each piece of data to be evaluated meets the format requirements corresponding to each piece of data to be evaluated; When the data to be evaluated meets the format requirements, the data to be evaluated are grouped; For each data group, extract at least one feature data corresponding to the data to be evaluated in each data group; Determine the standard data corresponding to each data group and the first weight coefficient corresponding to each data group, and under each data group, calculate the matching degree between the standard data and the characteristic data, wherein the matching degree is used to identify whether the characteristic data meets the requirements of the standard data; determine the evaluation sub-score corresponding to each data group according to the matching degree; Calculate the first evaluation score corresponding to the data to be evaluated according to the first weight coefficient and the evaluation sub-score corresponding to each data group; Obtaining a second evaluation score; Determine a second weight coefficient, calculate a target evaluation score based on the second weight coefficient and the first evaluation score and the second evaluation score; and output an evaluation result corresponding to the target evaluation score.
2. The method according to claim 1, characterized in that The method further comprises: Obtain historical evaluation data of the user to be evaluated corresponding to the data to be evaluated in the evaluation system, and determine the confidence level corresponding to the user to be evaluated based on the historical evaluation data; Determining the requirements of the standard data corresponding to each data group according to the confidence level, wherein different confidence levels correspond to different requirements of the standard data; The step of calculating the matching degree between the standard data and the characteristic data for each data group, wherein the matching degree is used to identify whether the characteristic data meets the requirements of the standard data, and determining the evaluation sub-score corresponding to each data group according to the matching degree, further includes: According to the requirements of different standard data determined by the confidence level, it is determined whether the feature data meets the requirements of the standard data and the degree of matching is determined.
3. The method according to claim 1, characterized in that The step of grouping the data to be evaluated further includes: Performing a first grouping process on the data to be evaluated to obtain a plurality of first data groups, wherein the data to be evaluated in the plurality of first data groups obtained by the first grouping process do not overlap; Using a first data processing algorithm, performing a first screening operation on the data to be evaluated in each first data group according to the importance value; performing a second grouping process on the data to be evaluated in the first data group after the first screening, so as to obtain a plurality of second data groups under each first data group, wherein the data to be evaluated in the plurality of second data groups obtained by the second grouping process overlap; A second screening operation is performed on the data to be evaluated in the second data group according to the data characteristic values.
4. The method according to claim 3, characterized in that The step of performing a second grouping process on the data to be evaluated in the first data group after the first screening to obtain a plurality of second data groups under each first data group also includes: Performing a combination operation on the plurality of pieces of data to be evaluated in the first data group after the first screening, wherein each combined data packet obtained by the combination operation contains one or more pieces of data to be evaluated; Performing a second grouping process on the combined data under the first data group to obtain a plurality of second data groups; The step of performing a second screening operation on the data to be evaluated in the second data group according to the data characteristic value also includes: The combined features corresponding to each combined data are calculated as data feature values, and a second screening operation is performed on the combined data in the second data group according to the data feature values.
5. The method according to claim 4, characterized in that The step of calculating the matching degree between the standard data and the characteristic data in each data group further includes: Perform prediction analysis on each combined feature to obtain the predicted value corresponding to the combined feature; calculate the predicted residual value based on the combined feature and the predicted value; Calculate the negative entropy corresponding to the predicted residual value, and calculate the contribution value corresponding to the combined feature based on the calculated negative entropy; The similarity between the standard data and the combined data is calculated, and the matching degree between the standard data and the feature data is calculated based on the contribution value and similarity of the combined features.
6. The method according to claim 1, characterized in that The first evaluation score and the second evaluation score both include a plurality of evaluation sub-scores; The step of determining a second weight coefficient and calculating a target evaluation score according to the second weight coefficient and the first evaluation score and the second evaluation score also includes: Performing cross-fusion processing on multiple evaluation sub-scores included in the first evaluation score to obtain a first target score after the cross-fusion processing; Performing cross-fusion processing on multiple evaluation sub-scores included in the second evaluation score to obtain a second target score after the cross-fusion processing; The first target score and the second target score are cross-fused according to the second weight coefficient to obtain the target evaluation score.
7. The method according to claim 3, characterized in that The step of calculating the first evaluation score corresponding to the data to be evaluated according to the first weight coefficient and the evaluation sub-score corresponding to each data group also includes: Constructing an evaluation sub-model corresponding to each data group, and performing superposition processing on the sub-model corresponding to each data group according to a first weight coefficient and a preset superposition algorithm to construct a first evaluation model; Based on the first evaluation model, taking the evaluation sub-score corresponding to each data group as input, obtaining the first evaluation score; The step of obtaining the second evaluation score further includes: Calculate the second evaluation score according to the preset second evaluation model; The step of determining the second weight coefficient and calculating the target evaluation score according to the second weight coefficient and the first evaluation score and the second evaluation score also includes: According to a preset superposition algorithm, based on a second weight coefficient, the first evaluation model and the second evaluation model are superimposed to construct a target evaluation model; The target evaluation score is calculated according to the target evaluation model using the first evaluation score and the second evaluation score as input.
8. An evaluation device based on a type evaluation system, characterized in that: The device comprises: A data input module is used to obtain the data to be evaluated input by the user through the type evaluation system, wherein the data to be evaluated is the inspection data of whether the measuring equipment to be evaluated of the user to be evaluated meets the measurement standard, and the data to be evaluated includes multiple pieces of data; The data initial screening module is used to screen each piece of data to be evaluated according to a preset data format to determine whether each piece of data to be evaluated meets the format requirements corresponding to each piece of data to be evaluated; A data grouping module, used for grouping the data to be evaluated when the data to be evaluated meets the format requirements; A feature extraction module, used to extract at least one feature data corresponding to the data to be evaluated in each data group, for each data group; A first evaluation score calculation module is used to determine the standard data corresponding to each data group and the first weight coefficient corresponding to each data group, and under each data group, calculate the matching degree between the standard data and the feature data, the matching degree is used to identify whether the feature data meets the requirements of the standard data; determine the evaluation sub-score corresponding to each data group according to the matching degree; calculate the first evaluation score corresponding to the data to be evaluated according to the first weight coefficient and the evaluation sub-score corresponding to each data group; A second evaluation score calculation module, used to obtain a second evaluation score; The third evaluation score calculation module is used to determine the second weight coefficient, calculate the target evaluation score according to the second weight coefficient and the first evaluation score and the second evaluation score; and output the evaluation result corresponding to the target evaluation score.
9. A computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, wherein the computer program is used to execute the evaluation method based on the type evaluation system according to any one of claims 1 to 7.
10. A computer device, characterized in that: The computer device includes a memory and a processor, the memory has executable code, and when the executable code runs on the processor, the evaluation method based on the type evaluation system as described in any one of claims 1 to 7 is implemented.
