Test identification system evaluation system based on expert experience
By integrating expert opinions and building evaluation models in the experimental identification evaluation system, the complex and time-consuming problems of obtaining expert identification experience and integrating expert identification experience are solved, real-time and accurate evaluation of the experimental identification system and continuous optimization of the model are achieved, and the comprehensiveness of the evaluation is improved and accuracy.
Patent Information
- Application Number
- CN202411523480.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-30
- Publication Date
- 2024-12-03
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing test identification and evaluation system is complicated and time-consuming to obtain and integrate expert identification experience, and the expert identification experience is gradually outdated, resulting in the inability to accurately evaluate the test identification system in real time.
Design an experimental and identification system evaluation system based on expert experience, combining sensors and monitoring equipment through modules such as data collection, data processing, data feature extraction, expert identification experience integration, evaluation model construction, periodic evaluation and correction, evaluation model optimization. and test equipment, collect and process data in real time, integrate expert opinions, build and optimize evaluation models, and provide instant evaluation results.
It improves the comprehensiveness and accuracy of the evaluation, realizes real-time evaluation, reduces expert opinions, maintains the timeliness and adaptability of the model, and simplifies the process of obtaining and integrating expert identification experience.
Smart Images

Figure CN119067319A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of data processing, and in particular to an evaluation system for a test identification system based on expert experience. Background Art
[0002] The test identification system evaluation system usually consists of the following parts: Data collection and processing module, which is responsible for collecting various types of data in the test identification process, including test parameters, environmental variables, personnel learning performance quality, etc., and preprocessing and standardizing these data. Feature index extraction module, which can use data analysis technology and expert identification experience to extract key feature indicators from the collected data. Evaluation model construction module, based on the extracted feature indicators, combined with expert knowledge and experience, to build one or more evaluation models. The evaluation model construction module can quantitatively evaluate various aspects of the test identification system, such as efficiency, safety, cost-effectiveness, etc. Periodic evaluation and correction module, which is responsible for regularly evaluating the test identification system and setting correction parameters based on the evaluation results. Report and visualization module, which generates detailed evaluation reports, intuitively displays the evaluation results and key indicators through charts and data visualization tools, so that decision makers and relevant personnel can understand and use them.
[0003] The test appraisal system evaluation system takes into account multiple aspects of test appraisal quality, including personnel learning performance quality, equipment effectiveness, training programs, etc., to improve the comprehensiveness and accuracy of the evaluation. The evaluation model can be adjusted and optimized according to actual needs to adapt to different test appraisal scenarios and needs. The system can collect and process data in real time, provide immediate evaluation results, and help to adjust the appraisal plan and strategy in a timely manner. Through the intuitive user interface and report generation function, users can easily understand and use the evaluation results.
[0004] However, in practical applications, pure data analysis often ignores the value of expert appraisal experience in evaluation, which is indispensable in many cases. In the existing test appraisal evaluation system, expert appraisal experience is usually integrated into the evaluation process in a specific way. By using the experts' in-depth understanding of the test appraisal system, key characteristic indicators are quickly extracted. Subsequently, the experts assign values according to these characteristic indicators and construct an evaluation model based on expert knowledge and experience. The evaluation model based on expert knowledge and experience can more comprehensively reflect the actual situation of the test appraisal system. By periodically using this evaluation model to evaluate the test appraisal system, deficiencies in the system can be discovered in a timely manner, and corresponding correction parameters can be set according to the evaluation results, thereby continuously optimizing the effectiveness of the system.
[0005] Although the existing test identification and evaluation system has achieved certain results in combining expert identification experience, there are still the following technical pain points. Since the acquisition and integration of expert identification experience is a complex and time-consuming process, there are differences in the opinions of different experts. At the same time, the original expert identification experience gradually becomes outdated, and the evaluation model needs to be continuously updated and maintained to adapt to new situations. As a result, the test identification system evaluation system based on expert experience cannot be evaluated in real time and accurately. Summary of the invention
[0006] In view of the shortcomings of the prior art, the present invention provides a test identification system evaluation system based on expert experience to solve the problem that the test identification system evaluation system based on expert experience cannot perform real-time and accurate evaluation because the acquisition and integration of expert identification experience is a complex and time-consuming process, and there are differences in opinions among different experts. At the same time, the original expert identification experience gradually becomes outdated, and the evaluation model needs to be constantly updated and maintained to adapt to new situations.
[0007] In order to solve the above technical problems, the specific technical solutions of the present invention are as follows: The present invention provides an evaluation system for a test identification system based on expert experience, comprising: a server end, a device end, a user mobile terminal and a background data end, wherein the device end comprises a sensor, a monitoring device and a test device, the server end respectively establishes communication connections with the device end, the user mobile terminal and the background data end, the server end analyzes and processes data of the device end, the user mobile terminal and the background data end, generates an evaluation result of the test identification system based on expert identification experience, and transmits the evaluation result of the test identification system based on expert identification experience to the user mobile terminal and the background data end, and the server end comprises: a data acquisition module, a data processing module, a data feature extraction module, an expert identification experience integration module, an evaluation model construction module, a periodic evaluation module, a periodic correction module, an evaluation model optimization module and an evaluation result generation module; The data acquisition module is used to collect the original data of the evaluation system. The original data of the evaluation system includes test parameter data, environmental data, equipment data, personnel learning performance quality data, audio and video data, safety data and feedback data. The data acquisition module transmits the original data of the evaluation system to the data processing module. The data processing module pre-processes the original data of the evaluation system to obtain the pre-processed original data of the evaluation system. The data processing module transmits the preprocessed raw data of the evaluation system to the data feature extraction module, and the data feature extraction module extracts data features from the preprocessed data; The expert appraisal experience integration module collects and integrates expert appraisal opinions and builds an expert appraisal experience database. The evaluation model construction module combines the data features extracted by the data feature extraction module and the expert appraisal experience database of the expert appraisal experience integration module to build an expert appraisal experience evaluation model. The periodic evaluation module uses the expert appraisal experience evaluation model to regularly evaluate the test appraisal system and obtain the test appraisal system evaluation results. The periodic correction module corrects the expert appraisal experience evaluation model according to the test appraisal system evaluation results and obtains the corrected expert appraisal experience evaluation model. The evaluation model optimization module optimizes the corrected expert appraisal experience evaluation model according to the newly collected data of the data acquisition module and the historical data of the data acquisition module and obtains the optimized expert appraisal experience evaluation model. The evaluation result generation module visualizes the evaluation results.
[0008] Furthermore, the expert experience-based test identification system evaluation system provided by the present invention includes a data acquisition module, which includes: Test parameter data: including the start and end time of the identification, the identification stage division, and the identification goal setting; Environmental data: including the environmental conditions such as temperature, humidity, wind speed and light intensity when testing and identifying the system; Equipment data: including the working status, performance parameters and usage of the equipment used in the test and identification process; Personnel learning performance quality data: record the performance of the test personnel during the identification process, reaction time, operation accuracy and time to complete the task; Audio and video data: collect audio and video data of the test and identification site through cameras and microphones for subsequent test and identification behavior analysis or test and identification event review; Safety data: including records of safety events or hidden dangers during the test identification process; Feedback data: includes real-time feedback or evaluations provided by test subjects, coaches or observers during or after the test.
[0009] Furthermore, the expert experience-based test identification system evaluation system provided by the present invention includes a data processing module, which includes: Preprocess the raw data of the evaluation system, including checking and deleting duplicate records in the data set, using statistical methods to identify and process outliers in the data, and selecting to correct, delete or mark outliers.
[0010] Furthermore, the expert experience-based test identification system evaluation system provided by the present invention includes a data feature extraction module, which includes: The data feature extraction module extracts data features from the preprocessed data, including test parameter features, environmental features, equipment features, personnel learning performance quality features, audio and video features, safety features, and feedback features.
[0011] Furthermore, the expert experience-based test identification system evaluation system provided by the present invention includes an expert identification experience integration module, including: Collect pictures and electronic documents related to expert appraisal through expert appraisal reports, communication information during the expert appraisal process, and knowledge and experience information from the appraisal meeting; The collected language, text information, pictures, and electronic documents are marked with knowledge, and the information in the knowledge mark includes the name of the expert, the type of expert advice, and the time of the expert advice; Standardize the collected language, text information, images, and electronic documents, and set weights for the expert identification experience from different sources based on the experts' domain knowledge, experience level, and historical accuracy. The weights include scores based on historical accuracy or scores from expert panels. Using the weighted average decision-making method, the appraisal experience of different experts is integrated according to their weights, including integrating language, pictures and text, to obtain the integrated expert appraisal experience; The integrated expert appraisal experience is stored in a database to form an expert appraisal experience base. The expert appraisal experience base is verified regularly and the accuracy of the expert appraisal experience is evaluated by comparing it with the actual expert appraisal results.
[0012] Furthermore, the expert experience-based test identification system evaluation system provided by the present invention includes an evaluation model building module, including: The evaluation model construction module combines the test parameter characteristics, environmental characteristics, equipment characteristics, personnel learning performance quality characteristics, audio and video characteristics, safety characteristics and feedback characteristics with the expert appraisal experience library of the expert appraisal experience integration module to train the random forest model. The historical data of the evaluation system and the real-time data of the evaluation system are used as the training set data, and the expert appraisal experience library of the expert appraisal experience integration module is used as the verification set to construct an expert appraisal experience evaluation model.
[0013] Furthermore, the expert experience-based test identification system evaluation system provided by the present invention includes a periodic evaluation module, which includes: Regularly use the established expert appraisal experience evaluation model to evaluate the current test appraisal system, and obtain the test appraisal system evaluation results through the operation of the expert appraisal experience evaluation model. The regular time period is 24 hours apart. Compare the test and appraisal system evaluation results with the actual expert appraisal results to check whether the system quality, personnel learning performance quality, and equipment status in the evaluation results are consistent with the actual observation data; If the evaluation results of the test identification system are consistent with the actual expert identification results, the expert identification experience evaluation model is highly accurate; if there are differences between the evaluation results of the test identification system and the actual expert identification results, the expert identification experience evaluation model needs to be revised.
[0014] Furthermore, the expert experience-based test identification system evaluation system provided by the present invention includes a periodic correction module, which includes: There are differences between the test and identification system evaluation results and the actual expert identification results. The periodic correction module corrects the expert identification experience evaluation model according to the test and identification system evaluation results. Modifying the expert appraisal experience evaluation model includes adjusting the parameters of the expert appraisal experience evaluation model, updating the data features used by the expert appraisal experience evaluation model, and reintegrating the expert appraisal experience. The modified expert appraisal experience evaluation model is evaluated and verified. The verification process includes testing with new test appraisal data and re-comparing the new test appraisal system evaluation results with the actual expert appraisal results. If the test appraisal system evaluation results are consistent with the actual expert appraisal results, the expert appraisal experience evaluation model is modified.
[0015] Furthermore, the expert experience-based test identification system evaluation system provided by the present invention includes an evaluation model optimization module, which includes: Analyze data features using new data and historical data from the evaluation system, continuously collect and integrate new expert opinions, and use the new data features, historical data features, and new expert opinions from the evaluation system to optimize the expert evaluation experience evaluation model through genetic algorithms; Genetic algorithms are used to iterate data features and expert appraisal opinions, and new test appraisal data are added through selection, crossover and mutation operations. The new test appraisal data are used to optimize the expert appraisal experience evaluation model to obtain the optimized expert appraisal experience evaluation model, which is then verified and tested.
[0016] Furthermore, the expert experience-based test identification system evaluation system provided by the present invention includes an evaluation result generation module, which includes: Summarize the evaluation results generated by the expert appraisal experience evaluation model, which include the evaluation results of efficiency, safety, and cost-effectiveness; Use charts and data visualization tools to generate visualization charts of the evaluation results produced by the expert appraisal experience evaluation model; The evaluation results and visual charts generated by the expert appraisal experience evaluation model are organized into an evaluation report, which includes the evaluation purpose, evaluation method, evaluation results, problem analysis and improvement suggestions.
[0017] Beneficial effects of the present invention: By integrating expert appraisal opinions and constructing an expert appraisal experience database, the present invention integrates expert appraisal experience into the evaluation model, thereby improving the comprehensiveness and accuracy of the evaluation. Real-time data collection and processing, combined with the expert appraisal experience evaluation model to provide instant evaluation results, help to timely discover problems and make adjustments, thereby improving the real-time and accuracy of the evaluation. Through the periodic evaluation and correction module, as well as the evaluation model optimization module, the system can regularly verify and correct the evaluation model, and continuously optimize it according to new data to maintain the timeliness and accuracy of the model. The evaluation results are intuitively displayed using charts and data visualization tools, which is convenient for decision makers and relevant personnel to understand and use. Expert appraisal opinions are collected in a systematic way, and knowledge is marked and standardized, making it easier to obtain and integrate expert appraisal experience. Weights are set for expert appraisal experience from different sources, and they are integrated using the weighted average decision method, which effectively reduces the differences in appraisal opinions between experts, improves the adaptability and generalization ability of the model, optimizes the evaluation model using a genetic algorithm, and iterates in combination with new data and continuously collected expert appraisal opinions, thereby enhancing the adaptability and generalization ability of the model. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0019] Figure 1 A schematic diagram of a system provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0020] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described in conjunction with the specific embodiments of the present invention and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. The technical solutions provided by the embodiments of the present invention are described in detail below in conjunction with the drawings.
[0021] In order to better understand the purpose of the present invention, the present invention is described in further detail below.
[0022] The present invention provides an evaluation system for a test identification system based on expert experience, comprising: a server end, a device end, a user mobile terminal and a background data end, wherein the device end comprises a sensor, a monitoring device and a test device, the server end respectively establishes communication connections with the device end, the user mobile terminal and the background data end, the server end analyzes and processes data of the device end, the user mobile terminal and the background data end, generates an evaluation result of the test identification system based on expert identification experience, and transmits the evaluation result of the test identification system based on expert identification experience to the user mobile terminal and the background data end, and the server end comprises: a data acquisition module, a data processing module, a data feature extraction module, an expert identification experience integration module, an evaluation model construction module, a periodic evaluation module, a periodic correction module, an evaluation model optimization module and an evaluation result generation module; The data acquisition module is used to collect the original data of the evaluation system. The original data of the evaluation system includes test parameter data, environmental data, equipment data, personnel learning performance quality data, audio and video data, safety data and feedback data. The data acquisition module transmits the original data of the evaluation system to the data processing module. The data processing module pre-processes the original data of the evaluation system to obtain the pre-processed original data of the evaluation system. The data processing module transmits the preprocessed raw data of the evaluation system to the data feature extraction module, and the data feature extraction module extracts data features from the preprocessed data; The expert appraisal experience integration module collects and integrates expert appraisal opinions and builds an expert appraisal experience database. The evaluation model construction module combines the data features extracted by the data feature extraction module and the expert appraisal experience database of the expert appraisal experience integration module to build an expert appraisal experience evaluation model. The periodic evaluation module uses the expert appraisal experience evaluation model to regularly evaluate the test appraisal system and obtain the test appraisal system evaluation results. The periodic correction module corrects the expert appraisal experience evaluation model according to the test appraisal system evaluation results and obtains the corrected expert appraisal experience evaluation model. The evaluation model optimization module optimizes the corrected expert appraisal experience evaluation model according to the newly collected data of the data acquisition module and the historical data of the data acquisition module and obtains the optimized expert appraisal experience evaluation model. The evaluation result generation module visualizes the evaluation results.
[0023] The expert experience-based test identification system evaluation system provided by the present invention is based on expert identification experience, and includes multiple key modules: data collection, data processing, data feature extraction, expert identification experience integration, evaluation model construction, periodic evaluation and correction, evaluation model optimization, and evaluation result generation. The modules work together to form a complete and efficient evaluation process from the collection of raw data to the visualization of the final evaluation results. By combining data features and expert identification experience, the system can achieve regular evaluation of the test identification system and continuous optimization of the model.
[0024] Specifically, the expert experience-based test identification system evaluation system provided by the present invention includes a data acquisition module, which includes: Test parameter data: including the start and end time of the identification, the identification stage division, and the identification goal setting; Environmental data: including the environmental conditions such as temperature, humidity, wind speed and light intensity when testing and identifying the system; Equipment data: including the working status, performance parameters and usage of the equipment used in the test and identification process; Personnel learning performance quality data: record the performance of the test personnel during the identification process, reaction time, operation accuracy and time to complete the task; Audio and video data: collect audio and video data of the test and identification site through cameras and microphones for subsequent test and identification behavior analysis or test and identification event review; Safety data: including records of safety events or hidden dangers during the test identification process; Feedback data: includes real-time feedback or evaluations provided by test subjects, coaches or observers during or after the test.
[0025] Specifically, the expert experience-based test identification system evaluation system provided by the present invention includes a data processing module, which includes: Preprocess the raw data of the evaluation system, including checking and deleting duplicate records in the data set, using statistical methods to identify and process outliers in the data, and selecting to correct, delete or mark outliers.
[0026] Specifically, the expert experience-based test identification system evaluation system provided by the present invention includes a data feature extraction module, which includes: The data feature extraction module extracts data features from the preprocessed data, including test parameter features, environmental features, equipment features, personnel learning performance quality features, audio and video features, safety features, and feedback features.
[0027] Specifically, the expert experience-based test identification system evaluation system provided by the present invention includes an expert identification experience integration module, including: Collect pictures and electronic documents related to expert appraisal through expert appraisal reports, communication information during the expert appraisal process, and knowledge and experience information from the appraisal meeting; The collected language, text information, pictures, and electronic documents are marked with knowledge, and the information in the knowledge mark includes the name of the expert, the type of expert advice, and the time of the expert advice; Standardize the collected language, text information, images, and electronic documents, and set weights for the expert identification experience from different sources based on the experts' domain knowledge, experience level, and historical accuracy. The weights include scores based on historical accuracy or scores from expert panels. Using the weighted average decision-making method, the appraisal experience of different experts is integrated according to their weights, including integrating language, pictures and text, to obtain the integrated expert appraisal experience; The integrated expert appraisal experience is stored in a database to form an expert appraisal experience base. The expert appraisal experience base is verified regularly and the accuracy of the expert appraisal experience is evaluated by comparing it with the actual expert appraisal results.
[0028] Information collection: Collect the knowledge and experience of experts through various means (such as questionnaires, communication during expert appraisal, seminars). Collect not only language and text information, but also related pictures and electronic documents.
[0029] Knowledge tagging: Detailed knowledge tagging is performed on the collected information, and the tagging content includes the expert name, expert advice type and expert advice time.
[0030] Standardization processing: Standardize the collected language, text, images, and electronic documents to ensure the uniformity and comparability of the information.
[0031] Weights are set for different sources of expert identification experience based on their domain knowledge, experience level, and historical accuracy, with weights based on historical accuracy scores or expert panel ratings.
[0032] Experience integration: The weighted average decision-making method is used to integrate the experience of different experts according to the previously set weights.
[0033] The integration process includes integrating language, pictures and text, and ultimately forming integrated expert identification experience.
[0034] Experience database construction: The integrated expert appraisal experience is stored in the database to form an expert appraisal experience database to facilitate the visual display and rapid retrieval of knowledge.
[0035] Experience verification: Regularly verify the expert appraisal experience database and evaluate the accuracy and timeliness of the expert appraisal experience by comparing it with the actual expert appraisal results.
[0036] Through the above steps, the present invention can systematically and scientifically integrate the expert identification experience to form a set of expert identification experience database, providing strong support for the evaluation of the test identification system.
[0037] Specifically, the expert experience-based test identification system evaluation system provided by the present invention includes an evaluation model building module, including: The evaluation model construction module combines the test parameter characteristics, environmental characteristics, equipment characteristics, personnel learning performance quality characteristics, audio and video characteristics, safety characteristics and feedback characteristics with the expert appraisal experience library of the expert appraisal experience integration module to train the random forest model. The historical data of the evaluation system and the real-time data of the evaluation system are used as the training set data, and the expert appraisal experience library of the expert appraisal experience integration module is used as the verification set to construct an expert appraisal experience evaluation model.
[0038] Specifically, the expert experience-based test identification system evaluation system provided by the present invention includes a periodic evaluation module, which includes: Regularly use the established expert appraisal experience evaluation model to evaluate the current test appraisal system, and obtain the test appraisal system evaluation results through the operation of the expert appraisal experience evaluation model. The regular time period is 24 hours apart. Compare the test and appraisal system evaluation results with the actual expert appraisal results to check whether the system quality, personnel learning performance quality, and equipment status in the evaluation results are consistent with the actual observation data; If the evaluation results of the test identification system are consistent with the actual expert identification results, the expert identification experience evaluation model is highly accurate. If there are differences between the evaluation results of the test identification system and the actual expert identification results, the expert identification experience evaluation model needs to be revised.
[0039] Specifically, the expert experience-based test identification system evaluation system provided by the present invention includes a periodic correction module, which includes: There are differences between the test and identification system evaluation results and the actual expert identification results. The periodic correction module corrects the expert identification experience evaluation model according to the test and identification system evaluation results. Modifying the expert appraisal experience evaluation model includes adjusting the parameters of the expert appraisal experience evaluation model, updating the data features used by the expert appraisal experience evaluation model and reintegrating the expert appraisal experience. The modified expert appraisal experience evaluation model is evaluated and verified. The verification process includes testing with new test appraisal data and re-comparing the new test appraisal system evaluation results with the actual expert appraisal results. If the test appraisal system evaluation results are consistent with the actual expert appraisal results, the expert appraisal experience evaluation model is modified.
[0040] Specifically, the expert experience-based test identification system evaluation system provided by the present invention includes an evaluation model optimization module, which includes: Analyze data features using new data and historical data from the evaluation system, continuously collect and integrate new expert opinions, and use the new data features, historical data features, and new expert opinions from the evaluation system to optimize the expert evaluation experience evaluation model through genetic algorithms; Genetic algorithms are used to iterate data features and expert appraisal opinions, and new test appraisal data are added through selection, crossover and mutation operations. The new test appraisal data are used to optimize the expert appraisal experience evaluation model to obtain the optimized expert appraisal experience evaluation model, which is then verified and tested.
[0041] The expert experience-based test identification system provided by the present invention has a core part comprising an evaluation model building module, a periodic evaluation module, a periodic correction module and an evaluation model optimization module.
[0042] The evaluation model construction module uses a variety of data features and expert appraisal experience libraries to train the evaluation model. First, it combines the multi-dimensional data features of test parameters, environment, equipment, personnel learning performance quality, audio and video, safety, and feedback. Then, it introduces the expert appraisal experience library of the expert appraisal experience integration module to provide guidance based on expert knowledge for the model. Finally, the random forest algorithm is used for model training, with the evaluation system historical data and real-time data as the training set and the expert appraisal experience library as the verification set, so as to train the expert appraisal experience evaluation model.
[0043] The periodic evaluation module is responsible for regularly evaluating the test and identification system. It uses the established expert identification experience evaluation model to evaluate the current test and identification system every 24 hours, and compares the evaluation results with the actual expert identification results to check the consistency of key indicators such as system quality, personnel learning performance quality, equipment status, etc. This helps to verify the accuracy of the model and trigger the correction mechanism when differences are found.
[0044] When there is a deviation between the evaluation results and the actual situation, the periodic correction module will intervene to correct the model. The periodic correction module identifies inaccuracies in the model by analyzing the differences between the evaluation results and the actual situation, and corrects the model by adjusting model parameters, updating data features, reintegrating expert appraisal experience, etc. The corrected model will be tested and verified with new test appraisal data to ensure the effectiveness of the correction.
[0045] In order to continuously improve the performance and adaptability of the model, the evaluation model optimization module is responsible for continuously optimizing the evaluation model. The evaluation model optimization module uses newly collected data and historical data to deeply analyze the changing trends of data features, and continuously collects and integrates new expert opinions to ensure that the model always incorporates the latest professional knowledge. In addition, the evaluation model optimization module also uses genetic algorithms to iteratively optimize data features and expert opinions, and introduces new test identification data through selection, crossover and mutation operations to further improve the accuracy and generalization ability of the model.
[0046] The evaluation system provided by the present invention can realize a comprehensive, accurate and dynamic evaluation of the test identification system and provide strong support for continuous optimization. Specifically, the expert experience-based test identification system evaluation system provided by the present invention includes an evaluation result generation module, which includes: Summarize the evaluation results generated by the expert appraisal experience evaluation model, which include the evaluation results of efficiency, safety, and cost-effectiveness; Use charts and data visualization tools to generate visualization charts of the evaluation results produced by the expert appraisal experience evaluation model; The evaluation results and visual charts generated by the expert appraisal experience evaluation model are organized into an evaluation report, which includes the evaluation purpose, evaluation method, evaluation results, problem analysis and improvement suggestions.
[0047] The present invention effectively solves the problems of complex and time-consuming acquisition and integration of expert appraisal experience, divergent expert appraisal opinions, and outdated expert appraisal experience, thereby achieving real-time and accurate evaluation of the test appraisal system evaluation system. The specific solutions are as follows: Expert appraisal experience integration module: Collect expert appraisal opinions through questionnaires, communication during expert appraisal, and seminars. Perform knowledge tagging and standardization on the collected language, text information, pictures, and electronic documents. Set weights for expert appraisal experience from different sources based on the experts' domain knowledge, experience level, and historical accuracy. Use the weighted average decision method to integrate the experience of different experts to form an integrated expert appraisal experience database.
[0048] Dynamic optimization mechanism: Periodic evaluation and correction module: Regularly evaluate the test identification system and adjust the expert identification experience evaluation model based on the evaluation results to ensure its accuracy.
[0049] Evaluation model optimization module: Utilizes newly collected data and historical data, as well as continuously collected expert opinions, to optimize the evaluation model through genetic algorithms to adapt to new evaluation needs.
[0050] Data-driven decision support: The data collection module collects a wide range of multi-dimensional data such as test parameters, environment, equipment, personnel learning performance quality, audio and video, safety and feedback.
[0051] Data processing and feature extraction module: preprocesses and extracts features from raw data to provide high-quality input for the evaluation model.
[0052] Visual report decision support: Evaluation result generation module: The evaluation results are presented in the form of charts and visual reports to facilitate understanding and use by decision makers and relevant personnel.
[0053] The present invention not only simplifies the process of acquiring and integrating expert appraisal experience and reduces the divergence of expert appraisal opinions, but also ensures the timeliness of expert appraisal experience and the accuracy of the evaluation model through a dynamic optimization mechanism, thereby realizing real-time and accurate evaluation of the test appraisal system evaluation system.
[0054] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations, and the above-described embodiments of the present invention do not constitute a limitation on the scope of protection of the present invention.
Claims
1. A test identification system evaluation system based on expert experience, characterized in that: include: The server side, the device side, the user mobile terminal and the background data side, the device side includes sensors, monitoring equipment and test equipment, the server side establishes communication connections with the device side, the user mobile terminal and the background data side respectively, the server side analyzes and processes the data of the device side, the user mobile terminal and the background data side, generates the evaluation results of the test identification system of the expert identification experience, and transmits the evaluation results of the test identification system of the expert identification experience to the user mobile terminal and the background data side, the server side includes: data acquisition module, data processing module, data feature extraction module, expert identification experience integration module, evaluation model construction module, periodic evaluation module, periodic correction module, evaluation model optimization module and evaluation result generation module; The data acquisition module is used to collect the original data of the evaluation system. The original data of the evaluation system includes test parameter data, environmental data, equipment data, personnel learning performance quality data, audio and video data, safety data and feedback data. The data acquisition module transmits the original data of the evaluation system to the data processing module. The data processing module pre-processes the original data of the evaluation system to obtain the pre-processed original data of the evaluation system. The data processing module transmits the preprocessed raw data of the evaluation system to the data feature extraction module, and the data feature extraction module extracts data features from the preprocessed data; The expert appraisal experience integration module collects and integrates expert appraisal opinions and builds an expert appraisal experience database. The evaluation model construction module combines the data features extracted by the data feature extraction module and the expert appraisal experience database of the expert appraisal experience integration module to build an expert appraisal experience evaluation model. The periodic evaluation module uses the expert appraisal experience evaluation model to regularly evaluate the test appraisal system and obtain the test appraisal system evaluation results. The periodic correction module corrects the expert appraisal experience evaluation model according to the test appraisal system evaluation results and obtains the corrected expert appraisal experience evaluation model. The evaluation model optimization module optimizes the corrected expert appraisal experience evaluation model according to the newly collected data of the data acquisition module and the historical data of the data acquisition module and obtains the optimized expert appraisal experience evaluation model. The evaluation result generation module is used to visualize the evaluation results.
2. The test identification system evaluation system based on expert experience according to claim 1, characterized in that: Data acquisition module, including: Test parameter data: including the start and end time of the identification, the identification stage division, and the identification goal setting; Environmental data: including the environmental conditions such as temperature, humidity, wind speed and light intensity when testing and identifying the system; Equipment data: including the working status, performance parameters and usage of the equipment used during the test and identification process; Personnel learning performance quality data: record the performance of the test personnel during the identification process, reaction time, operation accuracy and time to complete the task; Audio and video data: collect audio and video data of the test and identification site through cameras and microphones for subsequent test and identification behavior analysis or test and identification event review; Safety data: including records of safety events or hidden dangers during the test identification process; Feedback data: includes real-time feedback or evaluations provided by test subjects, coaches or observers during or after the test.
3. The test identification system evaluation system based on expert experience as claimed in claim 1, characterized in that: Data processing module, including: Preprocess the raw data of the evaluation system, including checking and deleting duplicate records in the data set, using statistical methods to identify and process outliers in the data, and selecting to correct, delete or mark outliers.
4. The test identification system evaluation system based on expert experience as claimed in claim 1, characterized in that: Data feature extraction module, including: The data feature extraction module extracts data features from the preprocessed data, including test parameter features, environmental features, equipment features, personnel learning performance quality features, audio and video features, safety features, and feedback features.
5. The test identification system evaluation system based on expert experience as claimed in claim 1, characterized in that: Expert appraisal experience integration module, including: Collect pictures and electronic documents related to expert appraisal through expert appraisal reports, communication information during the expert appraisal process, and knowledge and experience information from the appraisal meeting; The collected language, text information, pictures, and electronic documents are marked with knowledge, and the information in the knowledge mark includes the name of the expert, the type of expert advice, and the time of the expert advice; Standardize the collected language, text information, images, and electronic documents, and set weights for the expert identification experience from different sources based on the experts' domain knowledge, experience level, and historical accuracy. The weights include scores based on historical accuracy or scores from expert panels. Using the weighted average decision-making method, the appraisal experience of different experts is integrated according to their weights, including integrating language, pictures and text, to obtain the integrated expert appraisal experience; The integrated expert appraisal experience is stored in a database to form an expert appraisal experience base. The expert appraisal experience base is verified regularly and the accuracy of the expert appraisal experience is evaluated by comparing it with the actual expert appraisal results.
6. The test identification system evaluation system based on expert experience according to claim 1, characterized in that: Evaluation model building blocks, including: The evaluation model construction module combines the test parameter characteristics, environmental characteristics, equipment characteristics, personnel learning performance quality characteristics, audio and video characteristics, safety characteristics and feedback characteristics with the expert appraisal experience library of the expert appraisal experience integration module to train the random forest model. The historical data of the evaluation system and the real-time data of the evaluation system are used as the training set data, and the expert appraisal experience library of the expert appraisal experience integration module is used as the verification set to construct an expert appraisal experience evaluation model.
7. The test identification system evaluation system based on expert experience as claimed in claim 1, characterized in that: Periodic assessment modules including: Regularly use the established expert appraisal experience evaluation model to evaluate the current test appraisal system, and obtain the test appraisal system evaluation results through the operation of the expert appraisal experience evaluation model. The regular time period is 24 hours apart. Compare the test and appraisal system evaluation results with the actual expert appraisal results to check whether the system quality, personnel learning performance quality, and equipment status in the evaluation results are consistent with the actual observation data; If the evaluation results of the test identification system are consistent with the actual expert identification results, the expert identification experience evaluation model is accurate; if there are differences between the evaluation results of the test identification system and the actual expert identification results, the expert identification experience evaluation model needs to be revised.
8. The expert experience-based test identification system evaluation system according to claim 7, characterized in that: Periodic correction module, including: There are differences between the test and identification system evaluation results and the actual expert identification results. The periodic correction module corrects the expert identification experience evaluation model according to the test and identification system evaluation results. Modifying the expert appraisal experience evaluation model includes adjusting the parameters of the expert appraisal experience evaluation model, updating the data features used by the expert appraisal experience evaluation model, and reintegrating the expert appraisal experience. The modified expert appraisal experience evaluation model is evaluated and verified. The verification process includes testing with new test appraisal data and re-comparing the new test appraisal system evaluation results with the actual expert appraisal results. If the test appraisal system evaluation results are consistent with the actual expert appraisal results, the expert appraisal experience evaluation model is modified.
9. The test identification system evaluation system based on expert experience according to claim 1, characterized in that: Evaluation model optimization module, including: Analyze data features using new data and historical data from the evaluation system, continuously collect and integrate new expert opinions, and use the new data features, historical data features, and new expert opinions from the evaluation system to optimize the expert evaluation experience evaluation model through genetic algorithms; Genetic algorithms are used to iterate data features and expert appraisal opinions, and new test appraisal data are added through selection, crossover and mutation operations. The new test appraisal data are used to optimize the expert appraisal experience evaluation model to obtain the optimized expert appraisal experience evaluation model, which is then verified and tested.
10. The test identification system evaluation system based on expert experience according to claim 1, characterized in that: Evaluation result generation module, including: Summarize the evaluation results generated by the expert appraisal experience evaluation model, which include the evaluation results of efficiency, safety, and cost-effectiveness; Use charts and data visualization tools to generate visualization charts of the evaluation results produced by the expert appraisal experience evaluation model; The evaluation results and visual charts generated by the expert appraisal experience evaluation model are organized into an evaluation report, which includes the evaluation purpose, evaluation method, evaluation results, problem analysis and improvement suggestions.
Citation Information
Patent Citations
Method for constructing equipment combat test identification index system
CN111832949A
Agricultural machinery test identification system and method
CN115774435A
Financial risk control method and system integrating deep learning and expert experience
CN117235444A
Rotary equipment health assessment method fusing expert experience and AI algorithm
CN117332246A
Foundation pit supporting performance evaluation method and system based on data analysis
CN118133672A
