Multi-dimensional intelligent evaluation expert review system and evaluation method

Abstract

The invention discloses a multi-dimensional intelligent assessment expert review system and method, and relates to the technical field of intelligent assessment, and the system comprises an intelligent interaction perception module, a multi-dimensional intelligent analysis module, a dynamic model adaptive optimization module, and an expert behavior deviation correction module. The intelligent interactive sensing module and the dynamic model adaptive optimization module provide deviation correction data support for the expert behavior deviation correction module, and the multi-dimensional intelligent analysis module performs multi-level and multi-dimensional fusion analysis based on data output feedback of the intelligent interactive sensing module and the dynamic model adaptive optimization module. By designing the multi-dimensional intelligent analysis module and the expert behavior deviation correction module, a deep analysis function on expert scoring data is realized, the problem of large result deviation caused by subjectivity and standard non-uniformity of expert scoring is solved, the influence of subjective prejudice on a review result is reduced, the scoring is more accurate and fair, and the evaluation efficiency is improved. And the reliability of the system in a complex review task is improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent evaluation technology, and in particular to an expert review system and evaluation method for multi-dimensional intelligent evaluation. Background Art

[0002] With the development of science and technology and society, various complex evaluation tasks have increasingly higher requirements for fairness, scientificity and efficiency. For example, in the review of scientific research projects, the selection of innovative products and the screening of technical solutions, the traditional manual review method is easily affected by the subjective bias of experts, inconsistent scoring standards and the increasing complexity of tasks, resulting in inaccurate or intransparent review results.

[0003] To solve these problems, the introduction of intelligent review systems has become a trend. By combining artificial intelligence, multimodal data analysis and dynamic optimization technology, the expert review system of multi-dimensional intelligent evaluation can efficiently handle complex evaluation tasks, automatically capture and analyze the behavioral characteristics of experts, and dynamically optimize the review model, thereby achieving the accuracy, fairness and transparency of the review process. This system provides new technical support for modern evaluation tasks and significantly improves the quality and efficiency of reviews.

[0004] 1. Patent document CN110097246B discloses an intelligent review method based on a multi-institutional shared expert database. The above patent achieves the purpose of multi-institutional shared expert database and improves review efficiency. At the same time, the review management agency achieves fine-grained management of the demanding institutions by granting different usage rights of the expert database to the sub-database clients, which improves the enthusiasm of the demanding institutions and the utilization rate of the expert database. However, the above patent cannot realize the function of in-depth analysis of expert scoring data.

[0005] 2. Patent document CN110826900B discloses a crowd-sourcing-based group intelligence review method. The above patent fully utilizes the online group intelligence resources of the Internet, ensuring the efficiency, convenience, scientificity and fairness of online review activities. However, the above patent cannot realize the function of capturing multimodal behavioral data in expert reviews.

[0006] 3. Patent document CN113887193B discloses a method, system, medium and electronic device for evaluating a degree thesis. The above patent realizes a more accurate and comprehensive evaluation of the thesis quality by clustering or classifying expert comments and combining the overall scores of other dimensions. However, the above patent cannot realize the function of dynamically adjusting model parameters and weight distribution.

[0007] 4. Patent document CN111008215B discloses an expert recommendation method that combines label construction with community relationship avoidance. The above patent effectively improves the work efficiency of expert selection, while improving the scientificity, fairness, independence and objectivity of scientific and technological evaluation or review. However, the above patent cannot realize the function of dynamically updating and optimizing the review model using feedback results.

[0008] In summary, the above patents cannot realize the functions of in-depth analysis of expert rating data, capture of multimodal behavior data in expert review, dynamic adjustment of model parameters and weight distribution, and dynamic update and optimization of review models using feedback results, resulting in problems such as strong subjectivity in review, poor fairness and scientificity, poor adaptability, and low review efficiency;

[0009] To this end, the present application proposes a multi-dimensional intelligent evaluation expert review system and evaluation method that can realize the functions of in-depth analysis of expert scoring data, capture of multimodal behavioral data in expert reviews, dynamic adjustment of model parameters and weight distribution, and dynamic update and optimization of review models using feedback results. Summary of the invention

[0010] The purpose of the present invention is to provide an expert review system and evaluation method for multi-dimensional intelligent evaluation, so as to solve the technical problems proposed in the above-mentioned background technology that the functions of in-depth analysis of expert scoring data, capture of multimodal behavior data in expert reviews, dynamic adjustment of model parameters and weight distribution, and dynamic update and optimization of the review model using feedback results cannot be realized, resulting in strong subjectivity in the review, poor fairness and scientificity, poor adaptability and low review efficiency.

[0011] To achieve the above-mentioned purpose, the present invention provides the following technical solutions: an expert review system for multi-dimensional intelligent evaluation, comprising an intelligent interactive perception module, a multi-dimensional intelligent analysis module, a dynamic model adaptive optimization module and an expert behavior deviation correction module, wherein the intelligent interactive perception module and the dynamic model adaptive optimization module provide deviation correction data support for the expert behavior deviation correction module, and the multi-dimensional intelligent analysis module performs multi-level and multi-dimensional fusion analysis based on the data output feedback of the intelligent interactive perception module and the dynamic model adaptive optimization module;

[0012] The multi-dimensional intelligent analysis module includes: data input, analysis technology and output results;

[0013] The data input includes structured data, i.e., expert rating tables, unstructured data, i.e., text descriptions, and multimodal behavioral data, wherein the multimodal data includes the expert's voice intonation, facial expressions, eye movement trajectories, gestures, and text content input in real time;

[0014] The analysis technology combines the BERT language model to perform text semantic analysis, builds a semantic network, uses a graph neural network GNN to process the multidimensional relationship of complex tasks, integrates multimodal behavior data and expert structured scoring data, and performs speech analysis and multi-dimensional comprehensive analysis on the review content based on a speech network and a dynamic weight model to generate preliminary review results. The review results are compared with the subsequent task execution effects through a closed-loop feedback mechanism to optimize the expert scoring behavior model and the system weight allocation model;

[0015] The output results provide multi-dimensional evaluation results, including accuracy of task completion, time efficiency and innovation evaluation scores.

[0016] Preferably, the intelligent interaction perception module collects the expert's scoring input and behavior data in real time, generates feature vectors and scoring data, and transmits the multimodal behavior data to the multi-dimensional intelligent analysis module;

[0017] The intelligent interactive perception module is equipped with multimodal sensors, including: high-definition cameras for capturing expert facial expressions and eye movement trajectories, microphone arrays for voice and intonation analysis, touch input devices for recording touch rhythm and strength, and brain wave perception devices;

[0018] Deep learning models are used to analyze experts’ facial expressions, voice intonation, gestures, and attention changes in real time. Deep learning models include OpenPose for posture estimation and Wav2Vec for speech analysis to evaluate experts’ real-time psychological state and consistency of behavioral decisions.

[0019] Preferably, the analysis results of the multi-dimensional intelligent analysis module are transmitted to the dynamic model adaptive optimization module for further adjustment, and preliminary scoring data are provided to the expert behavior deviation correction module for detection;

[0020] The dynamic model adaptive optimization module is based on reinforcement learning Deep Q-Learning. It combines expert historical scoring data with real-time scoring data, dynamically optimizes the evaluation algorithm, and automatically adjusts the expert's scoring weights and evaluation criteria to generate optimized comprehensive evaluation results. It also ensures decision transparency through the explanatory algorithm SHAP value.

[0021] Preferably, the scoring data after dynamic optimization by the dynamic model adaptive optimization module is transmitted to the expert behavior deviation correction module, and the scoring model is updated and returned to the multi-dimensional intelligent analysis module;

[0022] The expert behavior bias correction module uses the anomaly detection algorithm Isolation Forest to identify the biases in the expert scoring process, including empirical, emotional and social biases. By comparing the consistency of multi-dimensional expert scores, it provides personalized adjustment suggestions, seeks expert confirmation before correction to ensure respect for expert subjectivity, outputs the corrected scoring data and passes it to the collaborative feedback learning module, and the bias correction report is fed back to the expert terminal.

[0023] Preferably, the collaborative feedback learning module continuously updates the evaluation model based on the historical evaluation behaviors and evaluation results of the experts, and at the same time utilizes the differences in evaluation results across expert groups to promote collective learning optimization, and utilizes distributed learning Federated Learning to achieve model updates without leaking expert data;

[0024] The learning outcomes of the collaborative feedback learning module are returned to the dynamic model adaptive optimization module and the multi-dimensional intelligent analysis module to form an optimization closed loop.

[0025] Preferably, after the expert review system is initialized, the review tasks are dynamically allocated using a weighted recommendation algorithm based on task requirements, expert professional fields and historical task completion status through an intelligent resource allocation module.

[0026] Preferably, the dynamic weight optimization mechanism within the dynamic model adaptive optimization module uses the AHP hierarchical analysis method to initially set the dimension weights, combines expert behavior feedback, uses reinforcement learning to continuously optimize the weights, and dynamically adjusts the parameter configuration of the evaluation model based on the differences in characteristics of experts in different fields / tasks.

[0027] Preferably, the input data of the expert review system includes: task requirement description, expert information and background information of the object to be reviewed;

[0028] Task requirement description is structured data, including task type and evaluation criteria;

[0029] Expert information includes: expert professional field, historical rating records and preferences;

[0030] Background information of the object to be reviewed includes: project documents, solution description and technical indicators;

[0031] The output data of the expert review system includes: task characteristic analysis results and expert-task matching recommendations.

[0032] Preferably, the evaluation method comprises the following steps:

[0033] S1. Initialization task: receiving review task requirements, analyzing task characteristics, and assigning review experts through the intelligent resource allocation module;

[0034] S2, data collection: collect experts’ multimodal behavior data and structured scoring data through the intelligent interactive perception module;

[0035] S3. Comprehensive analysis: The multi-dimensional intelligent analysis module is used to perform voice analysis, weight allocation and multi-dimensional comprehensive analysis on the review content;

[0036] S4. Model optimization: Detect potential biases in scoring and generate correction suggestions through the expert behavior bias correction module.

[0037] Preferably, the evaluation method further comprises the following steps:

[0038] S5. Feedback learning: The evaluation results are compared and analyzed with the subsequent execution effects through the collaborative feedback learning module to optimize the system review algorithm;

[0039] S6. Generate the final multi-dimensional evaluation report, showing the comprehensive evaluation results and expert behavior characteristics analysis.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] 1. The present invention implements the function of in-depth analysis of expert scoring data by designing a multi-dimensional intelligent analysis module and an expert behavior deviation correction module, identifies scoring deviations in combination with behavioral characteristics, dynamically adjusts scoring weights, and generates comprehensive scoring results that are highly matched with task requirements. This solves the problem of large deviations in expert scoring due to subjectivity and inconsistent standards, reduces the impact of subjective bias on review results, makes scoring more accurate and fair, and improves the reliability of the system in complex review tasks;

[0042] 2. The present invention realizes the function of capturing multimodal behavior data in expert review by designing an intelligent interactive perception module, solves the problem that the traditional review method cannot quantify and analyze expert behavior data, improves the transparency and data support capability of the review process, and provides a reliable basis for subsequent intelligent analysis and deviation correction;

[0043] 3. The present invention realizes the function of dynamically adjusting model parameters and weight distribution by designing a dynamic model adaptive optimization module, which solves the problem that the traditional review model cannot adjust the weight of the scoring dimension in real time and is difficult to adapt to the requirements of diversified tasks, improves the accuracy and flexibility of the model review, and makes the results more in line with the task requirements;

[0044] 4. The present invention is designed with a collaborative feedback learning module to realize the function of dynamically updating and optimizing the review model using feedback results, which solves the problem that the traditional review system lacks self-learning and optimization capabilities and is difficult to continuously improve, gradually forms an efficient solution suitable for different review scenarios, and significantly improves the long-term practicality of the review system. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a schematic diagram of the review process of the present invention. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] See also Figure 1 , an embodiment provided by the present invention: an expert review system for multi-dimensional intelligent evaluation, comprising an intelligent interactive perception module, a multi-dimensional intelligent analysis module, a dynamic model adaptive optimization module and an expert behavior deviation correction module, wherein the intelligent interactive perception module and the dynamic model adaptive optimization module provide deviation correction data support for the expert behavior deviation correction module, and the multi-dimensional intelligent analysis module performs multi-level and multi-dimensional fusion analysis based on the data output feedback of the intelligent interactive perception module and the dynamic model adaptive optimization module;

[0048] The multi-dimensional intelligent analysis module includes: data input, analysis technology and output results;

[0049] The data input includes structured data, i.e., expert rating tables, unstructured data, i.e., text descriptions, and multimodal behavioral data, wherein the multimodal data includes the expert's voice intonation, facial expressions, eye movement trajectories, gestures, and text content input in real time;

[0050] The analysis technology combines the BERT language model to perform text semantic analysis, builds a semantic network, uses a graph neural network GNN to process the multidimensional relationship of complex tasks, integrates multimodal behavior data and expert structured scoring data, and performs speech analysis and multi-dimensional comprehensive analysis on the review content based on a speech network and a dynamic weight model to generate preliminary review results. The review results are compared with the subsequent task execution effects through a closed-loop feedback mechanism to optimize the expert scoring behavior model and the system weight allocation model;

[0051] The output result provides a multi-dimensional evaluation result, including the accuracy of task completion, time efficiency and innovation evaluation score;

[0052] The input data of the expert review system includes: task requirement description, expert information and background information of the object to be reviewed;

[0053] Task requirement description is structured data, including task type and evaluation criteria;

[0054] Expert information includes: expert professional field, historical rating records and preferences;

[0055] Background information of the object to be reviewed includes: project documents, solution description and technical indicators;

[0056] The output data of the expert review system include: task characteristic analysis results and expert-task matching suggestions;

[0057] Furthermore, data input includes the score sheet submitted by the expert through the terminal, with a score range of 0-10, including scoring data on technical innovation, feasibility of implementation and market potential, text comments filled in by the expert and behavioral data collected by multimodal sensors, including voice intonation, facial expressions, eye movement trajectories and gestures. The BERT model is used to extract semantic features and generate high-dimensional semantic vectors. The expert's emotional features such as concentration and emotional fluctuations and behavioral features such as consistency of movements during scoring are extracted through deep learning models such as OpenPose and Wav2Vec. Key concepts such as "breakthrough progress" and "engineering difficulties" in the expert's comments are extracted. The semantic associations between concepts are constructed through semantic similarity calculations. A semantic network diagram is constructed, in which nodes represent key concepts and edge weights represent association strength. The network structure is analyzed through the graph neural network GNN to identify high-impact nodes such as "breakthrough progress" that may be the core factors affecting the score.

[0058] Input text semantic vectors, multimodal behavioral features, and structured rating data into the Transformer architecture to establish a unified feature representation. Use a weighting mechanism to combine the importance weights of different data sources. For example, the weight of text comments is 0.4, the weight of behavioral features is 0.3, and the weight of structured ratings is 0.3. Analyze the consistency of experts' rating data and behavioral features. For example, experts give high scores to "technical innovation", but facial expressions are obviously uncertain. The expression consistency score in the behavioral features is 0.5. The system marks potential deviations for inconsistent behaviors for further processing by subsequent modules.

[0059] Based on task requirements, such as the weight of technological innovation being higher than market potential, the weights of each dimension are initially set, with technological innovation being 0.5, feasibility being 0.3, and market potential being 0.2. The weights are dynamically adjusted based on expert ratings and multimodal features. For example, if the behavioral characteristics of an expert indicate that his or her emotions fluctuate greatly when rating “feasibility of implementation,” the weight of the rating on this dimension is reduced, and the weighted summation method is used to generate a comprehensive rating result:

[0060]

[0061] Among them, W i is the weight, S iis the score of each dimension, i is the dimension number, and n is the number of dimensions;

[0062] The output results of the preliminary review include: comprehensive score, such as the comprehensive score of the project is 8.2, semantic network visualization: generate a semantic network diagram, mark high-impact nodes and key issues, such as "engineering problems", behavior consistency analysis: display the consistency between the expert behavior score and the actual score, mark possible deviations, and the preliminary review results are passed to the dynamic model adaptive optimization module and the expert behavior deviation correction module for subsequent optimization and adjustment;

[0063] Through the comprehensive processing of the multi-dimensional intelligent analysis module, the review results not only include the scoring data, but also provide in-depth analysis of expert behavior and semantic features. This module effectively improves the accuracy and transparency of the review work, helping decision makers to have a more comprehensive understanding of the project's strengths and potential problems.

[0064] See also Figure 1 , an embodiment provided by the present invention: an expert review system for multi-dimensional intelligent evaluation, after the expert review system is initialized, the intelligent resource allocation module dynamically allocates review tasks based on task requirements, expert professional fields and historical task completion status using a weighted recommendation algorithm;

[0065] The intelligent interaction perception module collects the expert's scoring input and behavior data in real time, generates feature vectors and scoring data, and transmits the multimodal behavior data to the multi-dimensional intelligent analysis module;

[0066] The intelligent interactive perception module is equipped with multimodal sensors, including: high-definition cameras for capturing expert facial expressions and eye movement trajectories, microphone arrays for voice and intonation analysis, touch input devices for recording touch rhythm and strength, and brain wave perception devices;

[0067] Use deep learning models to analyze experts’ facial expressions, voice intonation, gestures, and attention changes in real time. Deep learning models include OpenPose for posture estimation and Wav2Vec for voice analysis to assess experts’ real-time mental state and consistency of behavioral decisions.

[0068] Furthermore, the expert's facial expression changes, such as frowning and smiling, are collected through high-definition cameras, the expert's voice speech during the review process is collected through microphone arrays, the changes in the expert's gaze target and the dwell time are captured through eye trackers, and the expert's gesture changes, such as pointing and waving, are collected through motion capture equipment. The various behavioral characteristics of the expert are analyzed in real time through deep learning models, and multimodal behavioral analysis is performed.

[0069] Conduct consistency analysis on the emotional features of voice intonation and facial expression. For example, when an expert's tone is "excited" but his expression is "indifferent", it is marked as a potential inconsistent behavior. Match the expert's gaze area with the review content, such as the key points of the technical solution and the picture display. For example, if an expert gazes at the "technical innovation" description area for a long time and expresses an attitude of approval in his voice, it can help to judge his focus. Analyze the synchronization of gestures and voice content. For example, when an expert expresses a negative opinion, the "waving hand" action can enhance the credibility of the emotional features.

[0070] The speech, facial expression, eye movement and gesture features are vectorized and uniformly converted into high-dimensional feature representations. For example, the emotional feature vector of an expert is 0.8 (pleasure), 0.1 (anger), 0.1 (neutral), and the focus duration is 12 seconds (technical innovation). The multimodal fusion model Transformer is used to weight the features of different modalities. The weights are dynamically adjusted according to task requirements. For example, the weight of speech emotion analysis is increased in market potential assessment. The output multimodal behavior feature data is passed to the multi-dimensional intelligent analysis module to evaluate the accuracy of the comprehensive score and the correlation between the expert behavior features.

[0071] Through the work of the intelligent interactive perception module, the system can capture the multimodal behavior data of experts in real time, providing accurate and multi-dimensional information support for subsequent analysis. Especially in the analysis of behavioral consistency and the judgment of key focus areas, it effectively assists the multi-dimensional intelligent analysis module to optimize the fairness and transparency of the review results.

[0072] See also Figure 1 , an embodiment provided by the present invention: an expert review system for multi-dimensional intelligent evaluation, wherein the analysis results of the multi-dimensional intelligent analysis module are transmitted to the dynamic model adaptive optimization module for further adjustment, and preliminary scoring data are provided to the expert behavior deviation correction module for detection;

[0073] The dynamic model adaptive optimization module is based on reinforcement learning Deep Q-Learning. It combines the expert historical scoring data with the real-time scoring data to dynamically optimize the evaluation algorithm, automatically adjust the expert scoring weights and evaluation criteria, generate optimized comprehensive evaluation results, and ensure decision transparency through the explanatory algorithm SHAP value;

[0074] The dynamic weight optimization mechanism in the dynamic model adaptive optimization module uses the AHP hierarchical analysis method to initially set the dimension weights, combines expert behavior feedback, uses reinforcement learning to continuously optimize the weights, and dynamically adjusts the parameter configuration of the evaluation model based on the differences in the characteristics of experts in different fields / tasks;

[0075] Furthermore, the dynamic model adaptive optimization module receives the following data from the multi-dimensional intelligent analysis module: preliminary comprehensive scores, such as the preliminary scores of the project are 7.8 (innovation), 8.0 (feasibility), and 7.5 (commercial value); expert behavior characteristics: including scoring bias, emotional fluctuations, and scoring consistency; model weight distribution: the initial weights of each review dimension are 0.5 (innovation), 0.3 (feasibility), and 0.2 (commercial value). The module analyzes potential problems in the preliminary review results, including: expert scoring bias: an expert's score on the commercial value dimension is significantly lower than that of other experts, and there may be interference from personal preferences; conflict of importance between dimensions: the actual task requirements emphasize innovation, but the innovation dimension does not contribute enough in the preliminary comprehensive score;

[0076] The module dynamically adjusts the weights of each dimension according to task requirements to meet the following goals: Matching task requirements: Increase the weight of the innovation dimension from 0.5 to 0.6 and reduce the weights of other dimensions; Eliminate the influence of expert bias: Reweight or partially eliminate the expert scores with large deviations; The module uses the deep intensity learning algorithm Deep Q-Learning to continuously optimize the weight allocation strategy based on the feedback of task evaluation results; Set the reward function: Take the matching degree between the final review result and the actual task effect as the optimization goal, such as

[0077] R=-|S 评审 -S 任务需求 |

[0078] Among them, S 评审 is the comprehensive evaluation score, S 任务需求 Provide scoring criteria for actual task requirements;

[0079] The module combines the expert behavior data provided by the intelligent interactive perception module, including facial expression emotion features, voice emotion fluctuations and gestures. If an expert's behavior features are obviously inconsistent with the scoring results, for example, the score is high but the voice and expression show hesitation, the weight of the expert's score will be reduced, and the scores of experts with high behavioral consistency will be weighted and improved to enhance credibility. Through the Attention mechanism, the influence of each behavioral feature on the score optimization is learned, and a nonlinear mapping relationship between behavioral features and weight adjustment is constructed;

[0080] Recalculate the comprehensive score using the optimized weights:

[0081]

[0082] Among them, W i ' is the dimension weight after dynamic adjustment, S iFor preliminary scoring, for example, the optimized scores are 8.3 (innovation), 7.8 (feasibility), 7.2 (commercial value), 8.3 (innovation), 7.8 (feasibility), 7.2 (commercial value), 8.3 (innovation), 7.8 (feasibility), 7.2 (commercial value), which are more in line with the task requirements. Possible outliers in the scores (such as scores that significantly deviate from the scores of other experts) are identified. Outliers are removed or adjusted to reduce their impact on the overall results. The optimization results and adjusted scores are passed to the collaborative feedback learning module for further verification and learning of optimization strategies.

[0083] Through the work of the dynamic model adaptive optimization module, the preliminary review results were further optimized, eliminating individual expert bias and improving the fairness and rationality of the scoring results. The final review results are more in line with the task requirements, significantly improving the credibility and accuracy of the review system.

[0084] See also Figure 1 , an embodiment provided by the present invention: an expert review system for multi-dimensional intelligent evaluation, wherein the score data after dynamic optimization by the dynamic model adaptive optimization module is transmitted to the expert behavior deviation correction module, and the score model is updated and returned to the multi-dimensional intelligent analysis module;

[0085] The expert behavior deviation correction module uses the anomaly detection algorithm Isolation Forest to identify the deviations in the expert scoring process, including empirical, emotional, and social deviations. It provides personalized adjustment suggestions by comparing the consistency of multi-dimensional expert scores, and seeks expert confirmation before correction to ensure respect for expert subjectivity. It outputs the corrected scoring data and passes it to the collaborative feedback learning module, and the deviation correction report is fed back to the expert terminal;

[0086] Furthermore, the preliminary scoring results from the multi-dimensional intelligent analysis module, the voice, expression, eye movement and gesture features provided by the intelligent interactive perception module, and the mean and standard deviation of the overall score of the expert group are received as the basis for judging the deviation. For example, if an expert scores 9 in "technical innovation", but the voice emotion analysis shows hesitation (large fluctuations in tone) and a short gaze time (only 10% of the display time), the statistical analysis shows that the expert's score deviates significantly from other experts, and the deviation value exceeds 2 standard deviations;

[0087] Bias correction rules include: Behavior adjustment weight adjustment: If the behavior characteristics are inconsistent with the score, the score will be adjusted downward. For example, when the score of an expert in the "market potential" dimension is 30% higher than the average, but the emotional characteristics show "uncertainty" (confidence is only 0.4), the system adjusts the score of this dimension according to the weight to be closer to the mean; Score consistency optimization: Combined with the reference limit, the system corrects the part of the score that deviates significantly from the mean: S 修正 =S初始 +α·(S 均值 -S 初始 ) where α is the bias correction coefficient, which is dynamically adjusted according to the confidence of behavioral data; Expert rating preference compensation: Identify the expert's rating pattern on a certain dimension and compensate for other dimensions by weighted adjustment of the rating;

[0088] The expert behavior bias correction module effectively corrects the subjective bias in the scoring results, improving the fairness and scientificity of the review. The corrected scores are more in line with the overall standards of the team and reduce the discreteness of the review results.

[0089] See also Figure 1 , an embodiment provided by the present invention: an expert review system for multi-dimensional intelligent evaluation, wherein the collaborative feedback learning module continuously updates the evaluation model based on the historical evaluation behavior and evaluation results of the experts, and at the same time utilizes the differences in evaluation results across expert groups to promote collective learning optimization, and utilizes distributed learning Federated Learning to achieve model update without leaking expert data;

[0090] The learning results of the collaborative feedback learning module are returned to the dynamic model adaptive optimization module and the multi-dimensional intelligent analysis module to form an optimization closed loop;

[0091] Furthermore, the collaborative feedback learning module receives the corrected scoring data and historical review records output by the expert behavior deviation correction module: including scoring data, behavior characteristics and final task effects, and current task requirements: for example, the focus of this review task is on "technical innovation" and "implementation feasibility". The collaborative feedback learning module analyzes the core dimensions of the current task and their weight priorities, and dynamically adjusts the learning objectives;

[0092] The current task data is integrated with the historical task data to construct a multi-dimensional feature data set. The final task effect (such as the matching degree between the expert review results and the actual project success rate) is annotated as a supervision signal. The matching degree between the review results and the task requirements is predicted through a regression model (such as XGBoost or a neural network). The decision strategy of the review system is optimized using a deep reinforcement learning algorithm. The reward function is based on the matching degree of the results:

[0093] R=-|S 评审 -S 需求 |

[0094] According to the feedback data of the current task, the weight of multimodal features is adjusted in real time, such as increasing the weight of voice emotion features to strengthen their influence, recording the behavior patterns and scoring rules of experts in the review process, identifying high-credibility experts and learning their scoring styles first, and reducing the weight of experts with large scoring deviations in model optimization to avoid misleading the learning process. The system compares the optimized model score with the expert feedback to ensure that the results are transparent and explainable. If there is a significant deviation, it will be marked as abnormal data and enter the next round of optimization;

[0095] The collaborative feedback learning module achieves dynamic optimization and enhanced adaptability of the review model through deep learning of task data and historical records. The optimized system shows higher accuracy and robustness in subsequent tasks, significantly improving the efficiency and quality of the review.

[0096] Working principle: During the review process, the system captures the expert's multimodal behavior data, including voice, expression, eye movement and gesture, through the intelligent interaction perception module, and models the expert's emotional state, attention distribution and interaction behavior into a multi-dimensional feature vector as the input for subsequent analysis;

[0097] Then, after receiving the perception data, the system performs in-depth processing through a multi-dimensional intelligent analysis module, conducts a multi-dimensional evaluation of the score and corrects the expert scoring deviation. Combined with the key requirements of the review task, the system dynamically adjusts the proportion of the dimension weight to ensure that the review results match the actual needs;

[0098] Finally, the system evaluates the results of the review process through a collaborative feedback learning module, continuously optimizes model parameters, enhances the adaptability and robustness of the system, dynamically adjusts system parameters through feedback learning results, optimizes the mapping relationship between expert behavior characteristics and scoring results, and ensures the efficiency and accuracy of subsequent review tasks.

[0099] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A multi-dimensional intelligent evaluation expert review system, comprising an intelligent interactive perception module, a multi-dimensional intelligent analysis module, a dynamic model adaptive optimization module and an expert behavior deviation correction module, characterized in that: The intelligent interaction perception module and the dynamic model adaptive optimization module provide deviation correction data support for the expert behavior deviation correction module, and the multi-dimensional intelligent analysis module performs multi-level and multi-dimensional fusion analysis based on the data output feedback of the intelligent interaction perception module and the dynamic model adaptive optimization module; The multi-dimensional intelligent analysis module includes: data input, analysis technology and output results; The data input includes structured data, i.e., expert rating tables, unstructured data, i.e., text descriptions, and multimodal behavioral data, wherein the multimodal data includes the expert's voice intonation, facial expressions, eye movement trajectories, gestures, and text content input in real time; The analysis technology combines the BERT language model to perform text semantic analysis, builds a semantic network, uses a graph neural network GNN to process the multi-dimensional relationship of complex tasks, integrates multimodal behavior data and expert structured scoring data, and performs speech analysis and multi-dimensional comprehensive analysis on the review content based on a speech network and a dynamic weight model to generate preliminary review results. The review results are compared with the subsequent task execution effects through a closed-loop feedback mechanism to optimize the expert scoring behavior model and the system weight allocation model; The output results provide multi-dimensional evaluation results, including accuracy of task completion, time efficiency and innovation evaluation scores.

2. According to claim 1, the expert review system for multi-dimensional intelligent evaluation is characterized by: The intelligent interaction perception module collects the expert's scoring input and behavior data in real time, generates feature vectors and scoring data, and transmits the multimodal behavior data to the multi-dimensional intelligent analysis module; The intelligent interactive perception module is equipped with multimodal sensors, including: high-definition cameras for capturing expert facial expressions and eye movement trajectories, microphone arrays for voice and intonation analysis, touch input devices for recording touch rhythm and strength, and brain wave perception devices; Deep learning models are used to analyze experts’ facial expressions, voice intonation, gestures, and attention changes in real time. Deep learning models include OpenPose for posture estimation and Wav2Vec for speech analysis to evaluate experts’ real-time psychological state and consistency of behavioral decisions.

3. The multi-dimensional intelligent evaluation expert review system according to claim 1, characterized in that: The analysis results of the multi-dimensional intelligent analysis module are transmitted to the dynamic model adaptive optimization module for further adjustment, and preliminary scoring data is provided to the expert behavior deviation correction module for detection; The dynamic model adaptive optimization module is based on reinforcement learning Deep Q-Learning. It combines expert historical scoring data with real-time scoring data, dynamically optimizes the evaluation algorithm, and automatically adjusts the expert's scoring weights and evaluation criteria to generate optimized comprehensive evaluation results. It also ensures decision transparency through the explanatory algorithm SHAP value.

4. The multi-dimensional intelligent evaluation expert review system according to claim 3 is characterized by: The scoring data after dynamic optimization by the dynamic model adaptive optimization module is transmitted to the expert behavior deviation correction module, and the scoring model is updated and returned to the multi-dimensional intelligent analysis module; The expert behavior bias correction module uses the anomaly detection algorithm Isolation Forest to identify the biases in the expert scoring process, including empirical, emotional and social biases. By comparing the consistency of multi-dimensional expert scores, it provides personalized adjustment suggestions, seeks expert confirmation before correction to ensure respect for expert subjectivity, outputs the corrected scoring data and passes it to the collaborative feedback learning module, and the bias correction report is fed back to the expert terminal.

5. The multi-dimensional intelligent evaluation expert review system according to claim 4 is characterized by: The collaborative feedback learning module continuously updates the evaluation model based on the historical evaluation behaviors and evaluation results of experts, and at the same time utilizes the differences in evaluation results across expert groups to promote collective learning optimization, and uses distributed learning Federated Learning to achieve model updates without leaking expert data; The learning outcomes of the collaborative feedback learning module are returned to the dynamic model adaptive optimization module and the multi-dimensional intelligent analysis module to form an optimization closed loop.

6. The multi-dimensional intelligent evaluation expert review system according to claim 1, characterized in that: After the expert review system is initialized, the intelligent resource allocation module dynamically allocates review tasks using a weighted recommendation algorithm based on task requirements, expert professional fields and historical task completion status.

7. The multi-dimensional intelligent evaluation expert review system according to claim 1 is characterized by: The dynamic weight optimization mechanism in the dynamic model adaptive optimization module uses the AHP hierarchical analysis method to initially set the dimension weights, combines expert behavior feedback, and uses reinforcement learning to continuously optimize the weights. It dynamically adjusts the parameter configuration of the evaluation model based on the differences in the characteristics of experts in different fields / tasks.

8. The multi-dimensional intelligent evaluation expert review system according to claim 1, characterized in that: The input data of the expert review system includes: task requirement description, expert information and background information of the object to be reviewed; Task requirement description is structured data, including task type and evaluation criteria; Expert information includes: expert professional field, historical rating records and preferences; Background information of the object to be reviewed includes: project documents, solution description and technical indicators; The output data of the expert review system includes: task characteristic analysis results and expert-task matching recommendations.

9. An evaluation method for an expert review system for multi-dimensional intelligent evaluation, applicable to an expert review system for multi-dimensional intelligent evaluation according to any one of claims 1 to 8, characterized in that: The evaluation method comprises the following steps: S1. Initialization task: receiving review task requirements, analyzing task characteristics, and assigning review experts through the intelligent resource allocation module; S2, data collection: collect experts’ multimodal behavior data and structured scoring data through the intelligent interactive perception module; S3. Comprehensive analysis: The multi-dimensional intelligent analysis module is used to perform voice analysis, weight allocation and multi-dimensional comprehensive analysis on the review content; S4. Model optimization: Detect potential biases in scoring and generate correction suggestions through the expert behavior bias correction module.

10. The evaluation method of the expert review system for multi-dimensional intelligent evaluation according to claim 9 is characterized by: The evaluation method further comprises the following steps: S5. Feedback learning: The evaluation results are compared and analyzed with the subsequent execution effects through the collaborative feedback learning module to optimize the system review algorithm; S6. Generate the final multi-dimensional evaluation report, showing the comprehensive evaluation results and expert behavior characteristics analysis.

Images