74 results about "Meta evaluation" patented technology

The invention provides an evaluation model optimization and dynamic calibration method based on meta evaluation feedback, belongs to the field of artificial intelligence model evaluation, and constructs an evaluation system with a real-time feedback optimization capability through cooperative operation of a main evaluation model and a meta evaluation model in combination with a dynamic knowledge graph and a reinforcement learning framework. The main evaluation model is responsible for executing an original answer matching task, the meta evaluation model realizes error correction and logic optimization through variance analysis, semantic alignment and adversarial verification, the double models form a mutual verification mechanism through adversarial training, and dynamic disturbance is injected by utilizing a generative adversarial network to improve the robustness of the system. The adaptive ability of the evaluation system in a model iteration and knowledge extension scene is significantly improved, and a reliable evaluation criterion is provided for continuous optimization of an artificial intelligence model.

An evaluation system, method and program product based on a large language model relate to the technical field of artificial intelligence. The system obtains an evaluation object and an evaluation task configuration, constructs an evaluation structure graph containing theme segments, argument segments, basis segments and reference segments and their support, reference and conflict relations, filters key semantic units, and constructs a coupling representation between key semantic units, evaluation dimensions, dimension rules, candidate evidence segments, evidence positions and unit-evidence relationship markers; input the coupling representation into a large language model to generate an initial evaluation result; construct a statement disturbance set around the key semantic units that keeps the evaluation theme and core facts unchanged, obtain consistency indicators through re-evaluation, and calculate joint confidence by combining evidence coverage, evidence conflict and evidence chain drift; and perform evidence weighting and local calibration on low-confidence target evaluation dimensions. The scheme can improve the accuracy, stability, explainability and traceability of the evaluation result.

To provide an evaluation system, an evaluation method and a control program capable of quantitatively evaluating the effect of learning.SOLUTION: The evaluation system evaluates the effect of learning. At each time point before and after the learning, questions related to the content of the learning are asked to the subject of the learning. The evaluation system includes an acquisition unit, an evaluation unit, and an output unit. The acquisition unit acquires natural language data indicating an answer to a question. The evaluation unit evaluates the answer to the question by performing natural language processing on the natural language data. The output unit outputs an evaluation result of an answer to a question asked at each time point.SELECTED DRAWING: Figure 1

The system according to the embodiment aims to objectively and efficiently evaluate the abilities of players. [Solution] A system according to an embodiment includes a collection unit, an analysis unit, an evaluation unit, and a provision unit. The collection unit collects data on players. The analysis unit analyzes the data collected by the collection unit. The evaluation unit assigns scores based on the analysis results obtained by the analysis unit. The provision unit provides the scores assigned by the evaluation unit to the players.

The system of the embodiment aims to visualize the reliability of answers provided by the generation AI so that users can obtain appropriate information. [Solution] A system according to an embodiment includes a generation unit, a reliability evaluation unit, a visualization unit, an analysis unit, and a provision unit. The generation unit uses a generation AI to generate answers to questions from users. The reliability evaluation unit evaluates the reliability of the answers generated by the generation unit. The visualization unit visualizes the results evaluated by the reliability evaluation unit. The analysis unit analyzes the user's interests and concerns. The provision unit provides information selected based on specific criteria using the results analyzed by the analysis unit.

The system according to the embodiment aims to automate the process of evaluating the validity of estimates and enable flexible staffing. [Solution] A system according to an embodiment includes a reception unit, a collection unit, an evaluation unit, and a provision unit. The reception unit receives input of quotation data. The collection unit analyzes the quotation data received by the reception unit and collects data on similar past cases or market prices. The evaluation unit evaluates the validity of the quotation based on the data collected by the collection unit. The provision unit provides the evaluation results obtained by the evaluation unit.

The system according to the embodiment aims to assess an individual's stress level and provide personalized advice. [Solution] A system according to an embodiment includes a reception unit, an analysis unit, an evaluation unit, and a provision unit. The reception unit receives questions. The analysis unit analyzes the answers received by the reception unit. The evaluation unit evaluates the stress level based on the answers analyzed by the analysis unit. The provision unit provides personalized advice based on the results of the evaluation by the evaluation unit.

The system according to the embodiment aims to efficiently and accurately perform risk analysis and verdict prediction in civil trials. [Solution] A system according to an embodiment includes a risk analysis unit, an evidence analysis unit, a verdict analysis unit, and a simulation provision unit. The risk analysis unit identifies risks. The evidence analysis unit evaluates evidence based on the risks identified by the risk analysis unit. The verdict analysis unit predicts a verdict based on the evidence evaluated by the evidence analysis unit. The simulation provision unit provides a simulation based on the verdict outcome predicted by the verdict analysis unit.

The system according to the embodiment aims to evaluate the degree of AI generation of content provided by a user. [Solution] A system according to an embodiment includes a reception unit, an analysis unit, an evaluation unit, and a provision unit. The reception unit receives content from a user. The analysis unit analyzes the content received by the reception unit. The evaluation unit evaluates the AI ​​generation level based on the results of the analysis by the analysis unit. The provision unit provides the evaluation results obtained by the evaluation unit.

An object of the system according to the embodiment is to analyze a user's action and provide specific feedback.SOLUTION: A system includes an action analysis part, a scoring part, an evaluation part, and a feedback part. The action analysis unit analyzes an action of a user. The scoring unit scores the user's action analyzed by the action analysis unit. The evaluation unit evaluates the improvement with respect to the action of the user scored by the scoring unit. The feedback unit provides specific feedback to the action of the user evaluated by the evaluation unit.SELECTED DRAWING: Figure 1

An object of a system according to an embodiment is to evaluate the accuracy of an answer of a AI model and provide an improvement.SOLUTION: A system according to an embodiment includes an answer examination unit, an evaluation unit, a feedback providing unit, and an other-company AI model use unit. The answer examination unit examines an answer of the AI model. The evaluation part evaluates the accuracy of the answer examined by the answer examination part. The feedback providing unit provides a basis of evaluation and an improvement point based on a result evaluated by the evaluation unit. The counterpart company AI model using unit performs an examination using a plurality of counterpart company AI models.SELECTED DRAWING: Figure 1

An object of a system according to an embodiment is to efficiently advance learning in an examination study.SOLUTION: A system includes a question setting part, a digitalization part, a plan creation part, a progress management part, and an evaluation part. The question setting unit analyzes the learning progress or the degree of understanding of the user, and sets an optimum question based on the analysis. The digitizing section digitizes the plurality of reference books and makes them accessible to the user when needed. The plan creation unit creates an optimal learning plan based on the user's goal or learning progress. The progress manager manages the user's learning progress in real-time and provides feedback as needed. The evaluation unit evaluates the learning outcome of the user and determines whether to proceed to the next step.SELECTED DRAWING: Figure 1

Embodiments of the present specification provide a task processing model training method, a role-playing model training method and a task processing method, wherein the task processing model training method comprises: obtaining a plurality of sample reply contents, the plurality of sample reply contents being reply contents generated by a task processing model based on sample dialogue data; inputting the plurality of sample reply contents into a target evaluation model to obtain target reply indicators corresponding to the plurality of sample reply contents respectively, the target reply indicators being used to measure the quality of the corresponding sample reply contents, the target evaluation model being obtained by reinforcement learning of a cold start evaluation model based on a meta evaluation model, the meta evaluation model being trained based on sample dialogue data, a plurality of sample reply contents and sample reply analysis information; and adjusting parameters of the task processing model according to the target reply indicators to obtain a trained task processing model. Based on the target reply indicators, the training efficiency and alignment performance of the task processing model are improved.

The system according to this embodiment aims to dynamically update the evaluation criteria for ideas and efficiently select the best ideas. [Solution] The system according to the embodiment comprises a collection unit, an evaluation unit, and an update unit. The collection unit collects submitted ideas. The evaluation unit evaluates the ideas collected by the collection unit. The update unit updates the evaluation criteria for the ideas evaluated by the evaluation unit.

An object of a system according to an embodiment is to improve the efficiency of an interview process.SOLUTION: A system includes an interaction unit, an analysis unit, an evaluation unit, a selection unit, and a specification unit. The dialogue unit performs a dialogue with the candidate. The analysis unit analyzes the dialogue content obtained by the dialogue unit. The evaluation unit evaluates the strengths and characteristics of the candidates based on the contents analyzed by the analysis unit. The selection unit selects an appropriate second interviewer based on the strengths and characteristics evaluated by the evaluation unit. The specifying unit specifies an optimal position based on the strength and the characteristic evaluated by the evaluating unit.SELECTED DRAWING: Figure 1

An object of a system according to an embodiment is to effectively refine the skill of problem solving using generated AI.SOLUTION: A system according to an embodiment includes a task providing unit, an evaluation unit, an education resource providing unit, a task taking unit, and a promotion unit. The task providing unit provides a task using the generated AI. The evaluation unit evaluates the solution to the problem provided by the problem providing unit from the viewpoint of creativity and technical ability. The education resource providing unit provides an education resource that allows beginners to advanced users to develop skills. The problem take-in section takes in actual industrial and academic problems. The propagation promotion unit promotes the propagation and innovation of the generated AI.SELECTED DRAWING: Figure 1

An object of the system according to the embodiment is to evaluate hidden skills and personality characteristics of individuals and propose a career development plan based on the evaluation.SOLUTION: A system includes an evaluation part, an analysis part, a proposal part, and a company use part. The evaluator evaluates a skill or personality trait of the user. The analysis unit analyzes the evaluation result obtained by the evaluation unit. The proposing section proposes a career development plan based on the analysis result obtained by the analyzing section. A company use part allows the company to formulate an employee arrangement or raising plan on the basis of the plan proposed by the proposal part.SELECTED DRAWING: Figure 1

An object of the system according to the embodiment is to provide highly reliable information and solve user's questions and troubles.SOLUTION: A system includes a question analysis part, an information provision part, and a reliability evaluation part. The question analysis unit analyzes a question of a user. The information providing unit provides information based on the question analyzed by the question analysis unit. The reliability evaluation unit evaluates reliability of the information provided by the information providing unit.SELECTED DRAWING: Figure 1

An object of a system according to an embodiment is to improve the efficiency of employee performance evaluation.SOLUTION: A system includes a self-evaluation analysis part, a calendar connection part, a project evaluation part, and a target achievement evaluation part. The self-evaluation analysis unit analyzes the employee's self-evaluation. The calendar connector organizes the evaluation scores based on the self-evaluation analyzed by the self-evaluation analyzer. A project evaluation part evaluates the project on the basis of the evaluation points arranged by the calendar connection part. A target achievement evaluation part evaluates the degree of achievement to the set target on the basis of the information of the project evaluated by the project evaluation part.SELECTED DRAWING: Figure 1

This invention provides a method for optimizing and dynamically calibrating an evaluation model based on meta-evaluation feedback, belonging to the field of artificial intelligence model evaluation. This invention constructs an evaluation system with real-time feedback optimization capabilities through the collaborative operation of a main evaluation model and a meta-evaluation model, combined with a dynamic knowledge graph and a reinforcement learning framework. The main evaluation model is responsible for performing the original answer matching task, while the meta-evaluation model achieves error correction and logical optimization through variance analysis, semantic alignment, and adversarial verification. The two models form a mutual verification mechanism through adversarial training, and dynamic perturbations are injected using generative adversarial networks to improve system robustness. This significantly improves the adaptive capability of the evaluation system in model iteration and knowledge expansion scenarios, providing a reliable evaluation benchmark for the continuous optimization of artificial intelligence models.

An object of a system according to an embodiment is to analyze a user's action of writing a character and present a specific improvement measure.SOLUTION: A system includes an analysis part, an evaluation part, and an improvement plan presentation part. The analysis unit analyzes a user's action of writing a character in real time using the generated AI. The evaluation unit evaluates the writing pressure, the balance of the character, and the shape of the character based on the data acquired by the analysis unit. The improvement plan presentation unit presents a specific improvement plan on the basis of a result evaluated by the evaluation unit.SELECTED DRAWING: Figure 1

This system provides an analytical support device, analytical support method, and analytical support program that contribute to improving the accuracy of analysis of members and the organization by providing a list of evaluations of the introspection of multiple members. [Solution] The analysis support device 15 comprises an analyst information acquisition unit 23, a member information storage unit 21, an evaluation acquisition unit 25, and a display control unit 27. The analyst information acquisition unit 23 acquires first to third analysis elements for analyzing each of the multiple members that constitute the organization. The member information storage unit 21 stores the members' answers to each of the multiple items related to introspection, associating them with the items. The evaluation acquisition unit 25, with one member selected, acquires the level of the first to third analysis elements based on the answers of the selected member. The display control unit 27 displays a scatter plot image in which marks corresponding to the levels of each of the multiple members and three axes corresponding to the first to third analysis elements are arranged in a virtual three-dimensional space.

The present application relates to the technical field of intelligent agricultural data auditing, in particular to a non-corresponding auditing system for Internet of Things data sharing, which comprises data collection and association, multi-modal data fusion, self-adaptive calibration and calibration evaluation feedback units. The present application collects and associates multi-modal data through the data collection and association unit, extracts features using a convolutional neural network, obtains precise fusion feature vectors through deep data fusion by means of a multi-modal feature fusion model and various optimization techniques, dynamically adjusts calibration model parameters in combination with historical data by means of a reinforcement learning algorithm and a deep Q network in the self-adaptive calibration unit, and evaluates the calibration effect in the calibration evaluation feedback unit. If the effect is not as expected, optimization is fed back. Meanwhile, decision support is provided based on the calibrated data, effectively solving the problem of data accuracy caused by the complexity of the agricultural environment and improving the reliability of data auditing.

An object of a system according to an embodiment is to appropriately grasp desires and needs of employees and propose an optimal career plan and an appropriate arrangement of materials.SOLUTION: A system includes a collection unit, an analysis unit, an evaluation unit, and a proposal unit. The collection unit collects requests and needs of employees. The analysis unit analyzes the information collected by the collection unit and proposes a career plan. The evaluation part analyzes the resume of the applicant and evaluates the aptitude and skill. The proposal unit proposes an appropriate material arrangement based on the information obtained by the evaluation unit.SELECTED DRAWING: Figure 1

An object of a system according to an embodiment is to efficiently obtain advice from an expert outside a company in management decision.SOLUTION: A system includes a reception part, an analysis part, an advice part, and an evaluation part. The reception unit receives a question or consultation content of a user. The analysis unit analyzes the information received by the reception unit. The advice unit provides advice based on the information analyzed by the analysis unit. The evaluation unit evaluates the reliability of the advice provided by the advice unit.SELECTED DRAWING: Figure 1

The invention is suitable for the technical field of collection training, and provides a self-adaptive collection learning path planning method and system, and the method comprises the steps: constructing a three-dimensional procurement capability model containing compliance operation, a negotiation strategy and risk management and control, decomposing the procurement capability model into capability units, and setting a quantitative index; collecting operation, decision and assessment data in the simulation scene in real time, and generating a dynamic capability assessment matrix; constructing a three-dimensional space model based on the matrix, calculating the association strength and the lifting priority of the power unit, and outputting a personalized learning path; recombining the knowledge module according to the evaluation value change of the capability unit; dynamically adjusting model parameters in combination with policies and industries and re-planning a path; a cross-dimension learning path is generated when the plurality of capability units reach the standard. According to the method, dynamic evaluation, personalized path adjustment and cross-dimension capability integration can be realized, the problems of one-sided evaluation, fixed path and insufficient adaptability in the prior art are solved, and the accuracy and efficiency of auditory collection training are improved.

The system according to the embodiment aims to reduce the workload of teachers and maintain consistency in the evaluation of school report marks. [Solution] The system according to the embodiment includes a support unit, an analysis unit, an evaluation unit, and a monitoring unit. The support unit supports teachers in their work. The analysis unit analyzes students' academic performance data based on the work supported by the support unit. The evaluation unit calculates school report scores based on the data analyzed by the analysis unit. The monitoring unit continuously monitors the school report scores calculated by the evaluation unit and reviews them based on specific conditions.

The system according to the embodiment aims to improve the fairness and efficiency of the idea evaluation process. [Solution] A system according to an embodiment includes a collection unit, an analysis unit, an evaluation unit, and a provision unit. The collection unit collects data. The analysis unit analyzes the data collected by the collection unit. The evaluation unit performs evaluation based on the analysis results obtained by the analysis unit. The provision unit provides the evaluation results obtained by the evaluation unit.

An object of a system according to an embodiment is to objectively perform work management and evaluation of employees.SOLUTION: A system includes a business management part, an evaluation part, and a prohibition part. The work management part manages the work of the employee. The evaluation part evaluates the performance and action of the employee. The prohibition unit prohibits direct intervention by a human supervisor.SELECTED DRAWING: Figure 1

The system according to this embodiment aims to verify the truthfulness of the inference results of the generated AI program with a high probability and to ensure reliability. [Solution] The system according to the embodiment comprises an evaluation unit, a verification unit, and an assignment unit. The evaluation unit evaluates the generated AI program. The verification unit verifies the truth or falsity of the inference results of the generated AI program evaluated by the evaluation unit. The assignment unit assigns certification batches based on the results verified by the verification unit.