29 results about "Scoring criteria" patented technology

The invention relates to the technical field of urban drainage pipe network detection and evaluation, in particular to an intelligent pipeline defect evaluation method based on a multi-modal large model. The evaluation method comprises the following steps: firstly, carrying out defect identification on a pipeline detection video based on a deep learning model to obtain a series of defect images containing defect types and positions, and carrying out statistics and segmented scoring on the positions of the defects and the number and distribution of pipe section defects; then, the obtained images with defects are transmitted to a multi-modal large model for deep analysis, scoring is performed from multiple aspects such as the position and size of the defects, the influence on the pipeline structure and the use stability, and the scoring standard and rule are unified by adopting fixed cue words; and finally, weighting by using the two scoring results to obtain the comprehensive score of each pipe section, and synthesizing the scores of each pipe section into the comprehensive score of the pipeline. The evaluation objectivity and accuracy are remarkably improved, the detection and evaluation efficiency is greatly improved, the maintenance decision is more scientific and accurate, and the pipe network management level and safety are improved.

The invention relates to the technical field of intelligent scoring, in particular to a teacher teaching skill multi-dimensional intelligent scoring method and system.The method comprises the following steps that based on a teaching behavior multi-dimensional evaluation system, scoring standard connectivity strength is recognized, behavior consistency and language stability are analyzed, a feature behavior set is extracted, adjacent areas are expanded, interference nodes are removed, and the scoring standard connectivity strength is obtained; marking isolated point and breakpoint complementation structures, combining small regions, removing fragments, extracting behavior coding sequences, clustering and distributing labels, identifying index mapping, and outputting a teacher teaching skill classification and partition identification label set. The judgment of behavior consistency and language expression stability on skill features is enhanced, the accuracy and real-time performance of an evaluation result are improved, particularly, the strength and deficiency of teachers are meticulously analyzed and fed back in a complex teaching scene, teacher growth and teaching quality optimization are effectively supported, the high efficiency and unbiasedness of an evaluation framework are improved, and the evaluation efficiency is improved. And valuable education decision support is provided.

This invention relates to the field of artificial intelligence education technology, specifically to a method and system for reviewing and providing feedback on physics, chemistry, and biology lab reports based on a multimodal large-scale model. The method includes: acquiring lab report question data, student answer data, and scoring standard data; vectorizing the question data and retrieving scoring criteria from a scoring knowledge base; selecting either a text-based or image-based scoring process based on the answer type, and calling a fine-tuned multimodal large-scale language model for scoring reasoning; calling an error identification module and a teaching suggestion generation module based on the scoring results; summarizing the complete review results and calculating the joint confidence level; and determining whether to push the results for manual review based on the confidence level. This invention can improve the accuracy, fine-grained understanding, and quality of teaching feedback in lab report review, enabling the self-evolution of the review system.

The application discloses a kind of vertical domain intelligent agent reinforcement learning method based on big language model adjudication mechanism and GRPO, which comprises: first, detailed scoring criteria are formulated according to task scenarios;Using vLLM engine to accelerate inference, multiple rounds of sampling are carried out for each decision of the intelligent agent, a big language model is used as a judge model, and each sample of the intelligent agent is scored according to the detailed scoring criteria defined by the user;After the intelligent agent inference is completed, multiple complete trajectories are obtained, and the influence weight of the decision on the final reply is calculated using the exponential decay method;The scores of all decisions of the entire trajectory are weighted and summed to obtain the total score of the entire trajectory;Then, according to the loss calculation formula of GRPO, the corresponding importance sampling ratio, in-group advantage and other parameters are calculated, and finally the loss value is calculated, and the parameters are optimized by back propagation. The present application combines the exponential decay credit allocation mechanism and the in-group advantage optimization strategy of GRPO to realize efficient, stable and interpretable reinforcement learning of the multi-step decision-making process of the intelligent agent.

This invention discloses a method, device, and medium for safety training in the power industry based on virtual reality, relating to the field of power simulation technology. The method includes: acquiring raw data of operational behavior, including timestamps, hand spatial trajectories, action triggering states, and power equipment numbers; generating action segment data based on the hand spatial trajectories and action triggering states, and constructing action chain data based on the timestamps and power equipment numbers; calculating and generating sequence deviation scores and path offset scores based on the action chain data and standard process data; generating evaluation score records based on the sequence deviation scores, path offset scores, and duration in the action segment data; and generating evaluation results containing step numbers, score values, and action offset markers based on the evaluation score records. This invention achieves automatic identification and deviation quantification evaluation of key action steps in power operation processes without the need for manual expert review or subjective scoring standards.

The application discloses a kind of based on dark mark random matching's method and system for evaluating bid, it is related to intelligent bid evaluation technical field, including: the dark mark file to be evaluated is divided into multiple scoring points and the scoring standard corresponding to each scoring point;First random grouping is carried out to all scoring points, and multiple sets of scoring modules are formed according to the number of experts;Each set of scoring items includes a preset number of scoring points;Each set of scoring items is secondly randomly distributed, so that each expert corresponds to a set of scoring items;The scoring points in each set of scoring items are thirdly randomly distributed, in the case that each scoring point is scored by at least two experts, the number of scoring points is less than the preset number of scoring points included in each set of scoring items, the scoring points after third random distribution are distributed to the experts corresponding to the set of scoring items;The scores of each expert for each scoring point are counted, and a scoring result is generated.The application can ensure that experts can see the real work and reduce bias and subjectivity.

The present application relates to the technical field of data processing, in particular to a multi-source data fusion cross-role collaborative quantification and instruction system and method, comprising: a data acquisition module, which acquires multi-source heterogeneous data of multiple operation roles; an individual characteristic representation module, which constructs an individual characteristic vector; a collaborative behavior knowledge base module, which stores a standardized behavior evaluation scale containing observable behavior items and hierarchical scoring criteria; a collaborative state quantification module, which calculates collaborative quantification characteristics based on the individual characteristic vector and the observable behavior items; a feature node identification module, which identifies global extreme value feature nodes, and sorts the contribution degrees of the global extreme value feature nodes and the collaborative quantification characteristics to the abnormal prediction of a target operation system; and an instruction generation module, which retrieves the observable behavior items based on the contribution degree sorting and generates a structured behavior correction instruction set.The present application realizes the quantitative representation and abnormal attribution of the cross-role collaborative state, and establishes an automatic mapping link from quantitative analysis to executable instructions.

An artificial-intelligence-based decision-assisting system and method are disclosed for generating ranked recommendations through adaptive multi-source analysis. The system includes a processor and a non-transitory computer-readable storage medium storing executable instructions that implement a scoring engine and a ranking engine. Decision parameters and user-defined weighting factors are received from user devices and combined with system-defined weighting factors retrieved from behavioral, historical, external-context, and scoring-criteria databases. Composite decision scores are computed and used to rank candidate options. The system iteratively updates weighting factors or rankings based on feedback data and dynamically restricts data retrieval to relevant parameters to reduce latency and improve throughput. Results are displayed via a graphical user interface, and anonymization procedures protect user identity. The method supports concurrent processing and adaptive learning to refine decision predictions.

Systems, devices, and methods for an automated grading system (111) configured to: retrieve, for each essay submission, a corresponding prompt, rubric, sample answers, and instructions from the database (110) using an essay question ID; transmit the essay submission, a batch of rubric items (112), and instructions to an evaluation assistant component (113); receive item-level scores and feedback (115) for each rubric item of the batch of rubric items from the evaluation assistant component (113); iterate the transmission and reception of each rubric item of the batch of rubric items and feedback until all rubric items are evaluated; transmit the item-level scores and feedback to a summarization assistant (116) configured to compile a structured feedback summary (117); deliver formatted feedback (118) to the student and to human graders (119); and implement guardrails (120) to flag abnormal submissions for human review and direct human grader intervention (121).

A method and system of fine-tuning large language models is disclosed. The method includes receiving a user input corresponding to an LLM. The user input includes a selection of a set of target layers, predefined scoring criteria, and a distribution ratio. The method further includes determining a score corresponding to each of a plurality of weights of each of the set of target layers based on the predefined scoring criteria; for each of the set of target layers, classifying the plurality of weights into a set of trainable weights and a set of non-trainable weights based on the score and the distribution ratio; and for each of the set of target layers, modifying the set of trainable weights using a domain-specific training dataset to obtain a fine-tunned LLM.

This invention discloses a method and system for evaluating large language models based on multi-model cross-evaluation. The method includes the following steps: Step S01. Select the current question-generating model from multiple candidate models, and use prompt word chains to sequentially achieve knowledge extraction, creative reorganization, and self-consistency of the evaluation system to generate a structured task package; Step S02. Send the task package to the remaining models except the question-generating model, and inject a unified answer prompt word into each model. Each model independently analyzes the questions in the task package and provides an answer; Step S03. Dynamically generate review prompt words according to the scoring criteria to guide the models to evaluate the answers of all models, obtain the evaluation results of each model, and return to Step S01 to execute the next round of evaluation until a preset round is reached; Step S04. Combine the scoring data from multiple rounds to obtain the evaluation ranking result. This invention can achieve an objective, fair, professional, and reliable automated evaluation of the performance of large language models.

To provide a system capable of comprehensively analyzing the grammar, punctuation, spelling, and consistency, clarity, and style of a sentence of an essay and reducing the scoring burden of a teacher.SOLUTION: The system includes a grammar analyzer, a punctuation analyzer, a spelling analyzer, a sentence consistency analyzer, a clarity analyzer, a style analyzer, a summary generator, and a scoring criteria setter. Using the generated AI, the grammar analyzer comprehensively analyzes the grammar of the essay, the punctuation analyzer comprehensively analyzes the punctuation of the essay, the spelling analyzer comprehensively analyzes the spelling of the essay, the sentence coherence analyzer comprehensively analyzes the sentence coherence of the essay, the clarity analyzer comprehensively analyzes the clarity of the essay, and the style analyzer comprehensively analyzes the style of the essay. The summary generation unit generates a summary from which the core content of the essay can be quickly grasped. The scoring standard setting unit sets a customizable scoring standard.SELECTED DRAWING: Figure 1

The application discloses a method for enhancing teaching evaluation credibility in an education large model, and the method comprises the following steps: obtaining a scoring prompt, obtaining first scoring evaluation information of answer content according to the scoring prompt; obtaining an interactive object corresponding to the first scoring evaluation information, and constructing the scores of different users into a constructed historical score sequence under the same scoring standard; obtaining a dynamic smoothing coefficient according to the historical score sequence; obtaining the feedback quality of the first scoring evaluation information according to the historical score sequence and the dynamic smoothing coefficient; in the case that the feedback quality of the first scoring evaluation information is high, determining that the first scoring evaluation information meets the standard, and sending the first scoring evaluation information to a first terminal. Through the change of the dynamic proportion of new and old information, the feedback quality of the first scoring evaluation information is quantitatively evaluated. This makes the fluctuation of the scoring results of different batches and different times controllable, and forms a scoring basis with strong interpretability.

This invention discloses an automatic scoring system and optimization method for picture-based writing in primary school Chinese language teaching, belonging to the field of intelligent education technology. The system includes an image understanding module, which analyzes the core elements, themes, and logic of images through a lightweight object detection and scene classification model; a text analysis module extracts text features from the dimensions of content relevance, language standardization, logical completeness, and imagination; a scoring model module outputs scores using a hybrid strategy of "rule-based scoring + model scoring"; a feedback generation module provides targeted suggestions; and an optimization module fine-tunes the model and dynamically updates the scoring criteria based on teacher feedback. The method achieves automated scoring through image analysis, text feature analysis, hybrid scoring, and feedback generation, and improves accuracy by combining grade-level adaptation and human-computer collaborative optimization. This solution solves the problems of low efficiency and inconsistent standards in manual grading, assists teachers in precise teaching, provides students with immediate guidance, adapts to the needs of digital education, and has significant practical value in teaching.

The invention discloses a student actual combat homework AI intelligent evaluation method, system and device based on a hybrid expert model and a storage medium. The method comprises the following steps: pre-constructing a structured expected answer containing dimension activation information for a teaching task; when the homework is evaluated, a dimension expert subset needing to be activated is determined according to dimension activation information in the expected answer; calling each activation dimension expert to output a structured evaluation evidence; converting the structured evaluation evidence into a dimension score according to a preset calculation rule; and only carrying out normalization processing on the weight of the activation dimension expert subset, and then carrying out weighted fusion to obtain a final scoring result. According to the invention, through solidifying the evaluation benchmark, dynamically activating the dimension subset, outputting the structured evidence and carrying out weight normalization fusion on the activation set, the problems of different scoring standards, total scale drift and unreviewable evaluation results in the prior art are solved, and automatic, consistent and explainable intelligent evaluation of student actual combat homework is realized.

This specification provides a model training method, a scoring criterion generation method, and an interaction data scoring method. The model training method includes: acquiring real interaction data of the target user in a text generation scenario; using an initial scoring criterion generation model to perform multi-dimensional analysis of the real interaction data to obtain a set of effective scoring criteria adapted to the target user's preferences, wherein any effective scoring criterion in the set corresponds to a scoring dimension of the real interaction data in text generation; constructing a scoring function based on the scoring dimensions of each effective scoring criterion in the set; and training the initial scoring criterion generation model using the scoring function as the training objective to obtain a target scoring criterion generation model. This allows the obtained target scoring criterion generation model to generate effective scoring criteria that highly match the target user's preferences, improving the personalized adaptation capability and scoring accuracy of the scoring criteria generation.

This allows for scoring based on multiple scoring criteria tailored to the learning objectives. [Solution] The scoring device 100 takes information about an English task and information about the answer to the task as input and generates one or more scoring results based on predetermined scoring criteria for the answer to the English task, based on a learning model that outputs scoring results based on predetermined scoring criteria. The scoring device 100 outputs one or more of the generated scoring results based on predetermined scoring criteria to the user.

Aspects of the present disclosure relate to structuring prompt frameworks in machine learning models. Embodiments include instructing a machine learning model via a prompt to generate an output according to a series of steps that reference one or more sections of the prompt. Embodiments include providing the machine learning model, via the prompt, with the one or more sections delineated with corresponding tags, each section of the one or more sections being referenced in the prompt via a corresponding tag. Embodiments include providing the machine learning model, via the prompt, with an output template indicating a target structure for the output and instructing the machine learning model to score the generated output according to a set of scoring criteria. Embodiments include instructing the machine learning model via the prompt to provide the output only when a calculated score, based on the scoring of the generated output, exceeds a threshold value.

The application discloses an education evaluation scene-oriented explainable scoring and directional regeneration method and system based on implication reasoning chain. The method constructs an implication reasoning chain from student answers to scoring standards for each scoring point, and labels the implication relationship type and confidence at each step. The score is calculated through an implication chain strength function, which naturally supports partial scoring. The implication chain breaking point and contradiction point are coded as a structured defect descriptor, which drives the content generation module to perform directional regeneration on the defect interval and provides accurate learning feedback for students. The application is applied to education scenes such as test paper marking and composition scoring. Experimental verification shows that the QWK reaches 0.86, and the directional regeneration saves 78% of the computing resources.

This suppresses the increase in processing load and the decrease in the accuracy of handwritten answer recognition and scoring. [Solution] The conversion unit 32 of the scoring support server 30 recognizes the answer frame, which has a set question type (answer format) and scoring criteria, from the answer sheet image data in which the answer corresponding to the question is expressed, and converts the portion expressed in the recognized answer frame into digital data in an answer format that matches the question type. If the evaluation value of the conversion is low, it refers to the question text, etc., and replaces it with the correct characters, etc. The scoring unit 33 switches between a simple automatic scoring mode, an AI automatic scoring mode requiring natural language processing, and a manual scoring mode depending on the type of question, and processes the information represented by the digital data according to the scoring criteria in the selected mode.

The embodiment of the invention provides an ESG rating system construction and scoring method and device, a processor and a medium. The method comprises the steps of obtaining to-be-scored data and latest standard data of each scoring object based on a customer type and an industry type, determining scoring indexes and hierarchical identifiers of the scoring indexes based on ESG definition information, the to-be-scored data and historical indexes, and determining a scoring standard of a target scoring index based on the latest standard data, constructing a scoring index subsystem based on the hierarchy identifier of each scoring index and the scoring standard of the target scoring index, and determining the target weight of each target scoring index based on the obtained sub-weight of each scoring index in the corresponding hierarchy and the scoring index subsystem, and constructing an ESG rating system of each scoring object based on a preset adjustment index factor, the scoring index subsystem and the target weight of each target scoring index. The accuracy and efficiency of an ESG rating system are improved, and valuable reference information is provided for related parties.

A system and method for guiding and constraining an Artificial Intelligence (AI) engine to provide personalized educational support to a user on an online learning platform using a multi-assistant framework is disclosed. The system and method access curriculum database and grading rubrics. User interaction data including user responses and selected units or topics is received. A free-response questions (FRQs) are generated using algorithms. The user responses to the FRQs are graded by utilizing the grading rubrics and providing projected score using FRQ grader assistant. The FRQ grader provides feedback aligned with scoring guidelines based on grading rubrics, and delivers projected score. A prompt is generated and transferred to AI engine to generate an assessment corresponding to the user performance based on a grading result on the user responses to FRQs. The generated FRQs, graded user responses, assessment, and projected scores are provided to the user on the online learning platform.

The invention relates to the technical field of artificial intelligence, and discloses an optimal answer recommendation method for group answering, a storage medium and a device, and the method comprises the steps: obtaining question stem data, and extracting question stem contents from the question stem data to form a standardized question information structure; acquiring student answering data in real time, and extracting an answering text in the student answering data; connecting the answer texts into an answer history chain according to the submission sequence of the answer data of the students, and dynamically updating the answer history chain; the standardized question information structure, the answer historical chain and the scoring prompt word template are integrated into scoring data; inputting the score data into the first large model to obtain a score output result including a confidence score and score interpretation; and pushing the first several pieces of student answer data with relatively high confidence scores in the latest score output result and the confidence scores and score explanations thereof to terminal equipment of the teacher user. The method can assist teachers in establishing objective scoring standards for group answering, and the teaching efficiency is improved.

The invention relates to a spoken language expression ability AI evaluation method and device, equipment and a medium. The method comprises the following steps: performing multi-granularity feature decoupling processing on an original voice signal to obtain acoustic, linguistic and sub-linguistic feature sets; generating a specific feature representation of each scoring task based on a shared underlying encoder and a task specific expert network; constructing a differential scoring rule graph to quantify a relationship between tasks; identifying a gradient conflict task pair through forward gradient prediction and conflict detection; implementing a dynamic parameter freezing strategy according to the conflict index to generate an optimized gradient direction; model parameters are updated in combination with a multi-target loss function of dynamic weight calibration; and finally, generating a scoring result through multi-task score fusion. By adopting the method, the problems of gradient conflicts, inconsistent scoring standard semantics and calibration failure in multi-task learning can be effectively solved through task relation modeling and dynamic gradient reconciliation, and the evaluation accuracy and stability are improved.

The invention discloses a subjective question automatic scoring consistency calibration method, device and equipment, and the method comprises the steps: defining an individual triple of a scoring strategy, and carrying out the initialization setting of the scoring strategy through a large language model; the samples are evaluated based on uncertainty sampling, and high-uncertainty samples are screened out; introducing an expert feedback loop, obtaining an expert correction score and constructing target probability distribution; and performing reverse optimization on the prompt vector by using KL divergence minimization. Repeatedly evaluating, feeding back and optimizing until a condition of stopping iteration is met; and using the calibrated prompt vector to score the total subjective question. According to the method, the problems of high computing power cost and unstable scoring standard confronted by traditional fine tuning can be solved by utilizing the parameter efficiency of prompting project optimization, the consistency of a scoring system is enhanced, the burden of manual scoring of experts is reduced to the greatest extent, and finally, the credibility of automatic scoring is improved.

Systems, devices, and methods for an automated grading system configured to: retrieve, for each essay submission, a corresponding prompt, rubric, sample answers, and instructions from the database using an essay question ID; transmit the essay submission, a batch of rubric items, and instructions to an evaluation assistant component; receive item-level scores and feedback for each rubric item of the batch of rubric items from the evaluation assistant component; iterate the transmission and reception of each rubric item of the batch of rubric items and feedback until all rubric items are evaluated; transmit the item-level scores and feedback to a summarization assistant configured to compile a structured feedback summary; deliver formatted feedback to the student and to human graders; and implement guardrails to flag abnormal submissions for human review and direct human grader intervention.

Disclosed herein are systems and methods for automated grading of written response tests. The method includes obtaining course material, test materials for a plurality of courses offered by an academic institution, a standardized scoring rubric for a plurality of tests and different courses, and grading guidelines comprising test scoring rules for the written response test or course. The method further includes analyzing the course materials and test materials using a trained rubric customizer LLM. The method also includes obtaining a written response from a learner for a test in a course taken by the learner. The method further includes analyzing the written response using two or more trained test grading LLM agents executing in parallel with each other. The method further includes combining, by a scoring engine, a plurality of criteria scores from the one or more trained test grading LLM agents. The method further includes generating a grading report.

The present disclosure relates to a method for providing a user interface for evaluating descriptive tests based on artificial intelligence. A method for providing a user interface for evaluating descriptive tests based on artificial intelligence according to one embodiment of the present disclosure allows for fairness and objectivity in the evaluation of descriptive tests and improves the efficiency of the test creator by having an artificial intelligence module analyze descriptive test question information to generate evaluation criteria and score the descriptive tests. Furthermore, by providing an interface for creating descriptive test questions, scoring criteria for descriptive test questions, and evaluating examinees' answers, the test creator can systematically manage descriptive test questions and objectively evaluate examinees' answers by establishing clear scoring criteria.

Provide a system. 【Solution means】 Means for receiving user input and analyzing the intention of the question based on the input, Means for referring to past test data and generating problem ideas based on the analysis results, Means for creating a specific problem statement from the generated problem ideas and checking logical consistency, Means for estimating the difficulty level of the problem statement, Means for analyzing the ideal answer and creating scoring criteria, Means for analyzing voice input and converting it into text data, Means for automatically generating problems using a generation AI model, Means for providing feedback to the user based on the evaluation results, A system including the above.