45 results about "Scoring criteria" patented technology

The invention provides an evaluation method, system and device for a large language model and a storage medium, and the method comprises the steps: constructing a sample data set which comprises input data and corresponding reference answers, inputting the input data into an evaluated model, and generating an output result; a multi-dimensional evaluation index system is designed according to task requirements, a dynamic weight is allocated to each evaluation index, each evaluation index is provided with scoring standard description, and the evaluation indexes comprise at least two items of context correlation, term consistency, language fluency and expression accuracy; combining the input data, the reference answer, the output result and the scoring standard description into a standardized input instruction, and calling an evaluation model to perform multi-dimensional scoring on the standardized input instruction to generate an evaluation result; and analyzing the evaluation result according to a preset threshold value, and generating a structured feedback suggestion containing an improvement direction. According to the method, rapid optimization and iteration of the large language model can be effectively supported.

The invention discloses an intelligent agent-based English composition scoring method and device, equipment and a storage medium. Comprising the following steps: inputting composition topics, composition contents and scoring standards into a scoring standard analysis agent; the scoring standard analysis agent selects a target analysis tool from a preset analysis tool group for analysis to obtain an analysis result; the analysis result comprises analysis of different scoring dimensions and corresponding suggested scores; and inputting the composition question, the composition content, the scoring standard and the analysis result into a scoring agent to obtain an English composition scoring result. Through a staged scoring mechanism, an analysis agent extracts key features to form an intermediate analysis result, and then a scoring agent performs secondary check and comprehensive judgment according to a scoring standard, so that the scoring accuracy and consistency are effectively improved. And meanwhile, the scoring standard is independently used as an external input parameter, so that the system can automatically adapt to different scoring systems, and the universality and the adaptability are enhanced.

The invention relates to the technical field of intelligent computing centers, intelligent computing centers and computing power infrastructures, and discloses a method and a device for distilling a reinforcement learning data set by an intelligent computing center for providing computing power, and the method comprises the steps: obtaining an original data set, generating at least one candidate answer corresponding to the original data set based on the original data set and a large model, respectively generating an answer thought corresponding to each candidate answer; based on the answer thought and the large model, generating an examination point corresponding to the answer thought; based on the answer thought, the examination point and the large model, generating a scoring standard corresponding to the examination point; and combining the original data set, the at least one candidate answer, the answer thought, the examination point and the scoring standard to obtain a reinforcement learning data set. Therefore, the reinforcement learning data set has diversity and complexity, and the large model can be trained subsequently according to the reinforcement learning data set, so that the expression and development potential of the large model in complex tasks and real application scenes is effectively enhanced, and the performance of the large model is improved.

The invention relates to the technical field of urban drainage pipe network detection and evaluation, in particular to an intelligent pipeline defect evaluation method based on a multi-modal large model. The evaluation method comprises the following steps: firstly, carrying out defect identification on a pipeline detection video based on a deep learning model to obtain a series of defect images containing defect types and positions, and carrying out statistics and segmented scoring on the positions of the defects and the number and distribution of pipe section defects; then, the obtained images with defects are transmitted to a multi-modal large model for deep analysis, scoring is performed from multiple aspects such as the position and size of the defects, the influence on the pipeline structure and the use stability, and the scoring standard and rule are unified by adopting fixed cue words; and finally, weighting by using the two scoring results to obtain the comprehensive score of each pipe section, and synthesizing the scores of each pipe section into the comprehensive score of the pipeline. The evaluation objectivity and accuracy are remarkably improved, the detection and evaluation efficiency is greatly improved, the maintenance decision is more scientific and accurate, and the pipe network management level and safety are improved.

The invention discloses a multi-agent-based automatic subjective question scoring method and device and a storage medium. The method comprises the following steps: obtaining an answer to be scored, scoring the answer to be scored according to a scoring standard through an overview agent, giving a scoring reason, obtaining a first score and a corresponding scoring reason, and scoring the accuracy, word use and definition of the answer to be scored through a detail review agent, so as to obtain a second score and a corresponding scoring reason. Performing logic verification on the scores and the corresponding scoring reasons through a logic verification intelligent agent, and under the condition that the scores and the corresponding scoring reasons pass the logic verification, judging whether the scores and the corresponding scoring reasons pass the logic verification; according to the technical scheme, debate is carried out through the overview agent and the detail review agent, the debate process between the overview agent and the detail review agent is supervised through the supervision agent, and the final score of the answer to be scored is generated under the condition that the agents are consistent, so that the accuracy of scoring the answer to the subjective question is improved.

The invention relates to the field of artificial intelligence, can be applied to business system platforms of financial science and technology, medical health and the like, and discloses an intelligent scoring method, device and equipment for call recording and a medium, and the method comprises the steps: obtaining to-be-scored recording data; performing voice recognition on the to-be-scored recording data to obtain call text data of the to-be-scored role; obtaining a score configuration file of the current service scene; according to the call text data and the scoring configuration file, generating a prompt word of each scoring item so as to prompt the big language model to score the call text data according to a corresponding scoring standard; and inputting the prompt word of each scoring item into the large language model to obtain a scoring result of each scoring item, and summarizing the scoring results of each scoring item to obtain a corresponding total score. By automatically identifying the call text data of the to-be-scored role and enabling the large language model to perform automatic scoring processing according to the unified scoring standard, the accuracy and reliability of call recording scoring are improved.

The invention relates to the technical field of intelligent scoring, in particular to a teacher teaching skill multi-dimensional intelligent scoring method and system.The method comprises the following steps that based on a teaching behavior multi-dimensional evaluation system, scoring standard connectivity strength is recognized, behavior consistency and language stability are analyzed, a feature behavior set is extracted, adjacent areas are expanded, interference nodes are removed, and the scoring standard connectivity strength is obtained; marking isolated point and breakpoint complementation structures, combining small regions, removing fragments, extracting behavior coding sequences, clustering and distributing labels, identifying index mapping, and outputting a teacher teaching skill classification and partition identification label set. The judgment of behavior consistency and language expression stability on skill features is enhanced, the accuracy and real-time performance of an evaluation result are improved, particularly, the strength and deficiency of teachers are meticulously analyzed and fed back in a complex teaching scene, teacher growth and teaching quality optimization are effectively supported, the high efficiency and unbiasedness of an evaluation framework are improved, and the evaluation efficiency is improved. And valuable education decision support is provided.

This invention relates to the field of artificial intelligence education technology, specifically to a method and system for reviewing and providing feedback on physics, chemistry, and biology lab reports based on a multimodal large-scale model. The method includes: acquiring lab report question data, student answer data, and scoring standard data; vectorizing the question data and retrieving scoring criteria from a scoring knowledge base; selecting either a text-based or image-based scoring process based on the answer type, and calling a fine-tuned multimodal large-scale language model for scoring reasoning; calling an error identification module and a teaching suggestion generation module based on the scoring results; summarizing the complete review results and calculating the joint confidence level; and determining whether to push the results for manual review based on the confidence level. This invention can improve the accuracy, fine-grained understanding, and quality of teaching feedback in lab report review, enabling the self-evolution of the review system.

The application discloses a kind of vertical domain intelligent agent reinforcement learning method based on big language model adjudication mechanism and GRPO, which comprises: first, detailed scoring criteria are formulated according to task scenarios;Using vLLM engine to accelerate inference, multiple rounds of sampling are carried out for each decision of the intelligent agent, a big language model is used as a judge model, and each sample of the intelligent agent is scored according to the detailed scoring criteria defined by the user;After the intelligent agent inference is completed, multiple complete trajectories are obtained, and the influence weight of the decision on the final reply is calculated using the exponential decay method;The scores of all decisions of the entire trajectory are weighted and summed to obtain the total score of the entire trajectory;Then, according to the loss calculation formula of GRPO, the corresponding importance sampling ratio, in-group advantage and other parameters are calculated, and finally the loss value is calculated, and the parameters are optimized by back propagation. The present application combines the exponential decay credit allocation mechanism and the in-group advantage optimization strategy of GRPO to realize efficient, stable and interpretable reinforcement learning of the multi-step decision-making process of the intelligent agent.

The present invention provides a method for marking subjective test papers based on large models, belonging to the field of teacher examination evaluation. First, collect and preprocess subjective question answer data from the self-owned examination database to construct a high-quality data set; secondly, design a detailed scoring standard, manually annotate dimensions such as the accuracy, logic, and language expression of the answer content to generate weighted average score labels; then, select a marking and scoring large model based on the Transformer architecture, and utilize its powerful semantic representation ability to construct a marking model including modules such as multi-head attention; finally, design a two-way semantic matching calculation method to measure the similarity between the answer and the reference answer as the basis for scoring. Through the above steps, the model can achieve efficient, accurate, and objective automatic marking of subjective questions, improving the intelligent level of teacher evaluation.

The invention belongs to the technical field of image analysis, and particularly relates to an exercise exercise effect evaluation method and system and a readable storage medium, the exercise exercise effect evaluation method comprises the following steps: obtaining exercise data to extract a relative position and a demonstration track of a table and a ball; acquiring current motion data to extract the current position of the ball; comparing the current position of the ball with the matched relative position to obtain an error value; simulating the motion trail of the ball according to the error value; comparing the motion track with the demonstration track to output a corresponding motion evaluation result; different scoring standards are established through different demonstration trajectories, whether the physical state of the ball and the physical state of the racket change or not can be judged, the situation that the current movement process is still evaluated due to change of objective conditions is avoided, meanwhile, an accurate and objective evaluation result is made for the movement process of the table tennis, and the evaluation efficiency is improved. And the systematicness and scientificity of evaluation are improved.

This invention discloses a method, device, and medium for safety training in the power industry based on virtual reality, relating to the field of power simulation technology. The method includes: acquiring raw data of operational behavior, including timestamps, hand spatial trajectories, action triggering states, and power equipment numbers; generating action segment data based on the hand spatial trajectories and action triggering states, and constructing action chain data based on the timestamps and power equipment numbers; calculating and generating sequence deviation scores and path offset scores based on the action chain data and standard process data; generating evaluation score records based on the sequence deviation scores, path offset scores, and duration in the action segment data; and generating evaluation results containing step numbers, score values, and action offset markers based on the evaluation score records. This invention achieves automatic identification and deviation quantification evaluation of key action steps in power operation processes without the need for manual expert review or subjective scoring standards.

The application discloses a kind of based on dark mark random matching's method and system for evaluating bid, it is related to intelligent bid evaluation technical field, including: the dark mark file to be evaluated is divided into multiple scoring points and the scoring standard corresponding to each scoring point;First random grouping is carried out to all scoring points, and multiple sets of scoring modules are formed according to the number of experts;Each set of scoring items includes a preset number of scoring points;Each set of scoring items is secondly randomly distributed, so that each expert corresponds to a set of scoring items;The scoring points in each set of scoring items are thirdly randomly distributed, in the case that each scoring point is scored by at least two experts, the number of scoring points is less than the preset number of scoring points included in each set of scoring items, the scoring points after third random distribution are distributed to the experts corresponding to the set of scoring items;The scores of each expert for each scoring point are counted, and a scoring result is generated.The application can ensure that experts can see the real work and reduce bias and subjectivity.

The present application relates to the technical field of data processing, in particular to a multi-source data fusion cross-role collaborative quantification and instruction system and method, comprising: a data acquisition module, which acquires multi-source heterogeneous data of multiple operation roles; an individual characteristic representation module, which constructs an individual characteristic vector; a collaborative behavior knowledge base module, which stores a standardized behavior evaluation scale containing observable behavior items and hierarchical scoring criteria; a collaborative state quantification module, which calculates collaborative quantification characteristics based on the individual characteristic vector and the observable behavior items; a feature node identification module, which identifies global extreme value feature nodes, and sorts the contribution degrees of the global extreme value feature nodes and the collaborative quantification characteristics to the abnormal prediction of a target operation system; and an instruction generation module, which retrieves the observable behavior items based on the contribution degree sorting and generates a structured behavior correction instruction set.The present application realizes the quantitative representation and abnormal attribution of the cross-role collaborative state, and establishes an automatic mapping link from quantitative analysis to executable instructions.

An artificial-intelligence-based decision-assisting system and method are disclosed for generating ranked recommendations through adaptive multi-source analysis. The system includes a processor and a non-transitory computer-readable storage medium storing executable instructions that implement a scoring engine and a ranking engine. Decision parameters and user-defined weighting factors are received from user devices and combined with system-defined weighting factors retrieved from behavioral, historical, external-context, and scoring-criteria databases. Composite decision scores are computed and used to rank candidate options. The system iteratively updates weighting factors or rankings based on feedback data and dynamically restricts data retrieval to relevant parameters to reduce latency and improve throughput. Results are displayed via a graphical user interface, and anonymization procedures protect user identity. The method supports concurrent processing and adaptive learning to refine decision predictions.

This invention provides a method, apparatus, device, and storage medium for autonomous evaluation of large models based on multi-agent technology. It generates an evaluation task set covering multiple scenarios and varying levels of difficulty by setting dimensional weights for basic and extended indicators. Based on this task set, a multi-agent evaluation system is constructed, comprising a questioner, an evaluator, and a reference agent. A parallel interaction mechanism is used to simultaneously initiate tasks to both the model to be evaluated and the reference agent. The evaluator agent compares the outputs of both agents according to multi-dimensional scoring criteria, and constructs a structured dataset from the task instances, output results, and scores. By analyzing this dataset, subsequent task sets are dynamically optimized, achieving a comprehensive and iterative evaluation of the large model's capabilities. This solves the problem that existing evaluation methods cannot fully reflect the true capability level of large models.

Systems, devices, and methods for an automated grading system (111) configured to: retrieve, for each essay submission, a corresponding prompt, rubric, sample answers, and instructions from the database (110) using an essay question ID; transmit the essay submission, a batch of rubric items (112), and instructions to an evaluation assistant component (113); receive item-level scores and feedback (115) for each rubric item of the batch of rubric items from the evaluation assistant component (113); iterate the transmission and reception of each rubric item of the batch of rubric items and feedback until all rubric items are evaluated; transmit the item-level scores and feedback to a summarization assistant (116) configured to compile a structured feedback summary (117); deliver formatted feedback (118) to the student and to human graders (119); and implement guardrails (120) to flag abnormal submissions for human review and direct human grader intervention (121).

The invention provides a work order quality evaluation method and device and a work order quality evaluation system, and the method comprises the steps: 1, obtaining a standard score of each work order according to a scoring standard, and obtaining a work order score of each work order according to the standard score of each work order and an artificial score; training an SVR model by using the spatial feature vector and the work order score of each work order to obtain a work order scoring model; thirdly, inputting the spatial feature vectors into a work order scoring model to obtain model scores of all work orders, and adjusting scoring standards according to the model scores of all the work orders and manual scores; a repeating step: sequentially repeating the first step, the second step and the third step at least once to obtain a plurality of work order scoring models; determining the work order scoring model with the minimum average scoring error as a final work order scoring model; and scoring the new work order by adopting the final work order scoring model. According to the method, the problem of relatively low efficiency caused by the fact that work order scoring depends on manual scoring in the prior art is solved.

A method and system of fine-tuning large language models is disclosed. The method includes receiving a user input corresponding to an LLM. The user input includes a selection of a set of target layers, predefined scoring criteria, and a distribution ratio. The method further includes determining a score corresponding to each of a plurality of weights of each of the set of target layers based on the predefined scoring criteria; for each of the set of target layers, classifying the plurality of weights into a set of trainable weights and a set of non-trainable weights based on the score and the distribution ratio; and for each of the set of target layers, modifying the set of trainable weights using a domain-specific training dataset to obtain a fine-tunned LLM.

This invention discloses a method and system for evaluating large language models based on multi-model cross-evaluation. The method includes the following steps: Step S01. Select the current question-generating model from multiple candidate models, and use prompt word chains to sequentially achieve knowledge extraction, creative reorganization, and self-consistency of the evaluation system to generate a structured task package; Step S02. Send the task package to the remaining models except the question-generating model, and inject a unified answer prompt word into each model. Each model independently analyzes the questions in the task package and provides an answer; Step S03. Dynamically generate review prompt words according to the scoring criteria to guide the models to evaluate the answers of all models, obtain the evaluation results of each model, and return to Step S01 to execute the next round of evaluation until a preset round is reached; Step S04. Combine the scoring data from multiple rounds to obtain the evaluation ranking result. This invention can achieve an objective, fair, professional, and reliable automated evaluation of the performance of large language models.

To provide a system capable of comprehensively analyzing the grammar, punctuation, spelling, and consistency, clarity, and style of a sentence of an essay and reducing the scoring burden of a teacher.SOLUTION: The system includes a grammar analyzer, a punctuation analyzer, a spelling analyzer, a sentence consistency analyzer, a clarity analyzer, a style analyzer, a summary generator, and a scoring criteria setter. Using the generated AI, the grammar analyzer comprehensively analyzes the grammar of the essay, the punctuation analyzer comprehensively analyzes the punctuation of the essay, the spelling analyzer comprehensively analyzes the spelling of the essay, the sentence coherence analyzer comprehensively analyzes the sentence coherence of the essay, the clarity analyzer comprehensively analyzes the clarity of the essay, and the style analyzer comprehensively analyzes the style of the essay. The summary generation unit generates a summary from which the core content of the essay can be quickly grasped. The scoring standard setting unit sets a customizable scoring standard.SELECTED DRAWING: Figure 1

The application discloses a method for enhancing teaching evaluation credibility in an education large model, and the method comprises the following steps: obtaining a scoring prompt, obtaining first scoring evaluation information of answer content according to the scoring prompt; obtaining an interactive object corresponding to the first scoring evaluation information, and constructing the scores of different users into a constructed historical score sequence under the same scoring standard; obtaining a dynamic smoothing coefficient according to the historical score sequence; obtaining the feedback quality of the first scoring evaluation information according to the historical score sequence and the dynamic smoothing coefficient; in the case that the feedback quality of the first scoring evaluation information is high, determining that the first scoring evaluation information meets the standard, and sending the first scoring evaluation information to a first terminal. Through the change of the dynamic proportion of new and old information, the feedback quality of the first scoring evaluation information is quantitatively evaluated. This makes the fluctuation of the scoring results of different batches and different times controllable, and forms a scoring basis with strong interpretability.

This invention discloses an automatic scoring system and optimization method for picture-based writing in primary school Chinese language teaching, belonging to the field of intelligent education technology. The system includes an image understanding module, which analyzes the core elements, themes, and logic of images through a lightweight object detection and scene classification model; a text analysis module extracts text features from the dimensions of content relevance, language standardization, logical completeness, and imagination; a scoring model module outputs scores using a hybrid strategy of "rule-based scoring + model scoring"; a feedback generation module provides targeted suggestions; and an optimization module fine-tunes the model and dynamically updates the scoring criteria based on teacher feedback. The method achieves automated scoring through image analysis, text feature analysis, hybrid scoring, and feedback generation, and improves accuracy by combining grade-level adaptation and human-computer collaborative optimization. This solution solves the problems of low efficiency and inconsistent standards in manual grading, assists teachers in precise teaching, provides students with immediate guidance, adapts to the needs of digital education, and has significant practical value in teaching.

The invention discloses a student actual combat homework AI intelligent evaluation method, system and device based on a hybrid expert model and a storage medium. The method comprises the following steps: pre-constructing a structured expected answer containing dimension activation information for a teaching task; when the homework is evaluated, a dimension expert subset needing to be activated is determined according to dimension activation information in the expected answer; calling each activation dimension expert to output a structured evaluation evidence; converting the structured evaluation evidence into a dimension score according to a preset calculation rule; and only carrying out normalization processing on the weight of the activation dimension expert subset, and then carrying out weighted fusion to obtain a final scoring result. According to the invention, through solidifying the evaluation benchmark, dynamically activating the dimension subset, outputting the structured evidence and carrying out weight normalization fusion on the activation set, the problems of different scoring standards, total scale drift and unreviewable evaluation results in the prior art are solved, and automatic, consistent and explainable intelligent evaluation of student actual combat homework is realized.

The invention discloses a multi-index classification weight scoring method and a related device, and belongs to the technical field of equipment performance evaluation. The method comprises the following steps: constructing an index evaluation system; the index evaluation system comprises index classification, indexes and index basic information; index classification is selected in the standard evaluation system according to the to-be-evaluated equipment; determining the weight of each index classification, and setting a plurality of indexes to which each index classification belongs and an index scoring standard; acquiring historical data of the to-be-evaluated equipment, and calculating each index score based on the index scoring standard; and according to the weight of each index classification and the index score, finally calculating to obtain a scoring result of the to-be-evaluated equipment. According to the method, the obtained scoring result is objective and reliable, whether longitudinal comparison of the same equipment in different periods or transverse comparison between different equipment can be carried out based on a unified scoring standard, and the fairness and objectivity of evaluation are improved.

The invention relates to the technical field of artificial intelligence, and discloses a chain teaching scoring system based on multi-modal self-adaptive generation of an education large model, and the system comprises a score marking module and a self-adaptive module. The score reviewing module is used for generating first score evaluation information of the answer content according to the score prompt and transmitting the first score evaluation information to the adaptive module; the self-adaption module is used for determining an interaction object and a scoring standard corresponding to the first score evaluation information, constructing scores of different users into a historical score sequence under the same scoring standard, evaluating the feedback quality of the first score evaluation information according to the historical score sequence, and outputting the first score evaluation information when the feedback quality of the first score evaluation information is high. And outputting the score. The association relationship and the change rule among multiple historical scores are converted into standardized numerical expression, the feedback quality of the output scores is kept within a high-quality range based on the characteristics of numerical expression, and the score result fluctuation is small.

The invention discloses a method for enhancing teaching evaluation credibility in an education large model, and the method comprises the steps: obtaining a scoring prompt, and obtaining first scoring evaluation information of answer content according to the scoring prompt; obtaining an interaction object corresponding to the first score evaluation information, and constructing scores of different users into a historical score sequence under the same score standard; obtaining a dynamic smoothing coefficient according to the historical score sequence; obtaining the feedback quality of the first score evaluation information according to the historical score sequence and the dynamic smoothing coefficient; and under the condition that the feedback quality of the first score evaluation information is high, determining that the first score evaluation information reaches the standard, and sending the first score evaluation information to the first terminal. And quantitatively evaluating the feedback quality of the first score evaluation information through the change of the dynamic proportion of the new and old information. Therefore, the fluctuation of scoring results of different batches and different times is controllable, and a scoring basis with high interpretability is formed.

This specification provides a model training method, a scoring criterion generation method, and an interaction data scoring method. The model training method includes: acquiring real interaction data of the target user in a text generation scenario; using an initial scoring criterion generation model to perform multi-dimensional analysis of the real interaction data to obtain a set of effective scoring criteria adapted to the target user's preferences, wherein any effective scoring criterion in the set corresponds to a scoring dimension of the real interaction data in text generation; constructing a scoring function based on the scoring dimensions of each effective scoring criterion in the set; and training the initial scoring criterion generation model using the scoring function as the training objective to obtain a target scoring criterion generation model. This allows the obtained target scoring criterion generation model to generate effective scoring criteria that highly match the target user's preferences, improving the personalized adaptation capability and scoring accuracy of the scoring criteria generation.

This allows for scoring based on multiple scoring criteria tailored to the learning objectives. [Solution] The scoring device 100 takes information about an English task and information about the answer to the task as input and generates one or more scoring results based on predetermined scoring criteria for the answer to the English task, based on a learning model that outputs scoring results based on predetermined scoring criteria. The scoring device 100 outputs one or more of the generated scoring results based on predetermined scoring criteria to the user.

The invention relates to the field of power data management, and discloses a dynamic multi-dimensional power data grade evaluation method and related equipment, and the method comprises the steps: constructing a power data multi-dimensional evaluation index system, and obtaining a plurality of pieces of core dimension data; according to the determined index weight of each piece of core dimension data, a quantitative scoring standard is formulated, and the index weight is dynamically adjusted according to the data life cycle and the service priority in combination with the scoring standard; performing preliminary scoring on the core dimension data by using a mixed deep learning model to obtain a preliminary scoring result; performing dynamic optimization on the preliminary scoring result, adjusting parameters of the hybrid deep learning model through a real-time feedback mechanism, and generating a final comprehensive score; scoring grades are divided, and grade evaluation of the dynamic multi-dimensional power data is carried out according to the final comprehensive scores and the divided scoring grades. According to the method, the power data multi-dimensional evaluation index system is constructed, the power data are comprehensively considered from multiple key angles, and the comprehensiveness and accuracy of power data evaluation are ensured.