Multi-competency intelligent scoring method and system

Abstract

The invention relates to a multi-competency intelligent scoring method and system, and belongs to the technical field of intelligent evaluation and talent evaluation. The method comprises the following steps: acquiring multi-source response data of a target evaluation object, executing evidence sufficiency and consistency analysis for each competency dimension, generating a judgment state identifier, and constructing a structural description; performing responsibility distribution on the evaluation evidence, mapping the evaluation evidence into a corresponding evidence model and generating a priority constraint relationship; performing time sequence consistency and stability analysis on the evidence model, and constructing a time sequence constraint rule to prohibit unconstrained score backtracking; and carrying out resolution processing on the continuous undetermined dimension, generating a scoring result through a preset risk scoring rule and a conservative determination strategy, and synchronously generating a scoring basis path and confidence information. According to the method, standard management and control of multi-source evidences and accurate constraint of the whole scoring process are realized, the scoring accuracy and traceability are effectively improved, and reliable support is provided for multi-scene talent evaluation decision.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of intelligent evaluation and talent evaluation, and particularly relates to a multi-competency intelligent scoring method and system. BACKGROUND

[0002] With the surge in demand for digital evaluation, intelligent scoring technology gradually replaces the traditional manual evaluation mode, but its core value is always focused on efficiency improvement and subjective speculation avoidance, and it cannot solve the underlying problems of multi-source evidence fragmentation, dynamic evaluation process time sequence and difficult disposal of undeterminable state in multi-competency evaluation, resulting in frequent problems of scoring deviation and decision support failure in actual application of existing solutions.

[0003] Specifically, the core shortcoming of the existing multi-competency intelligent scoring technology is reflected in the lack of key links in the whole evaluation process: in the face of multi-modal response data such as text, voice, and expression, the existing technology only stays at the level of simple keyword matching or data statistics, and does not establish evidence sufficiency verification and consistency checking logic for each competency dimension, resulting in the problems of "insufficient evidence but forced scoring" and "contradictory evidence but ambiguous judgment" being widespread; when the competency is in an undeterminable state, there is no standardized state description system to accurately anchor the core cause and influence range of the undeterminable state, making it impossible to provide evidence for subsequent evidence supplement or scoring adjustment; the role of different data modalities in each competency dimension evaluation is unclear, there is no clear responsibility allocation mechanism, and there is no scientific evidence priority constraint, often resulting in logical misplacement of secondary evidence dominating the scoring results; the time sequence attribute of the evaluation process is ignored, the consistency and stability of the response data in each round are not tracked and analyzed, and corresponding scoring backtracking constraints are not constructed, which may cause unreasonable modification of the ability judgment results formed in the early stage due to later data interference, further exacerbating scoring distortion; for the competency dimension that is continuously undeterminable, there is no systematic resolution path, and the risk scoring and conservative judgment strategy are mostly empirically set, so the reliability of the scoring results cannot be guaranteed, and there is no complete scoring evidence traceability chain, making it difficult to meet the professional needs of verifiable and reproducible evaluation results.

[0004] It can be seen that the existing technology has not achieved a fundamental breakthrough from efficiency-based scoring to accurate evaluation, and its shortcomings in evidence processing standardization, undeterminable state disposal, and time sequence risk prevention and control are in sharp contradiction with the core needs of multi-competency evaluation in accuracy, rigor, and traceability in multiple scenarios. SUMMARY

[0005] To solve the above problems in the prior art, the present application provides a multi-competency intelligent scoring method and system, and the purpose of the present application can be achieved by the following technical solutions: Comprising: S1: Obtain multi-source response data for a target evaluation object, perform evidence sufficiency and evidence consistency analysis on each competency dimension through a preset competency evaluation target, generate a corresponding competency judgment state identifier, and the judgment state identifier includes a determinable state and an indeterminable state; when at least one competency dimension is detected in an indeterminable state, an indeterminable state description structure is constructed; S2: Based on the indeterminable state description structure, perform evidence responsibility allocation processing on evaluation evidence from different data modalities; according to the judgment contribution type of each data modality in different competency dimensions, map the evaluation evidence to an evidence model that bears the judgment responsibility, and generate a corresponding evidence priority constraint relationship based on the responsibility type of the evidence model; S3: Perform the timing consistency and timing stability analysis on the evidence model; based on the response round order and the ability performance change trend in the response interaction process, by constructing a timing constraint rule, prohibit the unconstrained score backtracking adjustment on the competency dimension that has appeared the ability collapse or the change beyond the preset range; S4: When the competency dimension is continuously in an indeterminable state, perform indeterminable state resolution processing on each competency dimension according to the evidence priority constraint relationship and the timing constraint rule; generate a score result for the corresponding competency dimension through a preset risk score rule and a conservative judgment strategy, and synchronously generate a score basis path and a score confidence information associated with the score result.

[0006] Specifically, the specific process of performing evidence sufficiency and evidence consistency analysis on each competency dimension is: Determine the evidence sufficiency, preconfigure a list of core evaluation elements for each competency dimension, and count the number of evidence items matching the core evaluation elements for a single competency dimension, set the evidence quantity threshold and the element coverage threshold; Determine the evidence consistency, construct a two-dimensional verification logic, and text evidence is verified by a semantic comparison algorithm for uniformity of the same element expression; non-text evidence is verified by a feature association algorithm for logical fit with text expression.

[0007] Specifically, the specific process of constructing the indeterminable state description structure is: presetting an indeterminable state description core element; building a description framework using a tree data structure, filling in layers according to competency dimensions, indeterminable types, collected evidence, and missing contradictory information, generating independent description nodes for each indeterminable dimension, and establishing node association indexes through dimension coding.

[0008] Specifically, the specific process of the execution of the evidence responsibility allocation processing is: a data modality and competency dimension contribution table is established in advance, and different data modalities are preset to contribute to each competency dimension attribute; the dimension information and the indeterminable reason of the indeterminable state description structure are combined to match the corresponding data modality; the matched modality data is confirmed for the determination responsibility, and the missing element evidence is supplemented or the authenticity of the contradictory evidence is verified.

[0009] Specifically, the specific process of the mapping of the evidence model assuming the determination responsibility is: The determination responsibility type includes: core support type evidence, auxiliary support type evidence; The evidence model includes a data input layer, a semantic reinforcement layer, and a responsibility adaptation layer. The input layer inputs the corresponding data modality evaluation evidence and performs standardization processing; the semantic reinforcement layer refines core features related to the determination responsibility through a feature extraction algorithm; the responsibility adaptation layer matches and calibrates the core features with the preset responsibility determination standard to output the support strength parameter of the evidence on the responsibility.

[0010] Specifically, the specific process of generating the corresponding evidence priority constraint relationship is: Set evidence priority determination indicators, including the close degree of association with competency dimensions, evidence credibility, and evidence completeness; Preset fixed weights of each indicator, and calculate the evidence model priority score by weighted summation; Sort the priority sequence according to the score, and develop constraint rules; the high-priority evidence result is used as the determination basis first, and the low-priority evidence is only used when the high-priority evidence is insufficient, and the result needs to be cross-validated with the high-priority evidence to take effect.

[0011] Specifically, the specific process of executing the time sequence consistency and time sequence stability analysis is: Arrange the evidence model output results according to the response round timestamp to construct a time sequence data sequence; The time sequence consistency analysis compares the logical association of adjacent round evidence results to verify the consistency of the same evaluation element expression and conclusion; The time sequence stability analysis calculates the fluctuation difference value of consecutive round evidence results to verify the fluctuation difference value.

[0012] Specifically, the specific process of constructing the time sequence constraint rule is: preset the core elements of the time sequence constraint rule, including the ability collapse determination condition, the ability change threshold, and the score backtracking limit range; develop the ability collapse determination condition, delimit the ability change threshold, and clearly define the score backtracking limit, only based on the current and subsequent round evidence score, prohibit modification of the previously generated score result; integrate each element to form a complete time sequence constraint rule.

[0013] Specifically, the specific process of performing the indeterminacy state resolution processing on each competency dimension is: based on the evidence priority constraint relationship, preferentially calling a high-scored priority evidence model to supplement missing evidence or verify contradictory evidence; checking the sufficiency and consistency of the supplemented evidence, updating the corresponding competency dimension determination state identifier; if the preset condition is not met, combining the time sequence constraint rule to filter time sequence stable and effective evidence, using a phased resolution strategy to solve the core evaluation element problem first and then process the secondary elements; if multiple rounds of processing still do not meet the determination requirements, determining that it is not resolvable.

[0014] Specifically, the specific process of the preset risk score rule and the conservative determination strategy is: The preset risk score rule divides the risk score interval for each competency dimension, and matches the corresponding interval with the value according to the severity of the indeterminacy state; The preset conservative determination strategy adheres to the no-evidence-no-guessing scoring principle, and only scores based on effective evidence within a limited range. The evidence that is contradictory and cannot be verified is valued according to the conservative interpretation, and the scoring result correction condition is set.

[0015] Specifically, the specific process of generating the scoring result of the corresponding competency dimension is: filtering effective evidence according to the evidence priority and time sequence constraint rule; for the determinable dimension, generating the basic score according to the evidence model output and the scoring mapping standard; for the indeterminable dimension, generating the risk score according to the risk score rule and the conservative strategy; synchronously associating the scoring basis with the confidence information to form the complete scoring result of each dimension.

[0016] Specifically, a multi-competency intelligent scoring system comprises: A competency evidence verification and determination state module: obtaining multi-source response data for a target evaluation object, performing evidence sufficiency and evidence consistency analysis on each competency dimension through a preset competency evaluation target, and generating a corresponding competency determination state identifier, wherein the determination state identifier comprises a determinable state and an indeterminable state; when at least one competency dimension is detected to be in an indeterminable state, an indeterminable state description structure is constructed; An evidence accountability and priority constraint module: based on the indeterminable state description structure, performing evidence responsibility allocation processing on evaluation evidence from different data modalities; according to the determination contribution type of each data modality in different competency dimensions, mapping the evaluation evidence into an evidence model that bears the determination responsibility, and generating a corresponding evidence priority constraint relationship based on the responsibility type of the evidence model; Evidence timing verification and timing rule module: Perform timing consistency and timing stability analysis on the evidence model; Based on the response round order and the trend of capability performance changes during the response interaction process, construct timing constraint rules to prohibit unconstrained score backtracking adjustment for competency dimensions that have experienced capability collapse or changes exceeding the preset range; Undecidable State Resolution and Comprehensive Scoring Module: When the competency dimension is continuously in an undecidable state, the module performs undecidable state resolution processing on each competency dimension according to the evidence priority constraint relationship and time sequence constraint rules; and generates the corresponding competency dimension's scoring result through preset risk scoring rules and conservative judgment strategy, and simultaneously generates the scoring basis path and scoring confidence information associated with the scoring result.

[0017] The beneficial effects of this invention are as follows: (1) By setting up a dual-effect verification mechanism for competency dimension evidence and a structured description mechanism for undeterminable states, on the one hand, a list of core assessment elements is pre-configured for each competency dimension. The sufficiency is determined by statistically analyzing the number of evidence items and the coverage ratio of verification elements. At the same time, a dual-dimensional verification logic is constructed. The consistency verification of text and non-text evidence is completed by using semantic comparison and feature association algorithms, filtering out invalid and contradictory evidence from the source. On the other hand, a tree-shaped data structure is used to build an undeterminable state description framework. The content is filled in hierarchically according to competency dimension, undeterminable type, collected evidence summary, and missing / contradictory information. Node association index is established through dimension coding to accurately anchor the core causes and scope of influence of undeterminable states. This setting can completely avoid the drawbacks of "forced scoring due to insufficient evidence" and "fuzzy judgment of contradictory evidence" in traditional scoring. It provides clear and standardized guidance for subsequent evidence responsibility allocation, supplementary evidence verification, and other links, improving the reliability of the scoring base data and the standardization of the scoring process. (2) By setting up an integrated mechanism for evidence responsibility allocation, time constraints, resolution of undeterminable states, and risk scoring, precise control and synergistic efficiency are achieved across multiple stages: First, by matching appropriate evidence through a pre-set data modality-competency dimension contribution comparison table, the core / auxiliary support responsibilities of each modality are clarified. Then, priority scores are calculated based on indicators such as correlation and credibility, and constraint rules are formulated to ensure that high-value evidence takes the lead in scoring and to avoid secondary evidence interfering with the scoring logic. Second, an evidence model time sequence is constructed according to the response round timestamp. Consistency analysis is completed by comparing the logical correlation between adjacent rounds, and stability verification is completed by calculating the fluctuation difference between rounds. At the same time, the judgment conditions for capability collapse, the threshold for capability change, and the scoring return are clarified. First, it restricts the modification of previous scores for dimensions where significant changes in ability have occurred, effectively avoiding scoring distortion caused by temporal interference. Second, it calls upon models to supplement evidence or verify contradictions according to evidence priority, adopting a phased resolution strategy of "core elements first" to promote state transformation. For dimensions that cannot be resolved, it assigns values ​​according to standardized risk scoring ranges, adhering to the conservative judgment principle of "no speculation without evidence." At the same time, it generates a structured comprehensive report containing the scoring basis path and confidence level and retains process data. This setting ensures that the scoring results are highly consistent with the actual ability performance of the evaluated object, and that the scoring process is traceable and the results are verifiable. It provides data support for talent evaluation decisions in multiple scenarios such as interview selection and training assessment, improving the quality and efficiency of evaluation work. Attached Figure Description

[0018] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings.

[0019] Fig. 1 This is a flowchart illustrating a multi-competency intelligent scoring method according to the present invention. Fig. 2 This is a data flow diagram of a multi-competency intelligent scoring method according to the present invention. Detailed Implementation

[0020] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided.

[0021] Please see Figs. 1-2 A multi-competency intelligent scoring method; include: S1: Acquire multi-source response data for the target assessment object, and perform evidence sufficiency and evidence consistency analysis on each competency dimension according to the preset competency assessment target to generate corresponding competency judgment status identifiers. The judgment status identifiers include decidable and undecidable states. When at least one competency dimension is detected to be in an undecidable state, construct an undecidable state description structure. S2: Based on the undecidable state description structure, perform evidence responsibility allocation processing on the evaluation evidence from different data modalities; according to the judgment contribution type of each data modal in different competency dimensions, map the evaluation evidence to an evidence model that assumes judgment responsibility, and generate corresponding evidence priority constraint relationships based on the responsibility type of the evidence model; S3: Perform the temporal consistency and temporal stability analysis on the evidence model; based on the response round order and the trend of capability performance changes during the response interaction process, construct temporal constraint rules to prohibit unconstrained score backtracking adjustments for competency dimensions that have experienced capability collapse or changes exceeding the preset range; S4: When the competency dimension remains in an undecidable state, perform undecidable state resolution processing on each competency dimension according to the evidence priority constraint relationship and time sequence constraint rules; generate the corresponding competency dimension's score result through preset risk scoring rules and conservative judgment strategy, and simultaneously generate the scoring basis path and scoring confidence information associated with the score result.

[0022] Specifically, the process of performing sufficiency and consistency analysis of evidence for each competency dimension is as follows: To determine the sufficiency of evidence, a list of core assessment elements is pre-configured for each competency dimension. For a single competency dimension, the number of evidence items matching the core assessment elements from multi-source response data is counted, and thresholds for the number of evidence items and the coverage of elements are set. To determine the consistency of evidence, a two-dimensional verification logic is constructed. For textual evidence, the consistency of the expression of the same element is verified through semantic comparison algorithm; for non-textual evidence, the logical fit with the textual expression is verified through feature association algorithm.

[0023] Specifically, the process of constructing the undeterminable state description structure is as follows: preset the core elements of the undeterminable state description; build a description framework using a tree data structure, fill it in layers according to competency dimension, undeterminable type, collected evidence and missing contradictory information, generate an independent description node for each undeterminable dimension, and establish a node association index through dimension encoding.

[0024] In this embodiment, the core elements of the description of the undeterminable state include the name of the undeterminable competency dimension, the undeterminable type, the summary of the collected related evidence, the type of missing evidence, or contradictory evidence.

[0025] In this embodiment, the multi-source response data refers to response-related data collected from the target assessment object (such as interviewees or trainees) and covering multiple data formats, specifically including: core response data: voice response data (voice statements during the interview / assessment process) and text response data (written answers); auxiliary interaction data: facial expression image data during the interview process, voice tone feature data (such as speech rate and tone fluctuations), and response interval duration data (time difference between adjacent answers); processed data: a structured dataset that has been standardized in format, noise removed, and classified and archived according to response timestamps.

[0026] In this embodiment, the determinate and undeterminate states are based on the sufficiency and consistency analysis results of multi-source assessment evidence, and are a clear definition of whether each competency dimension has objective scoring basis. The specific definitions and understandings are as follows: A definite state refers to the assessment evidence for a certain competency dimension simultaneously meeting the "sufficiency requirement" and the "consistency requirement": In terms of sufficiency, the number of evidence items matching the preset core assessment elements under this dimension reaches the preset threshold, and the coverage ratio of the core assessment elements meets the preset standard, which can fully support the assessment of the competency level of this dimension; in terms of consistency, the expression of the same assessment element in textual evidence (such as response text, analysis report) is consistent and unambiguous, and the logical consistency between non-textual evidence (such as tone of voice, facial expression) and textual evidence meets the standard, with no conflicting judgment basis. At this time, this dimension has complete and reliable scoring support, and objective scores can be directly generated based on the output results of the evidence model.

[0027] An undecidable state refers to a competency dimension that does not simultaneously meet the above sufficiency or consistency requirements. Specifically, it includes two situations: First, insufficient evidence, that is, the number of evidence items matching the core assessment elements does not reach the preset threshold, or the coverage ratio of the core assessment elements is insufficient, resulting in an inability to fully reflect the competency level of that dimension; Second, inconsistent evidence, that is, contradictions in the internal expression of textual evidence, or logical inconsistencies between non-textual evidence and textual evidence, forming conflicting judgment criteria. In this case, the dimension lacks objective and effective scoring support and cannot be directly scored. It is necessary to attempt to transform it into a decidable state through subsequent evidence responsibility allocation, supplementary collection, contradiction verification, and other resolution processes. If the resolution is ineffective, it will be handled according to the preset risk scoring rules.

[0028] In this embodiment, the responsibility type refers to the contribution attribute of different data modalities to the judgment of each competency dimension. The specific responsibility attributes assigned to the evaluation evidence include: divided by the degree of contribution: core supporting responsibility (corresponding to core supporting evidence, such as the judgment of the "logical thinking" dimension by text response data) and auxiliary supporting responsibility (corresponding to auxiliary supporting evidence, such as the judgment of the "communication suitability" dimension by facial expression data); divided by specific function: responsibility to supplement missing element evidence (for the indeterminate state of insufficient evidence) and responsibility to verify the authenticity of contradictory evidence (for the indeterminate state of inconsistent evidence).

[0029] In this embodiment, the core features related to determining responsibility refer to the key features extracted from the evaluation evidence that are directly related to the assigned responsibility. These features are used to support the evidence model in completing responsibility matching and generating scoring criteria. Specifically, they include: textual evidence features: the expression of core evaluation elements corresponding to the responsibility, semantic logical integrity, and consistency of expression of the same element (such as the feasibility description features of the solution under the "problem solving" dimension); non-textual evidence features: facial expression features that are consistent with the textual expression logic (such as the facial expression fluctuation features corresponding to emotional stability) and voice tone features (such as the fluency features of speech under the communication expression dimension); and temporal related features: the stability features of responsibility-related performance in consecutive rounds of response (such as the response interval fluctuation features corresponding to mental agility).

[0030] Specifically, the specific process of the execution of evidence responsibility allocation is as follows: a comparison table of the contribution of the data modality and the competency dimension is established in advance, and the contribution attributes of different data modalities to each competency dimension are preset; the corresponding data modality is matched by combining the dimensional information of the undeterminable state description structure and the undeterminable cause; the responsibility for judgment is confirmed for the matched modality data, and the missing element evidence or the authenticity of contradictory evidence is supplemented.

[0031] Specifically, the process of mapping the evidence model for determining liability is as follows: The types of evidence used to determine liability include: core supporting evidence and auxiliary supporting evidence; The evidence model includes a data input layer, a semantic enhancement layer, and a responsibility adaptation layer. The input layer inputs corresponding data modality evaluation evidence and performs standardization processing; the semantic enhancement layer extracts and determines the core features related to responsibility through feature extraction algorithms; the responsibility adaptation layer matches and calibrates the core features with preset responsibility determination standards and outputs the support strength parameters of the evidence for responsibility.

[0032] Specifically, the process of generating the corresponding evidence priority constraint relationship is as follows: Establish criteria for prioritizing evidence, including its relevance to competency dimensions, credibility, and completeness. Each indicator is pre-defined with a fixed weight, and the priority score of the evidence model is calculated by weighted summation. Priority sequences are formed by ranking the scores from highest to lowest, and constraint rules are established. Priority evidence with higher scores is used as the primary basis for judgment, while priority evidence with lower scores is only used when there is insufficient priority evidence with higher scores. The results must be cross-validated with priority evidence with higher scores to be effective.

[0033] Specifically, the process of performing the timing consistency and timing stability analysis is as follows: Arrange the output results of each evidence model according to the response round timestamp to construct a time-series data sequence; The temporal consistency analysis compares the logical correlation of evidence results from adjacent rounds and verifies the consistency between the statements and conclusions of the same evaluation element. The time-series stability analysis calculates the fluctuation difference of evidence results in consecutive rounds and verifies the fluctuation difference.

[0034] Specifically, the process of constructing the time-series constraint rules is as follows: preset the core elements of the time-series constraint rules, including capability collapse judgment conditions, capability change thresholds, and scoring backtracking restriction ranges; formulate capability collapse judgment conditions, define capability change thresholds, clarify scoring backtracking restrictions, and only score based on evidence from the current and subsequent rounds, prohibiting modification of previously generated scoring results; integrate the various elements to form complete time-series constraint rules.

[0035] Specifically, the process of performing undecidable state resolution for each competency dimension is as follows: based on the evidence priority constraint relationship, priority evidence models with higher scores are prioritized to supplement missing evidence or verify contradictory evidence; the sufficiency and consistency of the supplemented evidence are verified, and the corresponding competency dimension judgment status identifier is updated; if the preset conditions are not met, time-stable and valid evidence is screened in combination with time-series constraint rules, and a phased resolution strategy is adopted to first solve the core assessment element problem and then process the secondary elements; if the judgment requirements are still not met after multiple rounds of processing, it is determined to be unresolvable.

[0036] Specifically, the process of the preset risk scoring rules and conservative judgment strategy is as follows: Pre-defined risk scoring rules are used to divide risk scoring ranges for each competency dimension, and values ​​are assigned to the corresponding ranges according to the severity of undetermined states. A conservative judgment strategy is preset, adhering to the principle of not speculating without evidence, scoring is based only on valid evidence with limited scope, and contradictory and unverifiable evidence is assigned a conservative interpretation, with conditions for correcting the scoring results set.

[0037] Specifically, the process of generating the corresponding competency dimension score results is as follows: valid evidence is screened according to evidence priority and time sequence constraint rules; for decidable dimensions, a basic score is generated based on the evidence model output and score mapping standard; for undecidable dimensions, a risk score is generated according to risk scoring rules and conservative strategies; and the score basis path and confidence information are synchronously associated to form a complete score result for each dimension.

[0038] In this embodiment, the actual operation of performing evidence sufficiency and consistency analysis for each competency dimension is as follows: First, based on the interview job type (such as technical R&D position), a list of core assessment elements for each competency dimension (such as logical thinking and technical problem-solving ability) is preset. For example, core elements such as "problem decomposition logic, solution organization, and causal relationship expression" are configured for the "logical thinking" dimension. Then, multi-source data such as candidates' voice responses, text answers, and facial expression videos are obtained through a data acquisition terminal. Data preprocessing tools are used to count the number of evidence items matching the core elements under each competency dimension. The preset evidence quantity threshold is 'a' items and the element coverage threshold is 'b'%. If a dimension has ≥a matching items and the coverage ratio is ≥b%, then the evidence is deemed sufficient. The consistency analysis adopts a two-dimensional verification logic. Textual evidence is verified for the consistency of the same element expression through the BERT semantic comparison algorithm. Non-textual evidence (such as facial expressions) is verified for the emotional features extracted through the CNN feature association algorithm to verify its logical fit with the text expression. If no semantic conflict or feature mismatch is detected, then the evidence is deemed consistent.

[0039] In this embodiment, the actual operation of mapping evaluation evidence to an evidence model that assumes judgment responsibility is as follows: First, based on the preset data modality-competency dimension contribution comparison table, the responsibility type of each data modality is determined. For example, text response data is the core supporting responsibility for the "logical thinking" dimension, and voice tone data is the auxiliary supporting responsibility. Core supporting evidence is matched with deep learning models (such as the Transformer model), and auxiliary supporting evidence is matched with traditional machine learning models (such as the SVM model). The model construction adopts a three-layer architecture. The input layer uses Python to standardize data, translating voice data into text and quantifying facial expression data into feature vectors. The semantic enhancement layer uses the TF-IDF algorithm to extract the core evaluation features of text evidence and the HOG algorithm to extract the key features of non-text evidence. The responsibility adaptation layer performs cosine similarity matching and calibration between the extracted features and the preset responsibility judgment criteria (such as the clarity judgment threshold of the "logical thinking" dimension), and finally outputs the support strength parameters of the cd interval, completing the mapping from evidence to evidence model.

[0040] In this embodiment, the actual operation of performing temporal consistency and temporal stability analysis is as follows: First, the output results of each evidence model (such as the logical thinking support strength parameter corresponding to each round of response) are organized into a temporal data sequence according to the response round order using timestamp synchronization technology; Temporal consistency analysis adopts the sliding window method to compare the semantic expression and conclusion of the same evaluation element (such as "problem decomposition logic") in the evidence results of two adjacent rounds, and determines the logical correlation through the LCS longest common subsequence algorithm. If the correlation score is ≥e, the temporal consistency is determined; Temporal stability analysis calculates the fluctuation difference of the evidence results of consecutive g rounds, with a preset stability threshold of f. If the fluctuation difference is ≤f, the temporal stability is determined; If a fluctuation difference >f or a correlation score <e is detected, it is marked as a temporal anomaly, and this dimension is included in the "prohibition of unconstrained scoring backtracking" scope when constructing temporal constraint rules in the future.

[0041] Specifically, a multi-competency intelligent scoring system includes: Competency Evidence Verification and Judgment Status Module: Acquires multi-source response data for the target assessment object, performs evidence sufficiency and evidence consistency analysis on each competency dimension according to the preset competency assessment target, and generates corresponding competency judgment status identifiers. The judgment status identifiers include decidable status and undecidable status. When at least one competency dimension is detected to be in an undecidable status, an undecidable status description structure is constructed. Evidence Determination and Priority Constraint Module: Based on the undecidable state description structure, it performs evidence responsibility allocation processing on the evaluation evidence from different data modalities; according to the judgment contribution type of each data modality in different competency dimensions, it maps the evaluation evidence to an evidence model that assumes judgment responsibility, and generates corresponding evidence priority constraint relationships based on the responsibility type of the evidence model; Evidence timing verification and timing rule module: Performs timing consistency and timing stability analysis on the evidence model; Based on the response round order and the trend of capability performance changes during the response interaction process, by constructing timing constraint rules, it prohibits the execution of unconstrained score retrospective adjustment for competency dimensions that have experienced capability collapse or changes exceeding the preset range; Undecidable State Resolution and Comprehensive Scoring Module: When the competency dimension is continuously in an undecidable state, the module performs undecidable state resolution processing on each competency dimension according to the evidence priority constraint relationship and time sequence constraint rules; and generates the corresponding competency dimension's scoring result through preset risk scoring rules and conservative judgment strategy, and simultaneously generates the scoring basis path and scoring confidence information associated with the scoring result.

[0042] In this embodiment, the sales skills training assessment of new employees is taken as an example (assessment dimensions: sales script application, customer needs mining, objection handling ability; assessment object: new employee Li). Note: The meanings of the preset letter thresholds are: i = evidence quantity threshold, j = element coverage threshold, km = support strength parameter range, n = time series consistency score threshold, p = time series stability fluctuation threshold, q = time series stability analysis round; the specific process is as follows: The training and assessment system synchronously collected Li's simulated customer communication voice responses, submitted needs analysis reports, and objection handling plans, as well as auxiliary interactive data such as facial expressions, tone of voice, and response intervals during simulated communication. After preprocessing to remove noise, the data was archived as a structured dataset according to the assessment stage timestamps. By comparing it with the pre-set core assessment element list for each competency dimension, the number of evidence items matching the elements and the element coverage ratio were counted to complete the dual-dimensional verification of evidence sufficiency and consistency. The "objection handling ability" was marked as undeterminable because no specific response steps were explained, evidence items < i, and element coverage did not reach j%. At the same time, a tree structure was constructed to hierarchically sort out the data by dimension, undeterminable type, collected evidence, and missing information, and other dimensions were linked through dimension coding.

[0043] By invoking a pre-defined data modality and competency dimension contribution comparison table, it is clarified that voice response data is the core supporting responsibility for handling objections, while text response data is the auxiliary supporting responsibility. Corresponding evidence models are matched accordingly, with deep learning models adapted to core supporting data and traditional machine learning models adapted to auxiliary supporting data. Data standardization, core feature extraction, and matching and calibration with responsibility judgment standards are completed in a three-layer architecture. Support strength parameters in the km interval are output to realize the mapping from evidence to model. Then, the model priority is calculated by weighting the correlation with dimension, evidence credibility, and completeness, and a constraint rule is formulated that core supporting evidence is given priority as the judgment basis, and auxiliary evidence is used to supplement and verify the validity.

[0044] Using timestamp synchronization technology, the evidence model outputs of the three competency dimensions are organized into a time-series data sequence according to the assessment rounds. A sliding window method is used to compare the logical correlation of the same assessment element in adjacent rounds to perform a time-series consistency analysis. If the correlation score is ≥ n, consistency is determined. The fluctuation difference of the results in consecutive rounds q is calculated to perform a time-series stability analysis. If the fluctuation difference is ≤ p, stability is determined. Abnormal situations are marked synchronously. Based on the analysis results, the judgment conditions for competency collapse, the threshold for competency change, and the range of scoring backtracking restrictions are preset to construct time-series constraint rules. Unconstrained scoring backtracking is prohibited for competency dimensions that have experienced competency collapse or changes exceeding the preset range. Scoring can only be based on evidence from the current and subsequent rounds, and previous results cannot be modified.

[0045] Based on the evidence priority constraint, the core supporting evidence model is called first. The assessment system adds questions to supplement missing evidence such as the dialogue for responding to objections and the steps for emotional reassurance. After supplementation, the sufficiency and consistency of the evidence are re-verified. If the standard is not met, stable and valid evidence is selected in combination with the time sequence constraint rule. The resolution is promoted in stages according to the core element priority strategy. If it is still impossible to determine after multiple rounds of processing, the scoring range is matched according to the severity of evidence deficiency according to the preset risk scoring rule. At the same time, a conservative judgment strategy is adhered to, and the scoring is limited only based on valid evidence. Evidence that cannot be verified due to contradictions is interpreted conservatively. Finally, the scoring results of each dimension, the scoring basis path and confidence information are integrated to generate a comprehensive assessment report containing Li's basic information, details of the scoring of each dimension, explanation of the indeterminability resolution, and confidence statistics. The report is pushed to the training instructor's terminal and the full-process assessment data is retained for future reference.

[0046] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A multi-competency intelligent scoring method, characterized in that, include: S1: Obtain multi-source response data for the target assessment object, and perform evidence sufficiency and evidence consistency analysis on each competency dimension according to the preset competency assessment target, and generate corresponding competency judgment status identifiers, including determineable status and undeterminable status; When at least one competency dimension is detected to be in an undecidable state, construct an undecidable state description structure. S2: Based on the undecidable state description structure, perform evidence responsibility allocation processing on the evaluation evidence from different data modalities; Based on the contribution type of each data modality in different competency dimensions, the assessment evidence is mapped to an evidence model that assumes judgment responsibility, and based on the responsibility type of the evidence model, a corresponding evidence priority constraint relationship is generated. S3: Perform temporal consistency and temporal stability analysis on the evidence model; Based on the order of response rounds and the trend of ability performance changes during the response interaction process, by constructing time-series constraint rules, it is prohibited to perform unconstrained score retrospective adjustment on competency dimensions that have experienced capability collapse or changes exceeding the preset range; S4: When the competency dimension remains in an undecidable state, perform undecidable state resolution processing on each competency dimension according to the evidence priority constraint relationship and time sequence constraint rules. By pre-setting risk scoring rules and conservative judgment strategies, scoring results for the corresponding competency dimensions are generated, and scoring basis paths and scoring confidence information associated with the scoring results are generated simultaneously.

2. The method according to claim 1, characterized in that, The specific process of performing sufficiency and consistency analysis of evidence for each competency dimension is as follows: To determine the sufficiency of evidence, a list of core assessment elements is pre-configured for each competency dimension. For a single competency dimension, the number of evidence items matching the core assessment elements from multi-source response data is counted, and thresholds for the number of evidence items and the coverage of elements are set. To determine the consistency of evidence, a two-dimensional verification logic is constructed. For textual evidence, the consistency of the expression of the same element is verified through semantic comparison algorithm; for non-textual evidence, the logical fit with the textual expression is verified through feature association algorithm.

3. The method according to claim 1, characterized in that, The specific process of constructing the undeterminable state description structure is as follows: preset the core elements of the undeterminable state description; build a description framework using a tree data structure, fill it in layers according to competency dimension, undeterminable type, collected evidence and missing contradictory information, generate an independent description node for each undeterminable dimension, and establish a node association index through dimension encoding.

4. The method according to claim 1, characterized in that, The specific process of assigning responsibility for evidence is as follows: a comparison table of the contribution of the data modality and the competency dimension is established in advance, and the contribution attributes of different data modalities to each competency dimension are preset; the corresponding data modality is matched by combining the dimensional information of the undeterminable state description structure and the undeterminable cause; responsibility for judgment is confirmed for the matched modality data, and the missing element evidence or the authenticity of contradictory evidence is supplemented.

5. The method according to claim 1, characterized in that, The specific process of mapping the evidence model for determining liability is as follows: The types of evidence used to determine liability include: core supporting evidence and auxiliary supporting evidence; The evidence model includes a data input layer, a semantic enhancement layer, and a responsibility adaptation layer. The input layer inputs corresponding data modality evaluation evidence and performs standardization processing; the semantic enhancement layer extracts and determines the core features related to responsibility through feature extraction algorithms; the responsibility adaptation layer matches and calibrates the core features with preset responsibility determination standards and outputs the support strength parameters of the evidence for responsibility.

6. The method according to claim 1, characterized in that, The specific process for generating the corresponding evidence priority constraint relationship is as follows: Establish criteria for prioritizing evidence, including its relevance to competency dimensions, credibility, and completeness. Each indicator is pre-defined with a fixed weight, and the priority score of the evidence model is calculated by weighted summation. Priority sequences are formed by ranking the scores from highest to lowest, and constraint rules are established. Priority evidence with higher scores is used as the primary basis for judgment, while priority evidence with lower scores is only used when there is insufficient priority evidence with higher scores. The results must be cross-validated with priority evidence with higher scores to be effective.

7. The method according to claim 1, characterized in that, The specific process of performing the timing consistency and timing stability analysis is as follows: Arrange the output results of each evidence model according to the response round timestamp to construct a time-series data sequence; The temporal consistency analysis compares the logical correlation of evidence results from adjacent rounds and verifies the consistency between the statements and conclusions of the same evaluation element. The time-series stability analysis calculates the fluctuation difference of evidence results in consecutive rounds and verifies the fluctuation difference.

8. The method according to claim 1, characterized in that, The specific process of constructing the time-series constraint rules is as follows: preset the core elements of the time-series constraint rules, including capability collapse judgment conditions, capability change thresholds, and scoring backtracking restrictions; formulate capability collapse judgment conditions, define capability change thresholds, clarify scoring backtracking restrictions, and only score based on evidence from the current and subsequent rounds, prohibiting modification of previously generated scoring results; integrate all elements to form complete time-series constraint rules.

9. The method according to claim 1, characterized in that, The specific process for resolving the undecidable state for each competency dimension is as follows: Based on the evidence priority constraint relationship, priority evidence models with higher scores are prioritized to supplement missing evidence or verify contradictory evidence; the sufficiency and consistency of the supplemented evidence are verified, and the corresponding competency dimension judgment status identifier is updated; if the preset conditions are not met, time-stable and valid evidence is screened in combination with time-series constraint rules, and a phased resolution strategy is adopted to first solve the core evaluation element problem and then process the secondary elements; if the judgment requirements are still not met after multiple rounds of processing, it is determined to be unresolvable.

10. The method according to claim 1, characterized in that, The specific process of the preset risk scoring rules and conservative judgment strategy is as follows: Pre-defined risk scoring rules are used to divide risk scoring ranges for each competency dimension, and values ​​are assigned to the corresponding ranges according to the severity of undetermined states. A conservative judgment strategy is preset, adhering to the principle of not speculating without evidence, scoring is based only on valid evidence with limited scope, and contradictory and unverifiable evidence is assigned a conservative interpretation, with conditions for correcting the scoring results set.

11. The method according to claim 1, characterized in that, The specific process for generating the corresponding competency dimension score results is as follows: valid evidence is screened according to evidence priority and time sequence constraint rules; for decidable dimensions, basic scores are generated based on the evidence model output and score mapping standards; for undecidable dimensions, risk scores are generated according to risk scoring rules and conservative strategies; and the score basis path and confidence information are synchronously associated to form complete score results for each dimension.

12. A multi-competency intelligent scoring system, characterized in that, include: Competency Evidence Verification and Judgment Status Module: Acquires multi-source response data for the target assessment object, performs evidence sufficiency and evidence consistency analysis on each competency dimension according to the preset competency assessment target, and generates corresponding competency judgment status identifiers, which include a judgment status and an undeterminable status. When at least one competency dimension is detected to be in an undecidable state, construct an undecidable state description structure. Evidence liability determination and priority constraint module: Based on the undecidable state description structure, it performs evidence liability allocation processing on evaluation evidence from different data modalities; Based on the contribution type of each data modality in different competency dimensions, the assessment evidence is mapped to an evidence model that assumes judgment responsibility, and based on the responsibility type of the evidence model, a corresponding evidence priority constraint relationship is generated. Evidence timing verification and timing rule module: Performs timing consistency and timing stability analysis on the evidence model; Based on the order of response rounds and the trend of ability performance changes during the response interaction process, by constructing time-series constraint rules, it is prohibited to perform unconstrained score retrospective adjustment on competency dimensions that have experienced capability collapse or changes exceeding the preset range; Undecidable State Resolution and Comprehensive Scoring Module: When the competency dimension is continuously in an undecidable state, the undecidable state resolution process is performed on each competency dimension according to the evidence priority constraint relationship and the time sequence constraint rule. By pre-setting risk scoring rules and conservative judgment strategies, scoring results for the corresponding competency dimensions are generated, and scoring basis paths and scoring confidence information associated with the scoring results are generated simultaneously.

Images