An adaptive skill ability evaluation method and system based on a perturbation mechanism, an electronic device, and a storage medium

Abstract

This invention discloses an adaptive skill assessment method based on a perturbation mechanism, comprising: determining an initial ability score and a standard assessment score based on the answers to preset standard test questions; determining a perturbation difficulty coefficient based on the initial ability score; determining a target perturbation type based on the perturbation difficulty coefficient and the total number of perturbation types; determining perturbation test questions based on the target perturbation type and each preset standard test question; acquiring the answers to each perturbation test question and determining a perturbation assessment score based on the answers; determining a perturbation coping ability score based on the standard assessment score and the perturbation assessment score; and determining a comprehensive ability assessment score based on the perturbation coping ability score for each assessment skill dimension. This invention significantly improves the accuracy of assessing and identifying real-world practical skills through the assessment results of dynamically generated perturbation test questions. This invention also discloses a system, electronic device, and storage medium for implementing the above method.

Description

Technical Field

[0001] This invention relates to the field of computer data processing technology, and in particular to an adaptive skill assessment method, system, electronic device, and storage medium based on a perturbation mechanism. Background Technology

[0002] With the digital transformation of the life services industry, various life skills service platforms (such as home repair, equipment installation, and house cleaning) have gradually emerged. These platforms integrate professional service personnel in society (hereinafter referred to as "service providers") to provide users with convenient and efficient life services, effectively solving problems such as information asymmetry and inconsistent service standards in the traditional service industry.

[0003] To ensure service quality, platforms typically set entry requirements for technicians before they join, involving qualification reviews and skills assessments. Currently, the industry commonly uses online quizzes or offline written tests, focusing on theoretical content such as basic knowledge definitions, standard operating procedures, and safety regulations. For example, assessing air conditioning repair technicians might include questions like "What is the chemical name of refrigerant R22?" and "What is the standard duration for vacuuming?" However, this traditional assessment model has significant limitations. Because the content largely consists of static knowledge requiring memorization, some technicians resort to rote memorization of answers to questions to pass the exam. With sufficient memory, anyone can pass the assessment and obtain platform certification. This results in assessments that only reflect the examinee's "test-taking ability" and "memory." Traditional assessment methods lack simulation of unexpected situations in real service scenarios, making it impossible to directly and objectively evaluate their actual operational skills, troubleshooting abilities, and on-the-spot adaptability. Consequently, disputes often arise due to the inability to resolve complex real-world problems, leading to increased user complaints, decreased service experience, and potential risks to platform operations. This negatively impacts the platform's service quality. Summary of the Invention

[0004] To address the aforementioned problems in the prior art, this invention provides an adaptive skill ability assessment method, system, electronic device, and storage medium based on a perturbation mechanism. The technical problem to be solved by this invention is achieved through the following technical solution: The first aspect of this invention provides an adaptive skill ability assessment method based on a perturbation mechanism, comprising the following steps: Based on the responses of prospective employees to each pre-set standard test question for each target category in each assessment skill dimension, determine the initial ability score and standard assessment score for each assessment skill dimension. The perturbation difficulty coefficient is determined based on the initial ability score, preset basic difficulty value, and preset scaling factor for each assessment skill dimension. The target number of disturbance types is determined from the disturbance type set based on the disturbance difficulty coefficient, the total number of disturbance types, and the preset disturbance intensity weight; The perturbation test questions are determined using a large language model based on the target perturbation type, the standard scenario information, standard task information, standard answer information set in each preset standard test question, and the corresponding perturbation scenario information. Obtain the answer information for each assessment skill dimension of each perturbation test question, and determine the perturbation assessment score for each assessment skill dimension based on the similarity between the answer information for each assessment skill dimension of each perturbation test question and the perturbation answer information set in each perturbation test question; Based on the standard assessment score and the disturbance assessment score, determine the disturbance response capability score for each assessment skill dimension; The overall ability assessment score and corresponding assessment information are determined based on the disturbance response ability score for each assessment skill dimension and the preset dimension weight for each assessment skill dimension.

[0005] In one embodiment of the present invention, determining the initial ability score and standard assessment score for each assessment skill dimension based on the responses of the applicant to each preset standard test question for the target category includes: Based on the responses of prospective interns to each pre-set standard test question for each assessment skill dimension of the target category, the similarity of the standard answer information set for each pre-set standard test question for each assessment skill dimension, and the number of pre-set standard test questions, determine the standard assessment score for each pre-set standard test question and the initial ability score for each assessment skill dimension of all pre-set standard test questions.

[0006] In one embodiment of the present invention, the step of determining the target disturbance type quantity from the disturbance type set based on the disturbance difficulty coefficient, the total number of disturbance types, and a preset disturbance intensity weight includes: The number of target disturbance types is determined based on the disturbance difficulty coefficient and the total number of disturbance types. ; The perturbation types in the perturbation type set are sorted from high to low according to the preset perturbation intensity weight of the perturbation type; Filter the sorted set of perturbation types before... One perturbation type is used as the target perturbation type.

[0007] In one embodiment of the present invention, the step of determining the perturbation test question through a large language model based on the target perturbation type, standard scenario information, standard task information, standard answer information set, and corresponding perturbation scenario information in each preset standard test question includes: Based on the target perturbation type and the standard scenario information in each preset standard test question, the perturbation scenario information for each target perturbation type is determined using a large language model; The perturbation test question is determined by a large language model based on the standard scenario information, standard task information, standard answer information set, and corresponding perturbation scenario information in each preset standard test question.

[0008] In one embodiment of the present invention, determining the perturbation assessment score for each assessment skill dimension based on the similarity between the answer information for each assessment skill dimension of each perturbation test question and the set of perturbation answer information in each perturbation test question includes: The textual similarity of each perturbation test question for each assessment skill dimension is determined based on the similarity between the textual interpretation of the answer information for each perturbation test question and the textual interpretation of the perturbation answer information set for each perturbation test question. Based on the similarity between the image answer information of each assessment skill dimension of each perturbation test question and the image answer information of the perturbation answer information set, the image matching degree of each assessment skill dimension of each perturbation test question is determined; Based on the similarity between the step-by-step answer information of each assessment skill dimension of each perturbation test question and the step-by-step answer information of the perturbation answer information set, the step similarity of each assessment skill dimension of each perturbation test question is determined. The perturbation score for each assessment skill dimension is obtained by weighted summation based on the text similarity, image matching degree, step similarity, and corresponding modal weights; wherein, the modal weights are calculated based on the information entropy of the text similarity, image matching degree, and step similarity, respectively.

[0009] In one embodiment of the present invention, the formula for calculating the disturbance difficulty coefficient is as follows: in, This indicates the difficulty level of the disturbance; 0.1 represents the preset base difficulty value, and 0.8 represents the preset scaling factor. This represents the average of the initial ability scores across all assessment skill dimensions.

[0010] In one embodiment of the present invention, the formula for calculating the comprehensive score of the ability assessment is as follows: in, This indicates the overall score of the ability assessment. Indicates the first The preset weights for each assessment skill dimension. Indicates the first The score reflects the ability to cope with disturbances across various skill dimensions.

[0011] A second aspect of this invention provides an adaptive skill assessment system based on a perturbation mechanism, comprising: The first determination module is used to determine the initial ability score and standard assessment score for each assessment skill dimension based on the answers of the applicants to each preset standard test question for each target category. The second determining module is used to determine the disturbance difficulty coefficient based on the initial ability score, preset basic difficulty value and preset scaling factor for each assessment skill dimension; The third determining module is used to determine the target disturbance type number in the disturbance type set based on the disturbance difficulty coefficient, the total number of disturbance types and the preset disturbance intensity weight; The fourth determining module is used to determine the perturbation test question through a large language model based on the target perturbation type, the standard scenario information, standard task information, standard answer information set in each preset standard test question, and the corresponding perturbation scenario information. The first calculation module is used to obtain the answer information of each assessment skill dimension of each of the perturbation test questions, and determine the perturbation assessment score of each assessment skill dimension based on the similarity between the answer information of each assessment skill dimension of each of the perturbation test questions and the perturbation answer information set in each of the perturbation test questions. The second calculation module is used to determine the disturbance response capability score for each assessment skill dimension based on the standard assessment score and the disturbance assessment score. The evaluation module is used to determine the comprehensive ability evaluation score and corresponding evaluation information based on the disturbance response ability score of each assessment skill dimension and the preset dimension weight of each assessment skill dimension.

[0012] A third aspect of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement an adaptive skill capability assessment method based on a perturbation mechanism provided in the first aspect of the present invention.

[0013] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements an adaptive skill capability assessment method based on a perturbation mechanism provided in the first aspect of the present invention.

[0014] The beneficial effects of this invention are: This invention uses dynamically generated perturbation test questions to simulate the complexity and uncertainty of real-world service scenarios. This makes the assessment results not merely a reflection of theoretical scores, but an effective prediction of future service performance. The assessment results demonstrate the ability to transform theoretical knowledge into effective solutions, reflecting the true capabilities of prospective staff and improving the accuracy of the assessment results. This ensures the overall quality of platform services and user satisfaction from the source, reducing performance risks and customer complaints caused by insufficient capabilities of prospective staff (mentors). The difficulty and type of subsequent perturbation questions are dynamically adjusted based on the initial assessment of prospective staff, avoiding polarization. This ensures that highly capable individuals can fully demonstrate their potential to solve complex problems, while preventing less capable individuals from being unable to demonstrate their true level due to overly difficult questions. This matches the assessment pressure with the ability level, significantly improving the accuracy of the assessment system in identifying real-world practical skills.

[0015] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings.

[0016] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0017] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 A flowchart illustrating an adaptive skill assessment method based on a perturbation mechanism provided in an embodiment of the present invention; Figure 2 This is a block diagram of an adaptive skill assessment system based on a perturbation mechanism, provided as an embodiment of the present invention. Detailed Implementation

[0018] The present invention will be further described in detail below with reference to specific embodiments, but the implementation of the present invention is not limited thereto.

[0019] like Figure 1As shown, the first aspect of this invention provides an adaptive skill ability assessment method based on a perturbation mechanism, comprising the following steps: Step 11: Based on the answers of the applicants to each preset standard test question for each assessment skill dimension of the target product category, determine the initial ability score and standard assessment score for each assessment skill dimension.

[0020] Step 12: Determine the perturbation difficulty coefficient based on the initial ability score, preset basic difficulty value, and preset scaling factor for each assessment skill dimension.

[0021] Step 13: Determine the target disturbance type number from the disturbance type set based on the disturbance difficulty coefficient, the total number of disturbance types, and the preset disturbance intensity weight.

[0022] Step 14: Determine the perturbation test questions using a large language model based on the target perturbation type, the standard scenario information, standard task information, standard answer information set, and corresponding perturbation scenario information in each preset standard test question.

[0023] Step 15: Obtain the answer information for each assessment skill dimension of the perturbation test question. Based on the similarity between the answer information for each assessment skill dimension of the perturbation test question and the perturbation answer information set in the perturbation test question, determine the perturbation assessment score for each assessment skill dimension.

[0024] Step 16: Determine the disturbance response capability score for each assessment skill dimension based on the standard assessment score and the disturbance assessment score.

[0025] Step 17: Determine the overall ability assessment score and corresponding assessment information based on the disturbance response ability score for each assessment skill dimension and the preset dimension weight for each assessment skill dimension.

[0026] In this embodiment, dynamically generated perturbation test questions simulate the complexity and uncertainty of real-world service scenarios. This makes the assessment results not merely a reflection of theoretical scores, but an effective prediction of future service performance. The assessment results directly demonstrate the ability to transform theoretical knowledge into effective solutions, reflecting the true capabilities of prospective staff. This ensures the overall quality of platform services and user satisfaction from the outset, reducing performance risks and customer complaints caused by insufficient capabilities of prospective staff (mentors). The difficulty and type of subsequent perturbation questions are dynamically adjusted based on the initial assessment of prospective staff, avoiding polarization. This ensures that highly capable individuals can fully demonstrate their potential to solve complex problems, while preventing less capable individuals from failing to demonstrate their true level due to overly difficult questions. This matches assessment pressure with ability levels, significantly improving the accuracy of the assessment system in identifying real-world practical skills.

[0027] Based on the first aspect of the present invention, the second aspect of the present invention provides a more detailed description of an adaptive skill ability assessment method based on a perturbation mechanism. The second aspect of the present invention provides an adaptive skill ability assessment method based on a perturbation mechanism, comprising the following steps: Step 21: Based on the answers of the applicants to each pre-set standard test question for each assessment skill dimension of the target category, determine the standard assessment score for each pre-set standard test question for each assessment skill dimension and the initial ability score for each assessment skill dimension.

[0028] In this step, the personnel to be registered, who are also the skilled service providers to be registered, can choose at least one target category when registering on the platform. Target categories include home appliance repair, equipment installation, and housekeeping. They can further select subcategories, such as washing machine repair and air conditioner repair within home appliance repair. Each target category corresponds to a pre-set standard test question bank. The question bank stores standardized test questions for each category (such as basic knowledge point definitions and standard operating procedures). Each standardized test question consists of three parts: standard scenario information, etc. Standard task information and standard answer information set Standard scenario information refers to the description of the test scenario, such as a washing machine not draining; standard task information refers to the answer requirements, such as troubleshooting the problem and providing a solution; and standard answer information refers to the set of information provided. Includes steps, tools, and results, built upon industry standards and high-quality service cases. The standard answer information set contains multiple standard answer information. In actual operation scenarios, each scenario may use multiple methods with certain differences to process and solve the problem, and each method is a correct answer. Therefore, each test question can correspond to multiple correct answers, forming the standard answer information set.

[0029] Each standard test question contains multiple dimensions that assess assessment skills. ,Include K Each dimension can also be understood as a dimension of the answer; one assessment skill dimension corresponds to multiple standard test questions. For example, the assessment skills for home appliance repair include three dimensions. Preset weights for each assessment skill dimension. And satisfy (The weights are determined through industry expert scoring and training with real service scenario data to ensure they align with actual service needs.) The content of the standard answer information set for each standard test question is broken down according to the assessment skill dimensions. The multiple pieces of content after the breakdown are associated with the corresponding assessment skill dimensions, and each assessment skill dimension is associated with multiple standard answer information sets.

[0030] The standard test question bank is Single standard test question : : No. i Standard scenario information for the question; : No. i Standard task information for the question; : No. i The standard answer information set for the question (including) J A standard answer, broken down according to the assessment skills dimensions. For the first i Question No. k The standard answer for dimensions, such as , , Includes J (The answer to the problem location information is a standard answer).

[0031] Example, The washing machine is not draining. Troubleshoot the problem and provide a solution. Inspect the drain pipe, clean the filter, and test the drainage. .

[0032] Generally, the standard answer information set includes three modalities: textual explanation answer information, image answer information, and step-by-step answer information. Image answer information includes a box around the answer location in the image and the corresponding text description. The assessment skill dimension corresponding to the text description is also the assessment skill dimension to which the image at the boxed location belongs.

[0033] Furthermore, upon joining the platform, applicants will be directed to a standard test assessment interface to answer questions. After completion, the system will retrieve their answers and associate different content within those answers with corresponding assessment skill dimensions. This process is repeated for each applicant. i Answer information for the standard test questions, recorded as This includes textual explanations and answer information. Image-based answer information Step-by-step answer information The three modalities of response information can be broken down and evaluated according to the assessment skills dimension.

[0034] Example of a candidate's response: The text description is "Test drainage after cleaning the filter", but the image does not label key points, and the step sequence is [clean the filter, test drainage].

[0035] The answer information (text extracted from the answer annotation position in the image) is input into a natural language processing model (BiLSTM-CRF joint model or Transformer-based pre-trained language model) to classify the assessment skill dimensions. Each assessment skill dimension is a label, and the answer information of each question is classified according to the assessment skill dimension.

[0036] Specifically, based on the answers of prospective interns to each pre-set standard test question for each assessment skill dimension of the target category, the similarity of the standard answer information set for each pre-set standard test question for each assessment skill dimension, and the number of pre-set standard test questions, the standard assessment score for each pre-set standard test question and the initial ability score for each assessment skill dimension of all pre-set standard test questions are determined.

[0037] Here, the first The calculation process for the standard assessment score for each assessment skill dimension includes steps A1-A4: Step A1: Determine the textual similarity of each standard test question for each assessment skill dimension based on the similarity between the textual explanation of the answer information for each assessment skill dimension and the textual explanation of the answer information set of the standard answer.

[0038] In this step, for a test question's assessment skill dimension, the textual explanations of the answer information and the corresponding textual explanations of the answer information under that dimension are converted into high-dimensional word vectors using the BERT model (one word vector is generated for the textual explanations of the answer information as a whole, and another word vector is generated for the textual explanations of the answer information as a whole). i Test Question No. k The word vectors of the textual explanations and answers for each assessment skill dimension are: , No. i Test Question No. k The first assessment skill dimension j The textual explanation of each answer is given by the word vector of the answer information. Calculate the cosine similarity between the two to obtain the text similarity for each assessment skill dimension of each test question. The calculation formula is as follows: in, The value ranges from 1 to J , J For the first i The number of standard answer information items in the standard answer information set of the test questions, where • represents the vector dot product. Given the vector magnitude, we first calculate the cosine similarity between each answer and the response information under a specific skill dimension for a question. J 1. Similarity score, then take the value under that dimension. J The maximum value among the similarity scores, Indicates taking the first k The maximum similarity between the answer information of a dimension and all answer information (ensuring coverage of multiple reasonable expressions).

[0039] Step A2: Determine the image matching degree of each assessment skill dimension of each standard test question based on the feature matching degree between the image answer information of each assessment skill dimension of each standard test question and the image answer information of the standard answer information set.

[0040] In this step, a pre-trained CNN model is used to extract feature point vectors from the bounding boxes (answer annotations) in the image answer information, and the image matching degree is calculated with the extracted feature vectors of the image answer information. : in: For the first i Test Question No. k Assessment Skills Dimension j The first standard answer information t A feature point vector, Indicates the first i The first test question k The first part of the response information in the assessment skills dimension t A vector of feature points.

[0041] No. j The image answer information of the standard answer is totaling [number]. m The image response information also includes feature point vectors. m 1 eigenvector Indicates taking the first k The maximum similarity between the answer information and all answer information in each dimension. The matching degree between each standard answer and the answer information for each feature is calculated separately, yielding the results. J Each feature has a matching degree, and then the feature matching degree is taken under that dimension. J The maximum value among the feature matching degrees is the image matching degree, and the assessment skill dimension of the feature vector of the bounding position is also the assessment skill dimension of its corresponding text description.

[0042] Step A3: Determine the step similarity of each standard test question for each assessment skill dimension based on the similarity between the step-by-step answer information and the step-by-step answer information of the standard answer information for each assessment skill dimension.

[0043] In this step, in, The higher the value, the closer the answer is to the information provided. Indicates the first i Test Question No. k The steps for answering questions in the skills assessment dimension, with a length of [length missing]. , Indicates the first i Test Question No. k The standard answer information for the skills assessment dimension includes the steps and a length of [length missing]. The length here can be understood as the total number of steps. The dynamic time-normalized distance is used to address issues arising from minor differences or omissions in the sequence of steps. The recursive calculation formula is as follows: The first step in calculating the answer information Step and answer information When calculating the distance of a step, take the minimum value among "previous step, left step, and diagonal step", and add it to the difference of the current step. This applies to the i-th step of method A and the j-th step of method B. This indicates that the steps and functions are consistent. This indicates differences in steps.

[0044] Step A4: Calculate the standard assessment score for each assessment skill dimension of each standard test question by weighted summation based on text similarity, image matching degree, step similarity, and corresponding modal weights.

[0045] The modality weights are calculated based on the information entropy of text similarity, image matching degree, and step similarity, respectively.

[0046] In this step, the first i Standard test question number Standard assessment scores for each assessment skill dimension The calculation formula is: in, Modal weights representing text similarity Modal weights representing image matching degree Modal weights representing step similarity , , , , Information entropy is used to measure text. ,image ,step Three modal scores , , The greater the fluctuation (the smaller the entropy value), the stronger the mode's distinguishing ability and the higher its weight.

[0047] here: in, Indicates the first m The first mode in the 1st i The percentage of marks awarded for each question. Here, N This indicates the total number of test questions. , , Indicates the first m The first mode in the 1st i The similarity score of the standard test questions, that is, the score of the first question. i The similarity scores of all skills assessed in one modality are summed to obtain the result. For example, in the results obtained in step 251, the text similarity of a question across all dimensions is summed to obtain... The denominator is all N Test Questions Add them together.

[0048] For example, the information entropy of the three modalities are respectively ， Weight calculation: ； Total weights ； If the text similarity is 0.8, the image matching score is 0.4, and the step similarity is 0.7, then the perturbation score for this skill dimension is... .

[0049] Furthermore, the formula for calculating the initial ability score is: in, Indicates the first k The initial ability score for the assessment skills dimension. This represents the set of standard test items. This represents the total number of standard test items in the standard test item set.

[0050] Step 22: Determine the perturbation difficulty coefficient based on the initial ability score, preset basic difficulty value, and preset scaling factor for each assessment skill dimension.

[0051] In this step, the formula for calculating the disturbance difficulty coefficient is: in, This represents the perturbation difficulty coefficient, where 0.1 represents the preset base difficulty value and 0.8 represents the preset scaling factor (mapping the average ability to the [0,0.8] range, and covering a reasonable assessment range after adding the base difficulty). This represents the average of the initial ability scores across all assessment skill dimensions. , Difficulty level of disturbance It is positively correlated with the initial ability score (the higher the ability, the greater the difficulty of the perturbation), and the value range is constrained to [0.1, 0.9] (to avoid the difficulty being too low and meaningless, or too high and causing the assessment to fail).

[0052] Example of a highly skilled master craftsman: Then β = 0.1 + 0.8 × 0.9 = 0.82 (for a relatively high-difficulty perturbation); Intermediate-level master craftsman: Therefore, β = 0.1 + 0.8 × 0.5 = 0.5 (a medium-difficulty perturbation); Low-skilled craftsman: Then β = 0.1 + 0.8 × 0.2 = 0.26 (for a relatively low-difficulty perturbation).

[0053] Step 23: Determine the target disturbance type number from the disturbance type set based on the disturbance difficulty coefficient, the total number of disturbance types, and the preset disturbance intensity weight.

[0054] Step 23 includes steps 231-233: Step 231: Determine the number of target disturbance types based on the disturbance difficulty coefficient and the total number of disturbance types.

[0055] In this step, the number of target disturbance types The calculation formula is: in, This indicates the total number of disturbance types corresponding to the target product category.

[0056] Here, the set of perturbation types It covers common abnormal disturbance types in practical scenarios, and each disturbance type corresponds to a preset disturbance strength weight. This represents the intensity of the disturbance's impact on the standard task, quantified through expert assessment and the degree of fault impact. For example, Secondary fault ( ), Tool mismatch ( ), User misleading ( ), Environmental interference ( ), ,but That is, select 3 types of disturbance.

[0057] Step 232: Sort the disturbance types in the disturbance type set from high to low according to the preset disturbance intensity weight of the disturbance type.

[0058] In this step, the disturbance types are sorted from high to low by the preset disturbance intensity weights.

[0059] Step 233: Filter the previous perturbation types from the sorted set. One perturbation type is used as the target perturbation type.

[0060] For example, the top 3 perturbation types selected are: Secondary fault ( ), Tool mismatch ( ), User misleading ( ). Set of target disturbance types Step 24: Determine the perturbation test questions using a large language model based on the target perturbation type, the standard scenario information, standard task information, standard answer information set, and corresponding perturbation scenario information in each preset standard test question.

[0061] Step 24 includes steps 241-242: Step 241: Determine the perturbation scenario information using a large language model based on all target perturbation types and standard scenario information in each preset standard test question.

[0062] In this step, for each target disturbance type, disturbance scenario information is generated based on the standard scenario information in each preset standard test question. The disturbance scenario information corresponding to each preset standard test question must include... t The corresponding scenario information for each target disturbance type. Here, the disturbance scenario information must conform to the practical scenario of the preset standard test questions and cannot deviate from the preset standard test questions (e.g., the disturbance in the washing machine test question cannot be "air conditioner remote control malfunction"); the disturbance scenario information of each disturbance type within the combination is independent of each other and has no logical conflict (e.g., "drain pipe bend" and "tool mismatch" are independent anomalies). The disturbance scenario information is related to... Specific descriptions of the relevant exceptions.

[0063] Specifically, the first i Standard scenario information for pre-set standard test questions The target perturbation type, corresponding prompt words, and requirement prompt words are input into the large language model, and the output is perturbation scenario information. Example prompts include, but are not limited to: for secondary faults. Tool mismatch Misleading users Three types of generation and standard scenario information: washing machine not draining The descriptions should include three types of disturbance scenarios, and must be consistent with the practical scenario of a washing machine not draining; the disturbance scenario information must be independent of each other and logically consistent. Example output: User misoperation caused the drain pipe to bend (secondary fault) The specified wrench model is not compatible (tool mismatch). The user claimed that "the filter has been replaced and no inspection is needed" (this is misleading). )".

[0064] Step 242: Generate perturbation test questions using a large language model based on the standard scenario information, standard task information, standard answer information set, and corresponding perturbation scenario information in each preset standard test question.

[0065] In this step, the perturbation scenario for each question will be... Compared to standard scenarios Overlay, in conjunction with the standard task information for each question. Standard answer information set The assessment skills dimension uses a large language model to generate a perturbation set of answer information. Accordingly, each test question can have multiple correct answers. Includes J A perturbation of the answer information, Specifically, the disturbance scene Compared to standard scenarios and standard task information The output needs to include the corresponding content for each assessment skill dimension in the standard answer information set and output the corresponding assessment skill dimension. It should also provide multiple solutions as prompts and other required prompts to be input into the large language model. For example, prompts include, but are not limited to: "Based on the washing machine not draining..." “User misoperation caused the drain pipe to bend, the specified wrench model was incompatible, and the user claimed that the filter had been replaced and no inspection was needed.” In the scenario described above, please provide troubleshooting instructions and a solution. The specific operation requires that the output include the corresponding content for each assessment skill dimension in the standard answer information set and output the corresponding assessment skill dimension. Correct bends, adapt tools, perform a second inspection of the filter, test drainage, and clarify any misleading points. (This example shows a single perturbation in the answer information.) At the same time, the labels for the assessment skills dimension of each content in the answer information are output. Of course, the prompts can also include descriptions of other required requirements, such as providing operation images or multiple solutions.

[0066] Final output i The first pre-set standard test question corresponds to the first i Disturbance test questions Generally speaking, Similar to the standard answer information set, it includes, but is not limited to, textual explanations of answer information, image-based answer information, and step-by-step answer information. Image-based answer information includes a box around the answer location in the image and the corresponding text description. The assessment skill dimension corresponding to the text description is also the assessment skill dimension to which the image at the boxed location belongs.

[0067] In one feasible implementation, after generating the perturbation test questions, it is verified whether the generated perturbation test questions satisfy the validity constraints: Validity constraints , specifically, in, For the first i The intersection of the standard answer information set and the perturbation answer information set (i.e., the steps that can be completed without dealing with the perturbation). For the first i The number of intersections between the standard answer information set and the perturbation answer information set of a question. No. i The number of intersections between the perturbation answer information sets of the question. This indicates that over 70% of the valid steps need to be adjusted for perturbations to avoid path dependency. Each standard answer in the standard answer information set and each perturbation answer in the perturbation answer information set are calculated, and all results must satisfy validity constraints. If any one fails to satisfy a constraint, a new perturbation type is added.

[0068] If the validity constraint is not satisfied, then proceed as follows: Add the next one in descending order of height (the first one) t +1) Disturbance type, until the constraint is satisfied.

[0069] Example, standard question Scenario: "Washing machine not draining", standard answer ; Introducing target disturbance type Disturbance scene ; Perturbation valid answer (4 steps); Validity constraints for computation: (2 steps) then The constraint is not satisfied; Optimize perturbations: Add target perturbation types Regenerate perturbation test questions and perturbation answers. (5 steps); Recalculate the validity constraints: (Step 1) This satisfies the validity constraint.

[0070] Step 25: Obtain the answer information for each assessment skill dimension of each perturbation test question. Based on the similarity between the answer information for each assessment skill dimension of each perturbation test question and the perturbation answer information set in each perturbation test question, determine the perturbation assessment score for each assessment skill dimension.

[0071] After receiving the perturbation test questions, they are sent to the terminals of the personnel to be registered. The personnel answer the questions, and the answer information for each perturbation test question is obtained. This answer information is then categorized according to the assessment skill dimensions, resulting in the answer information for each assessment skill dimension of each perturbation test question. The answer information (text extracted from marked locations in images) is input into a natural language processing model (BiLSTM-CRF joint model or a Transformer-based pre-trained language model) for classification according to the assessment skill dimensions. Each assessment skill dimension is assigned a label, thus categorizing the answer information according to the assessment skill dimensions.

[0072] The specific process for determining the disturbance assessment score includes steps 251-254, which are the same as the calculation methods for steps A1-A4 of the standard assessment score calculation process, and can be calculated using the relevant parameters of the disturbance test question. Step 251: Determine the textual similarity of each assessment skill dimension of each perturbation test question based on the similarity between the textual interpretation of the answer information for each assessment skill dimension of each perturbation test question and the textual interpretation of the answer information in the perturbation answer information set.

[0073] In this step, for a test question's assessment skill dimension, the textual explanations of the answer information and the corresponding textual explanations of the answer information under that dimension are converted into high-dimensional word vectors using the BERT model (one word vector is generated for the textual explanations of the answer information as a whole, and another word vector is generated for the textual explanations of the answer information as a whole). i Test Question No. kThe word vectors of the textual explanations and answers for each assessment skill dimension are: , No. i Test Question No. k The first assessment skill dimension j The textual explanation of each answer is given by the word vector of the answer information. Calculate the cosine similarity between the two to obtain the text similarity for each assessment skill dimension of each test question. The calculation formula is as follows: in, The value ranges from 1 to J , J For the first i The number of perturbation answer pieces in the perturbation answer information set of the test question, where • represents the vector dot product. Given the vector magnitude, we first calculate the cosine similarity between each answer and the response information under a specific skill dimension for a question. J 1. Similarity score, then take the value under that dimension. J The maximum value among the similarity scores, Indicates taking the first k The maximum similarity between the answer information of a dimension and all answer information (ensuring coverage of multiple reasonable expressions).

[0074] Step 252: Determine the image matching degree of each assessment skill dimension of each perturbation test question based on the feature matching degree between the image answer information of each assessment skill dimension of each perturbation test question and the image answer information of the perturbation answer information set.

[0075] In this step, a pre-trained CNN model is used to extract feature point vectors from the bounding boxes (answer annotations) in the image answer information, and the image matching degree is calculated with the extracted feature vectors of the image answer information. : in: For the first i Test Question No. k Assessment Skills Dimension j The first perturbation of the answer information t A feature point vector, Indicates the first i The first test question k The first part of the response information in the assessment skills dimension t A vector of feature points.

[0076] No. j The image answer information of the perturbation answer information is totaling [number]. mThe image response information also includes feature point vectors. m 1 eigenvector Indicates taking the first k The maximum similarity between the answer information and all answer information in each dimension. The matching degree between each perturbed answer information and the answer information for each feature is calculated, yielding the results. J Each feature has a matching degree, and then the feature matching degree is taken under that dimension. J The maximum value among the feature matching degrees is the image matching degree, and the assessment skill dimension of the feature vector of the bounding position is also the assessment skill dimension of its corresponding text description.

[0077] Step 253: Determine the step similarity of each assessment skill dimension of each perturbation test question based on the similarity between the step-by-step answer information of the answer information and the step-by-step answer information of the perturbation answer information for each assessment skill dimension of each perturbation test question.

[0078] In this step, the step similarity for each perturbation test item across each assessed skill dimension is calculated. The calculation formula is: in, The higher the value, the closer the answer is to the information provided. Indicates the first i Test Question No. k The steps for answering questions in the skills assessment dimension, with a length of [length missing]. , Indicates the first i Test Question No. k The perturbation information in the answer information of the assessment skills dimension, with a length of [length missing]. The length here can be understood as the total number of steps. The dynamic time-normalized distance is used to address issues arising from minor differences or omissions in the sequence of steps. The recursive calculation formula is as follows: The first step in calculating the answer information Step and answer information When calculating the distance of a step, take the minimum value among "previous step, left step, and diagonal step", and add it to the difference of the current step. This applies to the i-th step of method A and the j-th step of method B. This indicates that the steps and functions are consistent. This indicates differences in steps.

[0079] Step 254: Perform a weighted summation based on text similarity, image matching degree, step similarity, and corresponding modal weights to obtain the perturbation assessment score for each assessment skill dimension of each perturbation test question.

[0080] The modality weights are calculated based on the information entropy of text similarity, image matching degree, and step similarity, respectively.

[0081] In this step, the first i Disturbance test question No. Disturbance assessment scores for each assessment skill dimension The calculation formula is: in, Modal weights representing text similarity Modal weights representing image matching degree Modal weights representing step similarity The modal weights are calculated based on information entropy using the same method as in step A4, and are calculated using relevant parameters from the perturbation test questions. This will not be elaborated further here.

[0082] Step 26: Determine the disturbance response capability score for each assessment skill dimension based on the standard assessment score and the disturbance assessment score.

[0083] In this step, the first The formula for calculating the disturbance response ability score for each assessment skill dimension is as follows: in, express The average value of the preset perturbation strength weights for each target perturbation type. Indicates the first i Disturbance test question No. The perturbation score of each assessment skill dimension Indicates the first i Standard test question number The standard assessment score for each skill dimension is [not specified]. This indicates the difficulty level of the disturbance. This indicates the total number of test questions, with the number of perturbation test questions and standard test questions being the same. If the score on perturbation questions is greater than or equal to the score on standard questions, it indicates a strong ability to handle perturbations; if the score drops sharply, it indicates the existence of path dependence.

[0084] Step 27: Determine the overall ability assessment score and corresponding assessment information based on the disturbance response ability score of each assessment skill dimension and the preset dimension weight of each assessment skill dimension.

[0085] The specific steps of this step are as follows: First, according to the... k Initial ability score in the assessment skills dimension and the k Perturbation response ability score in the assessment skills dimension Calculate the first kThe actual ability score in the assessment skills dimension, number k The actual ability score in the assessment skills dimension The calculation formula is as follows: in, This is the pre-balance coefficient, which focuses on the ability to cope with disturbances.

[0086] Then, the overall ability assessment score is calculated by summing the products of the actual ability score for each assessment skill dimension and the preset weights for each assessment skill dimension. The calculation formula is as follows: in, Indicates the first k Assess the true ability score in the skills dimension. Indicates the first k Preset weights for the assessment skills dimensions.

[0087] Then, based on the preset threshold Received assessment information: Preset threshold ,like Those deemed to have met the actual requirements are allowed to join.

[0088] like And exist The application was deemed to be a path-dependent fraudulent capability and its application was rejected.

[0089] like And all If the candidate is deemed to have failed to meet the actual ability standard, a make-up exam will be allowed.

[0090] In one feasible implementation, when And all When the time comes, iterative dynamic adjustments to the perturbation difficulty coefficient and target perturbation type are triggered, and a make-up test is conducted after the adjustment. Specifically, based on actual ability scores Adjusting the difficulty coefficient of disturbance : Based on the adjusted disturbance difficulty coefficient Adjust the target disturbance type: in, Indicates the current value. This represents the value after one round of iterative adjustment. After adjustment, return to steps 24-27 to generate the next round of perturbation test questions and obtain the comprehensive ability assessment score and corresponding assessment information. The difference between the comprehensive ability assessment scores of two consecutive rounds is also considered. The iteration adjustment terminates when the number of iterations equals 3.

[0091] This invention achieves closed-loop assessment through four modules: "adaptive perturbation generation → multimodal answer analysis → real ability assessment → dynamic adjustment and iteration". Based on the instructor's initial answer performance, it intelligently generates perturbation test questions that are appropriate to their ability level. By analyzing the perturbation response behavior, it accurately eliminates "path-dependent false abilities" and outputs skill scores that fit the real service scenario.

[0092] The overall calculation method of this invention can be summarized into five parts: 1. Initial ability pre-assessment: The instructor answers 1-2 standard questions, and the initial ability vector is calculated according to the skill dimension; 2. Adaptive perturbation generation: The perturbation difficulty is determined based on the initial ability, and perturbation questions with abnormal conditions are generated; 3. Multimodal answer analysis: Text, images, and steps are quantitatively scored according to dimensions, and the score of each question is obtained by weighting; 4. Real ability assessment: The scores of standard questions and perturbation questions are compared to quantify the ability to cope with perturbations and identify path dependencies; 5. Dynamic iterative adjustment: The perturbation parameters are iteratively optimized, and the final assessment result is output after the score stabilizes. The adaptive perturbation generation mechanism of this invention dynamically adjusts the difficulty and type of perturbation based on the initial ability, solving the problem of "insufficient differentiation" of traditional fixed perturbations, ensuring that high-ability instructors face high-difficulty perturbations, while low-ability instructors are not overly challenged. The multi-dimensional + multimodal assessment framework of this invention: One question covers multiple skill dimensions, and answers are provided in multiple modalities such as text, images, and steps, solving the pain points of "subjective scoring and single form" in practical assessments, and making the quantitative assessment more objective. The path dependency identification logic of this invention: by perturbation response effectiveness score, it accurately distinguishes between masters who "memorize the process by rote" and masters who "have real problem-solving ability", thus solving the core defect of traditional assessment that is "high score, poor practical skills".

[0093] In this invention, the perturbation type, dimension weight, and judgment threshold can all be obtained through training on the platform's historical service data (fault records, service satisfaction rates); cosine similarity, CNN feature extraction, and DTW algorithm are all open-source technologies that can be quickly implemented using TensorFlow and Scikit-learn; it supports adding new categories (such as decoration and housekeeping), requiring only the addition of standard test questions and perturbation types for the corresponding category, without needing to refactor the core logic, thus possessing scalability. This invention exposes the shortcomings of service providers' "insufficient emergency handling capabilities" through perturbation test questions, avoiding misjudgments of their competence due to average scores on standard questions, and ensuring that registered service providers possess genuine service capabilities.

[0094] like Figure 2As shown, a third aspect of the present invention provides an adaptive skill ability assessment system based on a perturbation mechanism, comprising: The first determining module 31 is used to determine the initial ability score and standard assessment score for each assessment skill dimension based on the answer information of the personnel to be admitted to each preset standard test question for the target category for each assessment skill dimension. The second determining module 32 is used to determine the perturbation difficulty coefficient based on the initial ability score, preset basic difficulty value and preset scaling factor for each assessment skill dimension; The third determining module 33 is used to determine the target disturbance type number in the disturbance type set based on the disturbance difficulty coefficient, the total number of disturbance types and the preset disturbance intensity weight; The fourth determination module 34 is used to determine the perturbation test questions based on the target perturbation type, the standard scenario information, standard task information, standard answer information set and corresponding perturbation scenario information in each preset standard test question through a large language model; The first calculation module 35 is used to obtain the answer information of each assessment skill dimension of each perturbation test question, and determine the perturbation assessment score of each assessment skill dimension based on the similarity between the answer information of each assessment skill dimension of each perturbation test question and the perturbation answer information set in each perturbation test question. The second calculation module 36 is used to determine the disturbance response capability score for each assessment skill dimension based on the standard assessment score and the disturbance assessment score. The assessment module 37 is used to determine the comprehensive assessment score and corresponding assessment information based on the disturbance response capability score of each assessment skill dimension and the preset dimension weight of each assessment skill dimension.

[0095] In one embodiment of the present invention, based on the responses of prospective employees to each preset standard test question for each assessment skill dimension of the target product category, an initial ability score and a standard assessment score for each assessment skill dimension are determined, including: Based on the responses of prospective interns to each pre-set standard test question for each assessment skill dimension of the target category, the similarity of the standard answer information set for each pre-set standard test question for each assessment skill dimension, and the number of pre-set standard test questions, determine the standard assessment score for each pre-set standard test question and the initial ability score for each assessment skill dimension of all pre-set standard test questions.

[0096] In one embodiment of the present invention, determining the target disturbance type number from the disturbance type set based on the disturbance difficulty coefficient, the total number of disturbance types, and a preset disturbance intensity weight includes: The number of target disturbance types is determined based on the disturbance difficulty coefficient and the total number of disturbance types. ; The perturbation types in the perturbation type set are sorted from high to low according to the preset perturbation intensity weight of the perturbation type; Filter the sorted set of perturbation types before... One perturbation type is used as the target perturbation type.

[0097] In one embodiment of the present invention, the perturbation test question is determined through a large language model based on the target perturbation type, standard scenario information, standard task information, standard answer information set, and corresponding perturbation scenario information in each preset standard test question, including: Based on the target perturbation type and the standard scenario information in each preset standard test question, the perturbation scenario information for each target perturbation type is determined through a large language model; The perturbation test questions are determined using a large language model based on the standard scenario information, standard task information, standard answer information set, and corresponding perturbation scenario information in each preset standard test question.

[0098] In one embodiment of the present invention, the perturbation assessment score for each assessment skill dimension is determined based on the similarity between the answer information for each perturbation test question and the set of perturbation answer information in each perturbation test question, including: The textual similarity of each perturbation test question for each assessment skill dimension is determined based on the similarity between the textual interpretation of the answer information for each perturbation test question and the textual interpretation of the perturbation answer information set for each perturbation test question. Based on the similarity between the image answer information of each assessment skill dimension of each perturbation test question and the image answer information of the perturbation answer information set, the image matching degree of each assessment skill dimension of each perturbation test question is determined. Based on the similarity between the step-by-step answer information of each assessment skill dimension of each perturbation test question and the step-by-step answer information of the perturbation answer information set, the step similarity of each assessment skill dimension of each perturbation test question is determined. The perturbation score for each assessment skill dimension is obtained by weighted summation based on text similarity, image matching degree, step similarity, and corresponding modal weights; wherein, the modal weights are calculated based on the information entropy of text similarity, image matching degree, and step similarity respectively.

[0099] In one embodiment of the present invention, the formula for calculating the disturbance difficulty coefficient is as follows: in, This indicates the difficulty level of the disturbance; 0.1 represents the preset base difficulty value, and 0.8 represents the preset scaling factor. This represents the average of the initial ability scores across all assessment skill dimensions.

[0100] In one embodiment of the present invention, the formula for calculating the comprehensive score of the ability assessment is as follows: in, This indicates the overall score of the ability assessment. Indicates the first The preset weights for each assessment skill dimension. Indicates the first The score reflects the ability to cope with disturbances across various skill dimensions.

[0101] A fourth aspect of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the adaptive skill capability assessment method based on a perturbation mechanism provided by the present invention described above.

[0102] A fifth aspect of the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the adaptive skill ability assessment method based on a perturbation mechanism provided in the above-described embodiments of the present invention.

[0103] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage system located remotely from the aforementioned processor.

[0104] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware systems.

[0105] The method provided in this invention can be applied to electronic devices. Specifically, the electronic device can be a desktop computer, a portable computer, a smart mobile terminal, a server, etc. No limitation is made herein; any electronic device that can implement this invention falls within the protection scope of this invention.

[0106] For system / electronic device embodiments, since they are basically similar to method embodiments, the description is relatively simple, and relevant parts can be found in the description of the method embodiments.

[0107] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A system that specifies functions in one or more boxes.

[0108] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0109] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0110] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. An adaptive skill ability assessment method based on a perturbation mechanism, characterized in that, Includes the following steps: Based on the responses of prospective employees to each pre-set standard test question for each target category in each assessment skill dimension, determine the initial ability score and standard assessment score for each assessment skill dimension. The perturbation difficulty coefficient is determined based on the initial ability score, preset basic difficulty value, and preset scaling factor for each assessment skill dimension. The target number of disturbance types is determined from the disturbance type set based on the disturbance difficulty coefficient, the total number of disturbance types, and the preset disturbance intensity weight; The perturbation test questions are determined using a large language model based on the target perturbation type, the standard scenario information, standard task information, standard answer information set in each preset standard test question, and the corresponding perturbation scenario information. Obtain the answer information for each assessment skill dimension of each perturbation test question, and determine the perturbation assessment score for each assessment skill dimension based on the similarity between the answer information for each assessment skill dimension of each perturbation test question and the perturbation answer information set in each perturbation test question; Based on the standard assessment score and the disturbance assessment score, determine the disturbance response capability score for each assessment skill dimension; The overall ability assessment score and corresponding assessment information are determined based on the disturbance response ability score for each assessment skill dimension and the preset dimension weight for each assessment skill dimension.

2. The method as described in claim 1, characterized in that, The process involves determining the initial ability score and standard assessment score for each assessment skill dimension based on the responses of prospective employees to each preset standard test question for the target product category, including: Based on the responses of prospective interns to each pre-set standard test question for each assessment skill dimension of the target category, the similarity of the standard answer information set for each pre-set standard test question for each assessment skill dimension, and the number of pre-set standard test questions, determine the standard assessment score for each pre-set standard test question and the initial ability score for each assessment skill dimension of all pre-set standard test questions.

3. The method as described in claim 1, characterized in that, The target disturbance types for which the number of target disturbance types is determined from the disturbance type set based on the disturbance difficulty coefficient, the total number of disturbance types, and the preset disturbance intensity weight include: The number of target disturbance types is determined based on the disturbance difficulty coefficient and the total number of disturbance types. ; The perturbation types in the perturbation type set are sorted from high to low according to the preset perturbation intensity weight of the perturbation type; Filter the sorted set of perturbation types before... One perturbation type is used as the target perturbation type.

4. The method as described in claim 1, characterized in that, The step of determining the perturbation test question using a large language model based on the target perturbation type, standard scenario information, standard task information, standard answer information set in each preset standard test question, and the corresponding perturbation scenario information includes: Based on the target perturbation type and the standard scenario information in each preset standard test question, the perturbation scenario information for each target perturbation type is determined using a large language model; The perturbation test question is determined by a large language model based on the standard scenario information, standard task information, standard answer information set, and corresponding perturbation scenario information in each preset standard test question.

5. The method as described in claim 1, characterized in that, The step of determining the perturbation assessment score for each assessment skill dimension based on the similarity between the answer information for each perturbation test question and the perturbation answer information set in each perturbation test question includes: The textual similarity of each perturbation test question for each assessment skill dimension is determined based on the similarity between the textual interpretation of the answer information for each perturbation test question and the textual interpretation of the perturbation answer information set for each perturbation test question. Based on the similarity between the image answer information of each assessment skill dimension of each perturbation test question and the image answer information of the perturbation answer information set, the image matching degree of each assessment skill dimension of each perturbation test question is determined; Based on the similarity between the step-by-step answer information of each assessment skill dimension of each perturbation test question and the step-by-step answer information of the perturbation answer information set, the step similarity of each assessment skill dimension of each perturbation test question is determined. The perturbation score for each assessment skill dimension is obtained by weighted summation based on the text similarity, image matching degree, step similarity, and corresponding modal weights; wherein, the modal weights are calculated based on the information entropy of the text similarity, image matching degree, and step similarity, respectively.

6. The method as described in claim 1, characterized in that, The formula for calculating the difficulty coefficient of the disturbance is: in, This indicates the difficulty level of the disturbance; 0.1 represents the preset base difficulty value, and 0.8 represents the preset scaling factor. This represents the average of the initial ability scores across all assessment skill dimensions.

7. The method as described in claim 1, characterized in that, The formula for calculating the comprehensive score of the ability assessment is as follows: in, This indicates the overall score of the ability assessment. Indicates the first The preset weights for each assessment skill dimension. Indicates the first The score reflects the ability to cope with disturbances across various skill dimensions.

8. An adaptive skill assessment system based on a perturbation mechanism, characterized in that, include: The first determination module is used to determine the initial ability score and standard assessment score for each assessment skill dimension based on the answers of the applicants to each preset standard test question for each target category. The second determining module is used to determine the disturbance difficulty coefficient based on the initial ability score, preset basic difficulty value and preset scaling coefficient of each assessment skill dimension; The third determining module is used to determine the target disturbance type number in the disturbance type set based on the disturbance difficulty coefficient, the total number of disturbance types and the preset disturbance intensity weight; The fourth determining module is used to determine the perturbation test question through a large language model based on the target perturbation type, the standard scenario information, standard task information, standard answer information set in each preset standard test question, and the corresponding perturbation scenario information. The first calculation module is used to obtain the answer information of each assessment skill dimension of each of the perturbation test questions, and determine the perturbation assessment score of each assessment skill dimension based on the similarity between the answer information of each assessment skill dimension of each of the perturbation test questions and the perturbation answer information set in each of the perturbation test questions. The second calculation module is used to determine the disturbance response capability score for each assessment skill dimension based on the standard assessment score and the disturbance assessment score. The evaluation module is used to determine the overall ability evaluation score and corresponding evaluation information based on the disturbance response ability score of each assessment skill dimension and the preset dimension weight of each assessment skill dimension.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the adaptive skill assessment method based on a perturbation mechanism as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the adaptive skill ability assessment method based on the perturbation mechanism as described in any one of claims 1 to 7.

Images