User ability assessment method, medium, and system

Abstract

The application discloses a user ability evaluation method, medium and system, wherein the method comprises the following steps: obtaining original answer data, calculating corresponding score standard deviation, and generating standardized score data; obtaining user answer behavior logs, and calculating user evaluation corresponding redo rate based on the user answer behavior logs; judging whether a fluctuation factor can be constructed; if yes, calculating the fluctuation factor, and generating a normalized vector according to the standardized score data and the fluctuation factor; constructing a fluctuation derivative based on a sliding window behavior fluctuation sequence generation mechanism; judging whether the fluctuation derivative meets dynamic adjustment conditions; if yes, performing dynamic weighted fusion based on the fluctuation derivative and the normalized vector to obtain a final risk score; if not, performing static weighted fusion based on the normalized vector to obtain the final risk score. The user ability can be effectively evaluated, the volatility of the user in the evaluation process can be effectively evaluated, and the accuracy of the evaluation result is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent assessment, and in particular to a user ability assessment method, medium and system. BACKGROUND

[0002] With the development of science and technology, more and more applications are developed to serve the user's ability assessment scenarios (for example, pre-employment ability assessment, education assessment, psychological assessment, etc.).

[0003] In the related art, when assessing the ability of a user, a weighted average based on the score of the user is mostly used. That is, the ability of the user is directly evaluated according to the average value of individual behavior or task score. This method has strong intuitiveness and high calculation efficiency. However, by this method, the volatility of the user assessment process in the time or dimension level is completely ignored, so that potential risk users may not be identified, affecting the accuracy of the assessment results. SUMMARY

[0004] The present application aims to at least partially solve one of the technical problems in the related art. To this end, one object of the present application is to propose a user ability assessment method that can effectively assess the ability of a user and effectively evaluate the volatility of the user in the assessment process, thereby improving the accuracy of the assessment results.

[0005] In a first aspect, the embodiments of the present application propose a user ability assessment method, comprising: obtaining original answer data corresponding to user assessment, and calculating a corresponding score standard deviation based on the original answer data, and generating standardized score data based on the score standard deviation; obtaining a user answer behavior log recorded during user assessment, and calculating a rework rate corresponding to user assessment based on the user answer behavior log; determining whether a volatility factor can be constructed according to the user answer behavior log, the score standard deviation and the rework rate; if yes, calculating the volatility factor based on the standardized score data, and generating a normalized vector according to the standardized score data and the volatility factor; constructing a volatility derivative based on a sliding window behavior volatility sequence generation mechanism; determining whether the volatility derivative meets a dynamic adjustment condition; if yes, performing dynamic weighted fusion based on the volatility derivative and the normalized vector to obtain a final risk score; if no, performing static weighted fusion based on the normalized vector to obtain a final risk score.

[0006] In some embodiments, the standardized score data is generated by the following formula:

[0007]

[0008]

[0009]

[0010] wherein, represents the number of valid questions, represents the question set completed in the user evaluation process, represents the standardized score, represents the original score, represents the mean of the original score, represents the standard deviation of the score.

[0011] In some embodiments, the redo rate is calculated by the following formula:

[0012]

[0013] wherein, represents the redo rate, represents the total number of redo events of all questions in the user evaluation process, represents the number of valid questions.

[0014] In some embodiments, determining whether the fluctuation factor can be constructed according to the user answer behavior log, the score standard deviation and the redo rate includes: determining whether the behavior log is missing; if yes, determining that the fluctuation factor cannot be constructed; if no, determining whether the score standard deviation is greater than or equal to a lower limit value; if the score standard deviation is less than the lower limit value, determining that the fluctuation factor cannot be constructed; if the score standard deviation is greater than or equal to the lower limit value, determining whether the redo rate is greater than a redo rate threshold; if the redo rate is greater than the redo rate threshold, determining that the fluctuation factor cannot be constructed; if the redo rate is less than or equal to the redo rate threshold, determining that the fluctuation factor can be constructed.

[0015] In some embodiments, if the fluctuation factor cannot be constructed, directly normalizing the standardized score data to obtain a corresponding normalization result, and performing calculation based on the normalization result to obtain a final risk score.

[0016] In some embodiments, performing static weighted fusion based on the normalization vector to obtain a final risk score, including:

[0017] performing linear fusion based on the normalization vector to obtain a linear fusion result, and performing nonlinear enhancement on the linear fusion result to obtain a final risk score;

[0018] The normalization vector is expressed by the following formula:

[0019]

[0020] wherein, denotes the normalized vector, denotes the normalization result of the standardized score, denotes the normalization result of the volatility factor;

[0021]

[0022] wherein, denotes the linear fusion result, denotes the weight parameter;

[0023]

[0024] wherein, denotes the final risk score obtained by the nonlinear enhancement, and the parameter denotes the steepness of the nonlinear function, denotes the risk median value.

[0025] In some embodiments, the dynamic weighted fusion is performed based on the following formula to obtain the final risk score:

[0026]

[0027]

[0028]

[0029]

[0030] wherein, denotes the volatility derivative, denotes the timestamp corresponding to the last valid question, denotes the timestamp corresponding to the first valid question, denotes the last window number in the standard deviation sequence, denotes the dynamic weight, denotes the risk response slope mapping coefficient, denotes the dynamic risk index, denotes the final risk score obtained by the dynamic fusion, and the parameter denotes the steepness of the nonlinear function, denotes the risk median value.

[0031] In a second aspect, the embodiments of the present application provide a computer readable storage medium, which stores a user ability evaluation program. When the user ability evaluation program is executed by a processor, the user ability evaluation method described above is implemented.

[0032] In a third aspect, an embodiment of the present application provides a user ability evaluation system, comprising: a preprocessing module, configured to obtain original answer data corresponding to a user evaluation, and calculate a score standard deviation based on the original answer data, and generate standardized score data based on the score standard deviation; a behavior evaluation module, configured to obtain a user answer behavior log recorded during the user evaluation, and calculate a rework rate corresponding to the user evaluation based on the user answer behavior log; a judging module, configured to judge whether a fluctuation factor can be constructed according to the user answer behavior log, the score standard deviation and the rework rate; a calculation module, configured to calculate the fluctuation factor based on the standardized score data when the fluctuation factor can be constructed, and generate a normalized vector according to the standardized score data and the fluctuation factor; a construction module, configured to construct a fluctuation derivative based on a sliding window behavior fluctuation sequence generation mechanism; the judging module is further configured to judge whether the fluctuation derivative meets a dynamic adjustment condition; a dynamic weighting module, configured to perform dynamic weighted fusion based on the fluctuation derivative and the normalized vector to obtain a final risk score when the fluctuation derivative meets the dynamic adjustment condition; and a static weighting module, configured to perform static weighted fusion based on the normalized vector to obtain a final risk score when the fluctuation derivative does not meet the dynamic adjustment condition.

[0033] In some embodiments, the standardized score data is generated by the following formula:

[0034]

[0035]

[0036]

[0037] wherein, represents the number of valid questions, represents a question set completed in the user evaluation process, represents the standardized score, represents the original score, represents the mean of the original score, represents the score standard deviation.

[0038] The present application has the beneficial effect that when evaluating the ability of a user, the user answer behavior log is collected to evaluate the behavior of the user, which can effectively evaluate the user and effectively evaluate the behavior fluctuation of the user, thereby improving the accuracy and reference value of the evaluation result.

[0039] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0040] Figure 1 This is a flowchart illustrating a user capability assessment method according to an embodiment of the present invention;

[0041] Figure 2 This is a block diagram of a user competency assessment system according to an embodiment of the present invention. Detailed Implementation

[0042] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0043] The user capability assessment method of this invention is described below with reference to the accompanying drawings.

[0044] Please see Figure 1 , Figure 1 This is a flowchart illustrating a user competency assessment method according to an embodiment of the present invention, such as... Figure 1 As shown, this user competency assessment method includes the following steps:

[0045] S101: Obtain the original answer data corresponding to the user assessment, calculate the corresponding score standard deviation based on the original answer data, and generate standardized score data based on the score standard deviation.

[0046] In some embodiments, standardized score data is generated using the following formula:

[0047]

[0048]

[0049]

[0050] in, Indicates the number of valid questions. This represents the set of questions completed during the user assessment process. Represents the standardized score. This represents the raw score. This represents the mean of the raw scores. This represents the standard deviation of the score.

[0051] As an example, firstly, user answer records (i.e., raw answer data) are imported from the assessment platform's database; preferably, each user answer record may include: user ID u, question identifier q, and raw score. , Assessment Session ID s, Response Timestamp The maximum score set for the question Dimensional tags of the question Next, field integrity and boundary value checks are performed on each user's answer record to ensure that all numeric fields are valid non-negative real numbers, all time fields conform to the complete time format, and all identifier fields are present and unique. Any user's answer record that does not meet the conditions will be automatically removed and will not participate in further processing.

[0052] After the structured data is processed, each raw score in the original answer data is standardized to eliminate the proportional differences introduced by different question score settings. This standardization process uses Z-score variation, and the calculation formula is as follows:

[0053]

[0054]

[0055]

[0056] in, Indicates the number of valid questions. This represents the set of questions completed during the user assessment process. Represents the standardized score. This represents the raw score. This represents the mean of the raw scores. This represents the standard deviation of the score.

[0057] It is understandable that, through the above formula, each raw score can be mapped to a standard control centered at zero with a standard deviation of one, for use in the subsequent construction of mean and variance indicators.

[0058] S102, obtain the user's answer behavior log recorded during the user evaluation, and calculate the redo rate corresponding to the user evaluation based on the user's answer behavior log.

[0059] In some embodiments, the redo rate is calculated using the following formula:

[0060]

[0061] in, This indicates the redo rate. This represents the total number of redo events for all questions during the user evaluation process. Indicates the number of valid questions.

[0062] As an example, first, the user behavior log is read from the behavior tracking system. Preferably, each user behavior log records all interaction events during the user's answering process, including page loading, starting answering, submission action, redo behavior, etc. Each log record contains event type , event trigger timestamp , user number u, assessment session number s, event associated question identifier q. Next, the log sequence of a unified user in a single session is arranged in ascending order of time, and its behavior trajectory is constructed. Then, the trajectory is analyzed by time difference, and the interval time of continuous events is calculated . If the time interval is less than the minimum event effective threshold , the system determines that it is repeated triggering and merges; if it is greater than or equal to the maximum allowed dwell time , it is split into independent sessions to prevent behavior drift from interfering with the assessment structure.

[0063] Next, in order to support the explicit modeling of behavior volatility, first, the volatility label field and the behavior stability signal variable are generated; the system counts the number of behavior events triggered by a unified user on the same question, extracts the "repeat answer" count, and calculates the behavior redo rate of the user session based on it:

[0064]

[0065] Among them, represents the redo rate, represents the total number of redo events of all questions in the user's assessment process, represents the number of valid questions.

[0066] As an example, a threshold judgment mechanism can be introduced: if the session meets the following two conditions (that is, the global standard deviation exceeds the stability threshold); and (that is, the redo rate exceeds the behavior anomaly threshold), the session is labeled as behaviorally unstable, and the label field is set to .

[0067] Finally, the system encapsulates each user's standardized score sequence , session score mean , standard deviation , redo rate , volatility label , session identifier and timestamp index in a structured data table as the only standard input source for subsequent risk index modeling process.

[0068] S103, determining whether the fluctuation factor can be constructed according to the user answer behavior log, the score standard deviation and the redo rate.

[0069] In some embodiments, determining whether the fluctuation factor can be constructed according to the user answer behavior log, the score standard deviation and the redo rate comprises: determining whether the behavior log is missing; if yes, determining that the fluctuation factor cannot be constructed; if no, determining whether the score standard deviation is greater than or equal to a lower limit value; if the score standard deviation is less than the lower limit value, determining that the fluctuation factor cannot be constructed; if the score standard deviation is greater than or equal to the lower limit value, determining whether the redo rate is greater than a redo rate threshold; if the redo rate is greater than the redo rate threshold, determining that the fluctuation factor cannot be constructed; if the redo rate is less than or equal to the redo rate threshold, determining that the fluctuation factor can be constructed.

[0070] That is, considering that the user answer behavior log may be missing or incomplete in part of the evaluation tasks, the system introduces a "behavior missing compensation mechanism" in the fluctuation label generation process. That is, when the user answer behavior log is missing or the number of events is insufficient to form an effective trajectory, the fluctuation index is no longer forcibly constructed; at the same time, if the user's answer standard deviation is in an extreme position in the distribution of all users, the system determines that the fluctuation factor cannot be constructed.

[0071] S104, if yes, calculating the fluctuation factor based on the standardized score data, and generating a normalized vector according to the standardized score data and the fluctuation factor.

[0072] That is, in order to eliminate the scale bias between users, the standardized score data and the fluctuation factor are normalized to obtain the corresponding normalized vector.

[0073] As an example, the mean index of the standardized score of each user in a single evaluation session and the fluctuation factor are calculated, and uniform normalization processing is performed on the two factors. Specifically, first, the score sequence of all the questions completed by each user in a complete evaluation session is aggregated to calculate the standardized ability mean index . The formula is defined as follows:

[0074]

[0075] wherein, the standardized ability mean index is denoted by the standardized score is denoted by the number of valid questions is denoted by the set of questions completed by the user in the evaluation process is denoted by, and the mean index is used to measure the overall ability performance of the user under a unified standard, and is one of the dominant factors in the risk index structure for measuring performance.

[0076] ​Then, the standard deviation of the answer score sequence of the same user in the session is calculated to obtain the behavior stability index (i.e. fluctuation factor). The formula is as follows:

[0077]

[0078] In the formula, represents the variance value of the normalized score sequence, representing the degree of user behavior fluctuation in the current evaluation. Its calculation method is based on the square value of the deviation between the normalized score of each question and the normalized mean value of all questions, and the average is taken. The variable and have the same meaning as described above. The index is one of the core structure variables in the risk score logic of the application, and the larger the value is, the greater the fluctuation in the user's answering process, the worse the behavior stability, and the higher the risk value should be.

[0079] Then, to realize the comparability of the risk input index between different users, the system normalizes the above two indexes. The normalization processing uses the maximum and minimum values of all samples as the reference interval, and the values are uniformly mapped to the [0, 1] interval. The normalization processing formula is as follows:

[0080]

[0081]

[0082] wherein, denotes the normalized result of the normalized score, denotes the normalized result of the fluctuation factor; denotes the set of ability mean values of all users in the evaluation task in the evaluation system, denotes the set of fluctuation indexes of all users; and respectively represent the minimum value and the maximum value in the set, which are used to construct the upper and lower limits of the interval of linear normalization mapping. This normalization mechanism ensures that the model has a unified numerical scale when processing user data under different distribution conditions, avoiding bias caused by external factors such as evaluation batch, question difficulty, etc. on risk calculation.

[0083] Finally, the system combines the two normalized indexes to construct an input vector:

[0084]

[0085] The vector is the main input variable of the risk index model, and the first dimension represents the ability performance and the second dimension represents the stability degree. The structured tensor is stored in the risk sample table to provide input for subsequent steps of collaborative risk modeling.

[0086] In some embodiments, if the volatility factor is not constructible, the normalized score data is directly normalized to obtain a corresponding normalization result, and a calculation is performed based on the normalization result to obtain a final risk score.

[0087] That is, if the volatility factor is not constructible, the ability factor path is completely fallbacked to.

[0088] As an example to ensure that the risk index model always has stable output capability under various input scenarios, the system first performs integrity verification on the normalized input vector before performing index construction. If the system detects that the user has a structural missing behavior indicator in the current evaluation session, which is manifested in that the number of behavior log events of the user is less than the behavior judgment threshold set by the system , or the score standard deviation of the user is less than the lower limit of stability judgment , the system will determine that the behavior volatility indicator of the current user does not have model applicability.

[0089] Specifically, at this time, the ability factor is constructed by the following formula:

[0090]

[0091] wherein, represents the final risk score under the condition that the volatility factor is not constructible, represents the normalization result of the normalized score, which uses the ability factor to score alone. This mechanism ensures that the model does not produce breaks and trigger abnormal calculation behavior when the input condition is incomplete, constituting a fault-tolerant control closed loop in the index modeling link of the present application.

[0092] S105, a behavior volatility sequence generation mechanism based on a sliding window is used to construct a volatility derivative.

[0093] As an example, in order to ensure that the volatility derivative structure has computability, stability and structure expression ability, the system uses a behavior volatility sequence generation mechanism based on a sliding window to construct a volatility derivative. Specifically, first, the continuous answer record sequence of the user in the current session is extracted, combined with the timestamp of each question, and a sliding window set with a window size of is constructed in the answer order. The system calculates the standard deviation on each window to obtain the trajectory of the behavior volatility sequence distribution over time.

[0094] Then, on the basis of the sequence, the system calculates the overall trend of volatility, to form a first derivative approximation (volatility derivative) by dividing the difference between the standard deviations at the beginning and end by the total time span of the user's responses:

[0095]

[0096] The volatility derivative depicts the evolving trend of behavior stability, with a positive value indicating that the user's behavior volatility gradually increases, and a negative value indicating that the volatility tends to be stable. The system only enables the dynamic adjustment mechanism when the derivative is positive, and if the derivative is negative or undefined (e.g., missing timestamps or too few response samples), the behavior risk is not considered to have increased, and the system will fall back to the static path (i.e., perform static weighted fusion).

[0097] It should be noted that while the system is executing the fallback path, it will record the source field of the risk score for the sample and set the flag bit , which is used by subsequent modules (including reporting systems, audit interfaces, or supervision models) to track the actual path decision source of the risk model. This mechanism ensures that the model does not produce breaks or trigger abnormal calculation behavior when the input conditions are incomplete, and constitutes a fault-tolerant control closed loop in the exponential modeling link.

[0098] S106, determine whether the volatility derivative meets the dynamic adjustment condition.

[0099] S107, if yes, perform dynamic weighted fusion based on the volatility derivative and the normalized vector to obtain the final risk score.

[0100] S108, if no, perform static weighted fusion based on the normalized vector to obtain the final risk score.

[0101] In some embodiments, performing static weighted fusion based on the normalized vector to obtain the final risk score includes:

[0102] performing linear fusion based on the normalized vector to obtain a linear fusion result, and performing nonlinear enhancement on the linear fusion result to obtain the final risk score;

[0103] The normalized vector is represented by the following formula:

[0104]

[0105] wherein, the normalized vector is represented by the normalized result of the standardized score is represented by the normalized result of the volatility factor is represented by

[0106]

[0107] wherein, This represents the result of linear fusion. Indicates the weighting parameter;

[0108]

[0109] in, The final risk score obtained by nonlinear enhancement is represented by the parameter. Indicates the steepness of a nonlinear function. This represents the median risk value.

[0110] In other words, when volatility factors can be constructed but volatility derivatives cannot, static weighted fusion based on normalized vectors is used to obtain the final risk score.

[0111] As an example, firstly, to regulate the degree of influence of the two types of factors, the system introduces configurable weighting coefficients into the main structure of the risk index. The linear fusion results for constructing the initial risk index are as follows:

[0112]

[0113] in, This represents the result of linear fusion. Indicates the weighting parameter; This represents the normalized result of the standardized score. This represents the normalized result of the volatility factor, where the inverse value of the capability factor is expressed in the expression. This is used to illustrate the relationship that "the higher the capability, the lower the risk," while volatility factor It directly represents its contribution to stability risk. Parameter Controlling the relative weights of the two in the overall risk, when When the value approaches 0, the model focuses more on capability factors; when... When the value approaches 1, the risk output is primarily driven by behavioral volatility. This weighting parameter can be configured based on the sensitivity of different assessment tasks, and can be set uniformly through the backend strategy system or injected in real time, supporting scenario adaptation.

[0114] Next, to enhance the model's ability to identify samples in high-risk intervals, a nonlinear risk response function was used to adjust the original risk index. Mapping is performed to generate a risk index. The calculation formula is as follows:

[0115]

[0116] in, This represents the final risk score obtained through nonlinear enhancement, with values ​​still ranging from [0,1]. Parameters represents the steepness of the nonlinear function, controls the response intensity of the model to samples near the threshold; is the median value of risk, preferably, when , When the original risk score exceeds , the output risk value will accelerate growth, thereby expanding the distribution gap between high-risk samples and improving the model's identification resolution in critical risk sections. This Sigmoid gain function has the advantages of strong interpretability, stable gradient, and differentiability.

[0117] It should be noted that when the system executes the static scoring path, in order to enhance the clarity and interpretability of the scoring structure, a static fusion score variable is explicitly constructed. This variable preserves the linear combination result of the original ability factor and the volatility factor before the final score is generated, and is used for interpretation and analysis, parameter adjustment visualization, model fusion expansion, and path consistency design. Through this variable, the system can achieve unified modeling of the static path and dynamic path scoring structures, improving the stability and transparency of the overall model.

[0118] In some embodiments, dynamic weighted fusion is performed based on the following formula to obtain the final risk score:

[0119]

[0120]

[0121]

[0122]

[0123] wherein, represents the volatility derivative, represents the timestamp corresponding to the last valid question, represents the timestamp corresponding to the first valid question, i.e., the window start time in the behavior trajectory, represents the last window number in the standard deviation sequence, represents the dynamic weight, represents the risk response slope mapping coefficient, represents the dynamic risk index, represents the final risk score obtained by dynamic fusion, and the parameter represents the steepness of the nonlinear function, represents the median value of risk.

[0124] As an example, first, to enhance the dynamic expression ability of the risk model under the influence of behavior volatility, the system first constructs a user behavior derivative-driven dynamic weight coefficient (i.e., dynamic weight). The dynamic weight The intensity response representing the fluctuation trend of user behavior in the evaluation session s is specifically constructed as follows:

[0125]

[0126] wherein, is the risk response slope mapping coefficient, represents the fluctuation derivative, which is used to represent the change trend of user behavior fluctuation in the evaluation process, and is the core index for constructing the dynamic weight adjustment mechanism.

[0127] To ensure that the fluctuation derivative structure has calculability, stability and structure expression ability, the system uses a behavior fluctuation sequence generation mechanism based on a sliding window. The specific process is as follows: the system first extracts the continuous answer record sequence of the user in the current session , combines the time stamp of each question , and constructs a sliding window set with a window size of in the answer order . The system calculates the standard deviation on each window to obtain the trajectory of the time sequence distribution of the behavior fluctuation .

[0128] On the basis of the sequence, the system calculates the overall change trend of the fluctuation, and forms the fluctuation derivative by dividing the difference between the head and tail standard deviations by the total time span of the user's answers:

[0129]

[0130] The fluctuation derivative describes the evolution trend of behavior stability, and a positive value indicates that the user's behavior fluctuation gradually increases, and a negative value indicates that the fluctuation tends to be stable. The system only enables the dynamic adjustment mechanism when the derivative is positive, and if the fluctuation derivative is negative or undefined (such as missing time stamp or too few answer samples), it is considered that the behavior risk has not increased, and the system will fall back to the static path.

[0131] To avoid dramatic disturbance of the weight caused by abnormal samples, the system sets a maximum derivative response threshold , and when the fluctuation derivative calculation result exceeds the upper limit, it is truncated to to ensure that the risk weight control mechanism is not activated by outlier behavior. The final generated dynamic weight will be used to replace the static weight in step three, entering the dynamic fusion path.

[0132] Based on this dynamic weight, the system reconstructs the risk fusion structure as follows:

[0133]

[0134] In the above formula, Dynamic risk index, which combines the contribution of ability and volatility factors and adjusts its structure weight according to the trend of behavior change. The structure supports the sensitive response of the risk index to the behavior volatility, so that the model has the ability to identify and dynamically adjust the individual behavior variation.

[0135] Subsequently, the system sends the dynamic fusion result to the risk enhancement function to generate the final risk score:

[0136]

[0137] Wherein, is a risk slope control parameter, used to adjust the steepness of the function response; is a risk critical center point. When , the output value will rise significantly, thereby enhancing the distinguishability between high-risk samples.

[0138] In some embodiments, the final risk score can be The segmented level mapping function is designed as follows:

[0139]

[0140] The level label will be attached to the risk output record as the decision input for the downstream recommendation, report and matching strategy module. The upper threshold and interval can be uniformly set in the system deployment configuration, and differentiated configuration according to the type of assessment task is also supported.

[0141] In the output result writing and disk writing stage, the system will record the following core fields to build a structured output package (Risk Output Record):

[0142] · : the final risk score value, normalized and enhanced;

[0143] · : the enhanced pre-score result constructed in step three, used for subsequent explainability display;

[0144] · : the fusion weight value generated by the behavior derivative, representing the contribution proportion of the ability and volatility factors in the current session;

[0145] · : whether the risk fusion path uses a static ability score fallback flag, 1 indicating that the fallback path is enabled;

[0146] · : whether the dynamic weight calculation is disabled, whether to use the default static weight ;

[0147] · : Risk level label, used to match recommendation strategy with report presentation;

[0148] · : Timestamp field of score generation, used for system log record and task chain traceability.

[0149] The system writes this structured output record into a cache database for real-time recommendation module to pull, and synchronously writes into persistent storage (such as relational database score table or object storage structure log) for supporting subsequent evaluation report generation, behavior trajectory visualization, model debugging and user historical score comparison.

[0150] When the risk index output module encounters one of the following abnormal situations in operation:

[0151] · Behavior derivative is not derivable, behavior sequence data is missing;

[0152] · Nonlinear function parameter configuration error causes score to be not calculable;

[0153] · Grade threshold interval configuration is missing or conflicting;

[0154] The system will automatically call the bottomed-out fallback logic and set the risk score as:

[0155]

[0156] At the same time, record the fallback path marker field In order to quickly identify and locate the problem source in the log and exception detection module.

[0157] This step is the final link in the risk index generation process in the present application, which not only guarantees the normalization and interpretability of the score structure, but also realizes the complete encapsulation of the task logic, score level and structure field, so that the system has high stability, accessibility and general score expression ability for multi-module linkage.

[0158] In summary, the user ability assessment method according to embodiments of the present invention obtains the original answer data corresponding to the user assessment, calculates the corresponding score standard deviation based on the original answer data, and generates standardized score data based on the score standard deviation; obtains user answer behavior logs recorded during the user assessment, and calculates the redo rate corresponding to the user assessment based on the user answer behavior logs; determines whether a volatility factor can be constructed based on the user answer behavior logs, the score standard deviation, and the redo rate; if so, calculates the volatility factor based on the standardized score data, and generates a normalized vector based on the standardized score data and the volatility factor; constructs a volatility derivative based on a sliding window behavior volatility sequence generation mechanism; determines whether the volatility derivative meets the dynamic adjustment conditions; if so, performs dynamic weighted fusion based on the volatility derivative and the normalized vector to obtain a final risk score; if not, performs static weighted fusion based on the normalized vector to obtain a final risk score, thereby achieving effective assessment of user ability and effective evaluation of user volatility during the assessment process, improving the accuracy of assessment results.

[0159] Secondly, embodiments of the present invention provide a computer-readable storage medium storing a user competency assessment program thereon, which, when executed by a processor, implements the user competency assessment method as described above.

[0160] Thirdly, embodiments of the present invention propose a user capability assessment system, such as... Figure 2 As shown, the user competence assessment system includes: a preprocessing module 10, a behavior assessment module 20, a judgment module 30, a calculation module 40, a construction module 50, a dynamic weighting module 60, and a static weighting module 70.

[0161] The preprocessing module 10 is used to obtain the original answer data corresponding to the user assessment, calculate the corresponding score standard deviation based on the original answer data, and generate standardized score data based on the score standard deviation.

[0162] The behavior assessment module 20 is used to obtain user answer behavior logs recorded during user assessments and to calculate the redo rate of user assessments based on the user answer behavior logs.

[0163] The judgment module 30 is used to determine whether the fluctuation factor can be constructed based on the user's answer behavior log, score standard deviation and rework rate;

[0164] The calculation module 40 is used to calculate the volatility factor based on the standardized score data when the volatility factor is constructible, and to generate a normalized vector based on the standardized score data and the volatility factor.

[0165] Module 50 is used to construct the volatility derivative based on the sliding window-based behavior volatility sequence generation mechanism;

[0166] The determining module 30 is further configured to determine whether the fluctuation derivative meets a dynamic adjustment condition.

[0167] The dynamic weighting module 60 is configured to, when the fluctuation derivative meets the dynamic adjustment condition, perform dynamic weighted fusion based on the fluctuation derivative and the normalized vector to obtain a final risk score.

[0168] The static weighting module 70 is configured to, when the fluctuation derivative does not meet the dynamic adjustment condition, perform static weighted fusion based on the normalized vector to obtain the final risk score.

[0169] In some embodiments, the standardized score data is generated by the following formula:

[0170]

[0171]

[0172]

[0173] wherein, represents the number of valid questions, represents a set of questions completed by the user in the evaluation process, represents the standardized score, represents the original score, represents the mean of the original score, represents the standard deviation of the score.

[0174] It should be noted that the above description of the user ability evaluation method is also applicable to the user ability evaluation system, which will not be repeated here.

[0175] It is to be appreciated that the above description and the examples that follow are intended to be illustrative only and that changes can be made to the description and examples without departing from the scope of the application. Note also that the use of particular brand names in the description is solely for illustration and should not be construed as an endorsement of such brands.

[0176] It should be understood that aspects of the application can be implemented in hardware, software, firmware or a combination thereof. In the above embodiments, various steps or methods can be implemented in software or firmware that is stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any of the following technologies, known in the art, can be used: a hybrid of the technologies mentioned above, discrete logic circuitry having logic gates for implementing logic functions upon an application of data signals, application specific integrated circuits having appropriate combinational logic gates, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0177] In the description of the present application, reference has been made to the use of terms such as "one embodiment", "some embodiments", "an example", "a specific example" or "some examples" means that a particular feature, structure, material or characteristic is included in at least one embodiment or example of the present application. The illustrative examples given are not necessarily to be construed as preferred or advantageous or with the exclusion of other equally valid examples that can be particularly adapted to a given application. Moreover, such illustrative examples are not necessarily mutually exclusive as the various aspects of the application can be combined in any suitable manner. Furthermore, the illustrative examples given are not necessarily mutually exclusive as the various aspects of the application can be combined in any suitable manner.

[0178] In the description of the application, it should be understood that the orientation or positional relationship indicated by the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the application and simplifying the description, and does not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the application.

[0179] In addition, the terms "first", "second" are only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the technical features indicated. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In the description of the application, the meaning of "a plurality of" is at least two, such as two, three, etc., unless otherwise explicitly specified and limited.

[0180] In the present application, unless otherwise explicitly specified and limited, the terms "mounting", "connecting", "connecting", "fixing" and the like should be broadly understood, for example, it can be fixedly connected, or it can be detachably connected, or it can be integrated; it can be mechanically connected, or it can be electrically connected; it can be directly connected, or it can be indirectly connected through an intermediate medium; it can be the internal communication of two elements or the interaction relationship between two elements, unless otherwise explicitly limited. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0181] In the present application, unless otherwise explicitly specified and limited, the first feature is "on" or "under" the second feature, which can be direct contact between the first and second features, or indirect contact between the first and second features through an intermediate medium. Moreover, the first feature "above", "above" and "above" the second feature can be directly above or obliquely above the first feature, or only indicate that the horizontal height of the first feature is higher than that of the second feature. The first feature "below", "below" and "below" the second feature can be directly below or obliquely below the first feature, or only indicate that the horizontal height of the first feature is less than that of the second feature.

[0182] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as a limitation on the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above embodiments within the scope of the present application.

Claims

1. A method for measuring user ability, characterized by, The method comprises the following steps: obtaining original answer data corresponding to a user assessment, and calculating a score standard deviation corresponding to the original answer data based on the original answer data, and generating standardized score data based on the score standard deviation; obtaining a user answer behavior log recorded during the user assessment, and calculating a rework rate corresponding to the user assessment based on the user answer behavior log; determining whether a fluctuation factor can be constructed according to the user answer behavior log, the score standard deviation, and the rework rate; if yes, calculating the fluctuation factor based on the standardized score data, and generating a normalized vector according to the standardized score data and the fluctuation factor; constructing a fluctuation derivative based on a sliding window behavior fluctuation sequence generation mechanism; determining whether the fluctuation derivative meets a dynamic adjustment condition; if yes, performing dynamic weighted fusion based on the fluctuation derivative and the normalized vector to obtain a final risk score; if no, performing static weighted fusion based on the normalized vector to obtain a final risk score; wherein the static weighted fusion based on the normalized vector to obtain a final risk score comprises: performing linear fusion based on the normalized vector to obtain a linear fusion result, and performing nonlinear enhancement on the linear fusion result to obtain a final risk score; the normalized vector is expressed by the following formula: ； wherein, denotes a normalized vector, denotes a normalization of the standardized score, denotes a normalization of the volatility factor; ； wherein, represents a linear fusion result, represents a weight parameter; ； wherein, represents the final risk score resulting from the non-linear enhancement, the parameters represents the steepness of the non-linear function, represents the median value of the risk; performing dynamic weighted fusion based on the following formula to obtain a final risk score: ； ； ； ； wherein, denotes the volatility derivative, denotes the timestamp corresponding to the last valid question, denotes the timestamp corresponding to the first valid question, denotes the last window number in the standard deviation sequence, denotes the dynamic weight, denotes the risk response slope mapping coefficient, denotes the dynamic risk index, denotes the final risk score obtained by dynamic fusion, parameter denotes the steepness of the nonlinear function, denotes the risk median value.

2. The user ability assessment method of claim 1, wherein, the standardized score data is generated by the following formula: ； ； ； wherein, represents the number of valid items, represents the item identifier, represents the set of items completed by the user during the assessment process, represents the normalized score, represents the raw score, represents the mean of the raw scores, represents the standard deviation of the scores.

3. The user ability assessment method of claim 1, wherein, the rework rate is calculated by the following formula: ； wherein, represents the redo rate, represents the total number of redo events of all questions in the user evaluation process, represents the number of valid questions.

4. The user ability assessment method of claim 1, wherein, determining whether a fluctuation factor can be constructed according to the user answer behavior log, the score standard deviation, and the rework rate comprises: determining whether the behavior log is missing; if yes, determining that the fluctuation factor cannot be constructed; if no, determining whether the score standard deviation is greater than or equal to a lower limit value; if the score standard deviation is less than the lower limit value, determining that the fluctuation factor cannot be constructed; if the score standard deviation is greater than or equal to the lower limit value, determining whether the rework rate is greater than a rework rate threshold; if the rework rate is greater than the rework rate threshold, determining that the fluctuation factor cannot be constructed; if the rework rate is less than or equal to the rework rate threshold, determining that the fluctuation factor can be constructed.

5. The user ability assessment method of claim 1, wherein, if the fluctuation factor cannot be constructed, directly normalizing the standardized score data to obtain a corresponding normalized result, and calculating based on the normalized result to obtain a final risk score.

6. A computer-readable storage medium, characterized in that, A user ability assessment program is stored thereon, and the user ability assessment program is executed by a processor to implement the user ability assessment method of any one of claims 1-5.

7. A user ability assessment system, characterized by, comprises: a preprocessing module configured to obtain original answer data corresponding to a user assessment, and calculate a score standard deviation corresponding to the original answer data based on the original answer data, and generate standardized score data based on the score standard deviation; a behavior assessment module configured to obtain a user answer behavior log recorded during the user assessment, and calculate a rework rate corresponding to the user assessment based on the user answer behavior log; A judgment module is configured to judge whether a fluctuation factor can be constructed according to the user answering behavior log, the score standard deviation and the redo rate; A calculation module is configured to calculate the fluctuation factor based on the standardized score data when the fluctuation factor can be constructed, and generate a normalized vector according to the standardized score data and the fluctuation factor; A construction module is configured to construct a fluctuation derivative based on a behavior fluctuation sequence of a sliding window; The judgment module is further configured to judge whether the fluctuation derivative meets a dynamic adjustment condition; A dynamic weighting module is configured to perform dynamic weighted fusion based on the fluctuation derivative and the normalized vector to obtain a final risk score when the fluctuation derivative meets the dynamic adjustment condition; A static weighting module is configured to perform static weighted fusion based on the normalized vector to obtain a final risk score when the fluctuation derivative does not meet the dynamic adjustment condition; The static weighted fusion based on the normalized vector to obtain a final risk score comprises: performing linear fusion based on the normalized vector to obtain a linear fusion result, and performing nonlinear enhancement on the linear fusion result to obtain a final risk score; The normalized vector is expressed by the following formula: ； wherein, denotes a normalized vector, denotes a normalization of the standardized score, denotes a normalization of the volatility factor; ； wherein, represents a linear fusion result, represents a weight parameter; ； wherein, represents the final risk score resulting from the non-linear enhancement, the parameters represents the steepness of the non-linear function, represents the median value of the risk; The dynamic weighted fusion based on the following formula to obtain a final risk score: ； ； ； ； wherein, denotes the volatility derivative, denotes the timestamp corresponding to the last valid question, denotes the timestamp corresponding to the first valid question, denotes the last window number in the standard deviation sequence, denotes the dynamic weight, denotes the risk response slope mapping coefficient, denotes the dynamic risk index, denotes the final risk score obtained by dynamic fusion, parameter denotes the steepness of the nonlinear function, denotes the risk median value.

8. The user ability assessment system of claim 7, wherein, The standardized score data is generated by the following formula: ； ； ； wherein, represents the number of valid items, represents the item identifier, represents the set of items completed by the user during the assessment process, represents the standardized score, represents the raw score, represents the mean of the raw scores, represents the standard deviation of the scores.

Images