Target object training ability evaluation method and device

Abstract

This application provides a method for assessing the training ability of a target audience, applicable to the field of artificial intelligence. The method includes: obtaining a first score for the target audience based on their answers in a business training system and the knowledge points in the knowledge graph at the current moment; analyzing the behavioral characteristics of the target audience during the answering process using a time-series analysis model to obtain a second score and a guessing coefficient, where the guessing coefficient characterizes the degree of guessing in the target audience's answers; correcting the second score based on the guessing coefficient to obtain a corrected second score; obtaining a comprehensive score using a gating fusion mechanism based on the first score, the guessing coefficient, and the corrected second score; and obtaining the training ability level corresponding to the comprehensive score from a preset training ability level table, using the training ability level as the assessment result of the target audience's training ability. This application also provides a device, equipment, storage medium, and program product for assessing the training ability of a target audience.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and more specifically to a method, apparatus, device, medium, and program product for assessing the training capabilities of a target audience. Background Technology

[0002] The assessment of training capabilities of target subjects in general training systems has the following defects: (1) Misjudgment of explicit capabilities: General training systems rely on standardized tests and cannot identify whether the target subject has engaged in "guessing answers". "Guessing answers" will lead to inflated training scores of the target subject, which will affect the allocation of subsequent training programs for the target subject; (2) Blind spots in implicit capabilities: General training systems lack in-depth analysis of the cognition reflected in the target subject's answering process. For example, the high-frequency correction behavior of tellers in the "cross-border remittance declaration" module is not associated with relevant knowledge deficiencies, which will lead to inaccurate assessment of the target subject's training capabilities; (3) Single assessment dimension: It is difficult to quantify, for example, "speculative learning" such as temporarily looking up answers to complete tasks. Even with the use of existing assessment tools, it is impossible to effectively detect the target subject's guessing behavior, which will lead to incorrect assessment of the target subject's training capabilities and thus affect the allocation of training resources. Summary of the Invention

[0003] In view of the above problems, this application provides target training competence assessment methods, apparatus, equipment, media and procedures to improve the accuracy of target training competence assessment.

[0004] According to the first aspect of this application, a method for assessing the training ability of a target audience is provided, comprising: obtaining a first score for the target audience based on their answers in a business training system and the knowledge points in the knowledge graph at the current moment; analyzing the behavioral characteristics of the target audience during the answering process using a time-series analysis model to obtain a second score and a guessing coefficient for the target audience; wherein the guessing coefficient represents the degree of guessing in the target audience's answers; correcting the second score based on the guessing coefficient to obtain a corrected second score; obtaining a comprehensive score using a gating fusion mechanism based on the first score, the guessing coefficient, and the corrected second score; obtaining the training ability level corresponding to the comprehensive score from a preset training ability level table, and using the training ability level as the assessment result of the target audience's training ability.

[0005] According to an embodiment of this application, obtaining a first score for a target object based on the target object's answer results in the business training system and the knowledge points in the knowledge graph at the current moment includes: generating test questions corresponding to the business scenario in the business training system; calculating the similarity between the target object's answer results and the knowledge points corresponding to the test questions in the knowledge graph at the current moment; and obtaining a first score from a preset first score table based on the similarity; wherein the preset first score table is obtained using a machine learning algorithm based on historical answer data.

[0006] According to embodiments of this application, a knowledge graph is constructed in the following manner: domain knowledge is classified according to business type; entity recognition is performed on knowledge points in each business type to obtain entity relationships between knowledge points; each case in the case library is decomposed according to preset decomposition elements; wherein the preset decomposition elements include business objectives, data sources, data analysis methods, and conclusions; the decomposed data is associated with entity relationships to obtain a knowledge graph; wherein the knowledge graph is continuously updated through a knowledge graph incremental update algorithm.

[0007] According to embodiments of this application, a temporal analysis model is used to analyze the behavioral characteristics of a target object during the question-answering process to obtain a second score and a guessing coefficient for the target object. This includes: extracting temporal and global features of the behavioral characteristics using a temporal analysis model; wherein the temporal features represent local changes in the behavioral characteristics within a first preset time range, and the global features represent overall changes in the behavioral characteristics within a second preset time range, where the first preset time range is shorter than the second preset time range; processing the temporal features using a first sub-network in a hybrid neural network to obtain temporal feature values; wherein the temporal feature values ​​represent statistical values ​​of the temporal features; processing the global features using a second sub-network in a hybrid neural network to obtain global feature values; wherein the global feature values ​​represent statistical values ​​of the global features; fusing the temporal feature values ​​and global feature values ​​through a fusion layer of the hybrid neural network to obtain a second score; and calculating the guessing coefficient based on the difference between the temporal feature values ​​and a first preset threshold corresponding to the temporal feature values, and the difference between the global feature values ​​and a second preset threshold corresponding to the global feature values.

[0008] According to an embodiment of this application, the second score is corrected based on the guessing coefficient, including: correcting the second score based on the guessing coefficient, the attenuation coefficient of the guessing coefficient relative to the second score, the risk coefficient, and the violation operation parameter; wherein, the attenuation coefficient represents the degree of influence of the guessing coefficient on the second score, the risk coefficient represents the degree of violation risk of the customer data involved in the target object's answer, and the violation operation parameter represents the degree of violation operation generated by the target object during the answering process.

[0009] According to an embodiment of this application, the method further includes: obtaining a subsequent training strategy from a preset training strategy table based on the training ability level; wherein the preset training strategy table is obtained by statistical analysis of historical training strategies and the historical training performance of each target object; generating a new training strategy using a greedy strategy based on the subsequent training strategy; predicting the reward values ​​of the subsequent training strategy and the new training strategy respectively through a hierarchical decision network, and selecting the training strategy with the highest reward value as the final training strategy; training the target object according to the final training strategy, and re-obtaining the target object's first score, the corrected second score, and the guessing coefficient; repeating the operations of obtaining the training ability level, obtaining the final training strategy, and obtaining the first score, the corrected second score, and the guessing coefficient until the target object's first score, the corrected second score, and the guessing coefficient reach their respective preset scores.

[0010] According to an embodiment of this application, a hierarchical decision network is used to predict the reward values ​​of subsequent training strategies and new training strategies, respectively. This includes: constructing a state vector at the current moment based on the target object's first score, the corrected second score, the guessing coefficient, and the knowledge graph at the current moment; and inputting the state vector at the current moment into the hierarchical decision network so that the hierarchical decision network can predict the reward values ​​of subsequent training strategies and new training strategies under the state vector at the current moment based on the knowledge points in the knowledge graph at the current moment whose relevance to the current business scenario meets a preset relevance threshold.

[0011] A second aspect of this application provides a device for assessing the training capacity of a target audience, comprising:

[0012] The explicit score acquisition module is used to obtain the target's first score based on the target's answer results in the business training system and the knowledge points in the knowledge graph at the current moment;

[0013] The implicit score acquisition module is used to analyze the behavioral characteristics of the target object during the answering process using a time series analysis model to obtain the target object's second score and guessing coefficient; whereby the guessing coefficient represents the degree of guessing in the target object's answer.

[0014] The implicit score correction module is used to correct the second score based on the guessed coefficients, thus obtaining the corrected second score.

[0015] The comprehensive score acquisition module is used to obtain a comprehensive score based on the first score, the guessed coefficient, and the corrected second score using a gating fusion mechanism.

[0016] The training capability assessment module is used to obtain the training capability level corresponding to the comprehensive score from a preset training capability level table, and use the training capability level as the assessment result of the target object's training capability.

[0017] A third aspect of this application provides an electronic device comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method described above.

[0018] A fourth aspect of this application also provides a computer-readable storage medium having a computer program or instructions stored thereon, which, when executed by a processor, implement the steps of the above-described method.

[0019] The fifth aspect of this application also provides a computer program product, including a computer program or instructions that, when executed by a processor, implement the steps of the above-described method. Attached Figure Description

[0020] The above-mentioned contents, other objects, features and advantages of this application will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0021] Figure 1 The illustration schematically depicts an application scenario of the target training capability assessment method, apparatus, device, medium, and program product according to embodiments of this application;

[0022] Figure 2 A flowchart illustrating a method for assessing the training capabilities of a target audience according to an embodiment of this application is shown schematically.

[0023] Figure 3 A flowchart illustrating the first score acquisition process according to an embodiment of this application is shown schematically.

[0024] Figure 4 A flowchart illustrating the knowledge graph creation process according to an embodiment of this application is shown schematically.

[0025] Figure 5 A flowchart illustrating the second score and guessing coefficient acquisition process according to an embodiment of this application is shown in the illustration.

[0026] Figure 6 A flowchart illustrating the optimization of training strategies according to an embodiment of this application is shown in the schematic diagram.

[0027] Figure 7 This schematically illustrates a structural block diagram of a target object training capability assessment device according to an embodiment of this application; and

[0028] Figure 8 A block diagram schematically illustrates an electronic device suitable for implementing a method for assessing the training capabilities of a target audience, according to an embodiment of this application. Detailed Implementation

[0029] The embodiments of this application will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of this application. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of this application for ease of explanation. However, it will be apparent that one or more embodiments may be implemented without these specific details. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.

[0030] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0031] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0032] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0033] In the technical solution of this application, the user information (including but not limited to user personal information, user image information, user device information, such as location information) and data (including but not limited to data used for analysis, stored data, and displayed data) involved are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of related data all comply with relevant laws, regulations, and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entry points for users to choose to authorize or refuse.

[0034] In scenarios involving automated decision-making using personal information, the methods, devices, and systems provided in this application all offer users corresponding entry points for choosing to agree to or reject the automated decision-making results. If the user chooses to reject, the process proceeds to the expert decision-making stage. Here, "automated decision-making" refers to the activity of automatically analyzing and evaluating an individual's behavioral habits, interests, or economic, health, and credit status through computer programs, and then making a decision. Here, "expert decision-making" refers to the activity of making decisions by personnel who specialize in a particular field, possess specialized experience, knowledge, and skills, and have reached a certain level of professional expertise.

[0035] The embodiments of this application provide a method for assessing the training ability of a target audience. A first score, a second score, and a guessing coefficient are obtained based on the target audience's answering process in a business training system. A comprehensive score is then obtained using a gating fusion mechanism based on the first score, second score, and guessing coefficient. The training ability level corresponding to the comprehensive score is then obtained from a preset training ability level table. This training ability level can characterize the level of the target audience's training ability. In this way, the accuracy of assessing the target audience's training ability can be improved.

[0036] Figure 1 The diagram illustrates an application scenario of the target object training capability assessment method according to an embodiment of this application.

[0037] like Figure 1 As shown, application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 serves as a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0038] Users can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 via the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0039] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0040] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (this is just an example). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0041] It should be noted that the target training capability assessment method provided in this application embodiment can generally be executed by server 105. Correspondingly, the target training capability assessment device provided in this application embodiment can generally be located in server 105. The target training capability assessment method provided in this application embodiment can also be executed by a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105. Correspondingly, the target training capability assessment device provided in this application embodiment can also be located in a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105.

[0042] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0043] The following will be based on Figure 1 The described scene, through Figures 2-6 The method for assessing the training capabilities of target subjects according to embodiments of this application will be described in detail.

[0044] Figure 2 A flowchart illustrating a method for assessing the training capabilities of a target audience according to an embodiment of this application is shown.

[0045] like Figure 2 As shown, the target training capability assessment method 200 of this embodiment includes operations S210 to S250.

[0046] In operation S210, the target's first score is obtained based on the target's answer results in the business training system and the knowledge points in the knowledge graph at the current moment.

[0047] In operation S220, the behavioral characteristics of the target object during the answering process are analyzed using a time series analysis model to obtain the target object's second score and guessing coefficient; whereby the guessing coefficient represents the degree of guessing in the target object's answer.

[0048] In operation S230, the second score is corrected based on the guessed coefficients to obtain the corrected second score.

[0049] In operation S240, a gating fusion mechanism is used to obtain a comprehensive score based on the first score, the guessed coefficient, and the corrected second score.

[0050] In operation S250, the training ability level corresponding to the comprehensive score is obtained from the preset training ability level table, and the training ability level is used as the evaluation result of the target object's training ability.

[0051] In some embodiments, the target object can be an employee, trainee, student, etc.; the first score is an explicit score, representing the accuracy of the target object's answer. The first score is obtained based on the target object's answer. For example, when testing a target object for a teller position, assuming the business scenario is helping customers with deposit business, the business training system will generate corresponding test questions. For example, the generated test questions may be the operational procedures required for deposit business. After the target object performs the operation, the business training system compares the timestamp in the log file with the knowledge points related to deposit business in the knowledge graph at the current moment to determine whether the target object's operation steps are correct, and assigns a score to each operation step of the target object. The scores are accumulated to obtain the first score. As another example, when testing a target object for a data analysis position, assuming the business scenario is risk data analysis, the business training system will generate corresponding test questions. For example, the generated test questions may be data security Q&A test questions. After the target object completes the test, the business training system judges the accuracy of the target object's answer based on the target object's answer and the relevant knowledge points in the knowledge graph at the current moment for that business scenario, and assigns a score to obtain the first score.

[0052] In some embodiments, the second score is an implicit score that characterizes the cognitive level of the target subject during the answering process. For example, if the target subject frequently modifies their answer during the answering process, it may indicate that the target subject has a poor grasp of the knowledge points of the question and the answer given may be obtained through guessing. Or, if the target subject suddenly jumps to another page while answering a question, it may indicate that the target subject is using other means to find the answer to the question, which also reflects that the target subject's grasp of the question is not up to standard. All of these reflect the cognitive level of the target subject during the answering process. Therefore, by analyzing the behavioral characteristics of the target subject during the answering process and obtaining the second score, the cognitive level of the target subject during the answering process can be quantitatively assessed.

[0053] In some embodiments, the guessing coefficient is used to characterize the degree to which the target object guesses the answer. For example, when analyzing the behavioral characteristics of the target object during the answering process, if it is found that the frequency of the target object modifying the answer is greater than a preset modification frequency threshold, then the guessing coefficient is obtained based on the difference between the frequency of modifying the answer and the preset modification frequency threshold. For another example, if the number of times the target object switches to other web pages or modules during the answering process is greater than a preset switching threshold, it indicates that the target object may be searching for the answer and has the possibility of guessing the answer. The guessing coefficient is obtained based on the difference between the number of switching times and the preset switching threshold.

[0054] According to an embodiment of this application, a first score is obtained based on the target's answer results. By analyzing the behavioral characteristics during the answering process, the cognitive level of the target is assessed, and a second score and a guessing coefficient are obtained. The second score is corrected based on the guessing coefficient to further improve the accuracy of the cognitive level assessment. Based on the first score, the corrected second score, and the guessing coefficient, the training ability level of the target is determined. This allows for a more accurate assessment of the target's training ability from multiple dimensions, reducing the problem of inflated answer scores caused by guessing and random answering.

[0055] Figure 3 A flowchart illustrating the first score acquisition process according to an embodiment of this application is shown.

[0056] like Figure 3 As shown, the steps for obtaining the first score of the target object include operations S310 to S330.

[0057] When operating S310, test questions corresponding to the business scenario are generated on the business training system.

[0058] In operation S320, the similarity between the target object's answer and the knowledge point corresponding to the question in the knowledge graph at the current moment is calculated.

[0059] In operation S330, the first score is obtained from the preset first score table based on the similarity; wherein the preset first score table is obtained by using a machine learning algorithm based on historical answer data.

[0060] In some embodiments, during operation S310, conditional generative adversarial networks can be used to generate test questions corresponding to the business scenario. For example, for deposit and withdrawal business, test questions for verifying deposit and withdrawal operation procedures can be generated; for business consultation scenarios, test questions for relevant business consultation questions and answers can be generated.

[0061] In some embodiments, during operation S320, if the test question is a text-based question, the similarity between the answer and the corresponding knowledge point can be calculated using a semantic matching method. If the test question is an operational process test, the similarity between the answer and the corresponding knowledge point can be calculated based on the timestamps on the log files in the business training system. For example, in the scenario of handling deposit business, the correct operation process is identity verification, cash counting, transaction risk identification and recording, and transaction report generation. If the timestamps in the log files show that the actual operation process of the target object is identity verification, transaction risk identification and recording, cash counting, and transaction report generation, it indicates that there is a problem with the second and third steps of the target object's operation. When scoring the operation steps of the target object, the incorrect steps should be given negative scores, and the correct steps should be given positive scores. Finally, the first score is obtained. The value range of the first score is [0, 100]. The first score can represent the degree of coverage of the target object's knowledge blind spots.

[0062] According to the embodiments of this application, the training capabilities of target personnel in different positions are assessed in conjunction with business scenarios. Based on the target personnel's answers, a first score is obtained, which intuitively reflects the target personnel's ability level and improves the accuracy of ability assessment.

[0063] Figure 4 A flowchart illustrating the knowledge graph creation process according to an embodiment of this application is shown.

[0064] like Figure 4 As shown, the steps for building a knowledge graph include operations S410 to S430.

[0065] When operating S410, knowledge within the domain is classified according to business type, and entity recognition is performed on the knowledge points in each business type to obtain the entity relationships between the knowledge points.

[0066] When operating S420, each case in the case library is decomposed according to the preset decomposition elements; the preset decomposition elements include business objectives, data sources, data analysis methods, and conclusions.

[0067] In operation S430, the decomposed data is associated with entity relationships to obtain a knowledge graph; the knowledge graph is continuously updated through a knowledge graph incremental update algorithm.

[0068] In some embodiments, during operation S410, the business types include core business, risk management business, compliance and regulatory business, and data analysis business. Core business may include deposit and loan business, payment and settlement, foreign exchange management (such as cross-border transaction settlement), etc. Risk management business may include credit risk management, market risk management (such as interest rate sensitivity gap analysis), operational risk (such as risk customer identification, transaction risk, etc.), etc. Compliance and regulatory business may involve supervising whether various businesses comply with regulatory rules. Data analysis business may involve querying interest rate sensitive assets, analyzing various financial data, and performing linear regression in credit scoring scenarios. Analysis, etc.; the entities extracted from the knowledge points of each business type can be loan approval, customer age, corporate account, regulatory rules, etc. The entity relationships between the knowledge points can be represented in the form of triples, for example, (loan approval, mortgage registration, loan disbursement), (customer age, wealth management product, customer gender), (loan approval, regulatory rules, loan disbursement). Among them, (loan approval, mortgage registration, loan disbursement) can be marked as a process-dependent triple relationship, (customer age, wealth management product, customer gender) can be marked as a data management triple relationship, and (loan approval, regulatory rules, loan disbursement) can be marked as a rule-triggered triple relationship.

[0069] In some embodiments, during operation S420, each case in the case library is decomposed according to preset decomposition elements. This is to demonstrate the entire process of each case processing, and to deeply analyze the connections between the knowledge points in each case from the perspectives of the tools or methods used to process the data and the application scenarios of the case. For example, for a case of identifying high-risk customer groups, the business objective of the case may be to identify credit card fraud transactions, the data source may be transaction records, customer geographical location and device fingerprints, etc., the data analysis method may be the Isolation Forest algorithm or the Prior Association Rule algorithm, etc., and the final conclusion may be to intercept abnormal transactions of high-risk customer groups or to manually check the transactions of high-risk customer groups. By decomposing the case, the entire process of processing the case can be obtained, which is conducive to associating the data and knowledge points after the case decomposition with entities, and thus constructing a knowledge graph.

[0070] In some embodiments, during operation S430, the knowledge graph is continuously updated using a knowledge graph incremental update algorithm. That is, the algorithm continuously adds acquired domain-specific knowledge to the current knowledge graph and calculates the correlation between the added content and each knowledge point in the current knowledge graph. For example, assuming the added content is consumer rights protection, the correlation between consumer rights protection and each knowledge point in the current knowledge graph is calculated. The calculation shows that consumer rights protection has the highest correlation with the complaint handling process in the current knowledge graph. In this way, the knowledge graph can be continuously updated based on actual application conditions. Using the continuously updated knowledge graph to assess the training capabilities of the target audience can further improve the accuracy and reliability of the assessment.

[0071] According to the embodiments of this application, by decomposing each case in the case library, associating the decomposed data with domain knowledge, a knowledge graph is obtained and continuously updated. This is beneficial for subsequent optimization of training strategies, thereby further improving the training capabilities of the target audience.

[0072] Figure 5 A flowchart illustrating the second score and guessing coefficient acquisition process according to an embodiment of this application is shown.

[0073] like Figure 5 As shown, the steps for obtaining the second score and the guess coefficient include operations S510 to S550.

[0074] In operation S510, the temporal and global features of the behavioral features are extracted using a time series analysis model. The temporal features represent the local changes of the behavioral features within a first preset time range, while the global features represent the overall changes of the behavioral features within a second preset time range. The first preset time range is shorter than the second preset time range.

[0075] In operation S520, the first sub-network in the hybrid neural network is used to process the temporal features and obtain the temporal feature values; wherein, the temporal feature values ​​represent the statistical values ​​of the temporal features.

[0076] In operating S530, the second sub-network in the hybrid neural network is used to process global features and obtain global feature values; where global feature values ​​represent the statistical values ​​of global features.

[0077] When operating the S540, the temporal feature values ​​and global feature values ​​are fused through the fusion layer of the hybrid neural network to obtain the second score.

[0078] In operation S550, the guessing coefficient is calculated based on the difference between the time series feature value and the first preset threshold corresponding to the time series feature value, and the difference between the global feature value and the second preset threshold corresponding to the global feature value.

[0079] In some embodiments, during operation S510, behavioral characteristics may include answer time, answer correction behavior, answer distribution, webpage or answer module switching behavior, answer device data, and eye-tracking data. Answer time includes average answer duration and Shannon entropy of answer time. An average answer duration threshold for the target subject is obtained using statistical methods. If the average answer duration obtained using a time series analysis model is significantly less than the average answer duration threshold, it indicates that the target subject may have guessed during the answering process, potentially resulting in an inflated score. The Shannon entropy of answer time is calculated based on the target subject's multiple answer times. Shannon entropy reflects the stability of the target subject's training ability level. If the calculated Shannon entropy is greater than a preset Shannon entropy threshold, it indicates that the target subject's answer time fluctuates greatly. Further, based on statistical analysis methods, the average answer duration threshold can be set to 20 seconds, and the preset Shannon entropy threshold can be set to 1.5. Answer correction behavior includes the number of answer corrections and the time interval between the last correction and submission. This is achieved through statistical analysis. If the number of answer corrections exceeds the preset number of corrections, it indicates that the target's answering strategy is chaotic. If the time interval between the last correction and submission is less than the preset time interval, it is considered that the answer was given randomly, and the submitted answer was a guess. Furthermore, through statistical analysis, the preset number of corrections can be set to 3 times / minute, and the preset time interval can be set to 5 seconds. The answer distribution includes the selection rate of unpopular options and the option randomness index. The selection rate of unpopular options refers to the probability of choosing options that do not appear frequently. In multiple-choice questions, the case where the selection rate of unpopular options among the incorrect options exceeds the preset selection rate threshold is considered an abnormal situation. The option randomness index is calculated based on the multinomial distribution of the options. The larger the option randomness index, the greater the probability that the target's answer is guessed. The case where the option randomness index exceeds the preset option randomness index threshold is considered a guessed answer. Furthermore, according to statistical analysis, the preset selection rate threshold can be set to 20%, and the preset option randomness index threshold can be set to 0.8. Webpage or quiz module switching behavior refers to the behavior of the target user switching to other webpages or quiz modules during the quiz process. This switching behavior includes switching frequency and switching path entropy. When the switching frequency exceeds a preset frequency, the target user's quiz behavior is considered speculative searching, indicating that the answer provided by the target user contains elements of guesswork. Switching path entropy represents the degree of disorder in the switching path. When the switching path entropy exceeds a preset value, the switching path is considered to be random jumping, indicating that the target user is engaging in speculative behavior. The probability of finding the relevant information is higher. For example, when the target is answering questions about credit risk, the system might redirect them to a cross-bank payment clearing details page, a commemorative coin exchange page, and a wealth management product introduction page. These three pages have very low correlation with credit risk, and the correlation between these three pages is also very low. This situation is considered random redirection. Furthermore, based on statistical analysis methods, the preset switching frequency can be set to 2 times / 10 minutes, and the preset switching path entropy value can be set to 2.0. The answering device data includes the number of times answers were submitted from different locations and the percentage of time spent answering at night. If the number of remote login attempts exceeds the preset number, it indicates that external assistance may have been involved in the response. If the proportion of nighttime responses exceeds the preset proportion, it indicates that the target is undergoing intensive study. Responses between 10 PM and 6 AM the following day are defined as nighttime responses. Furthermore, based on statistical analysis methods, the preset number of remote login attempts can be set to 1, and the preset proportion can be set to 50%. Eye-tracking data includes the target's gaze dispersion and the duration of gaze monitoring rules. Gazing dispersion refers to the distance between the target's eye movements and the distance between the gaze points. The dispersion of the screen area being gazed upon is determined by eye-tracking devices such as eye trackers. A gaze dispersion greater than a preset threshold is considered inattentiveness during the question-answering process. The duration of the gaze monitoring rule refers to the cumulative duration of the gaze monitoring rule during the question-answering process. When the duration of the gaze monitoring rule is less than the preset gaze duration, it is considered that the target subject has not carefully read the monitoring rule. Furthermore, based on statistical analysis methods, the preset gaze dispersion threshold can be set to 0.7, and the preset gaze duration can be set to 10 seconds.

[0080] In some embodiments, during operation S520, the timing characteristic values ​​can be the target object's average answering time, the number of answer corrections, the number of times the target object logs in from different locations to answer questions, etc. For example, the target object's average answering time is 45 seconds, the number of answer corrections is 5 times, and the number of times the target object logs in from different locations to answer questions is 3 times, etc.

[0081] In some embodiments, during operation S530, the global feature value can be the Shannon entropy of the target object's answer time, the option randomness index, and the switching path entropy value, etc. For example, the Shannon entropy of the answer time is 2, the option randomness index is 0.5, and the switching path entropy value is 1.5.

[0082] In some embodiments, after operating S540 and fusing the temporal feature value and the global feature value, the fused value is mapped to a score to obtain a second score. The value range of the second score is [1,5]. The second score can reflect potential knowledge defects that were not captured by the first score.

[0083] In some embodiments, the training sample set of the hybrid neural network includes manually labeled guess samples, wherein the training sample set is a large number of historical answer records.

[0084] In some embodiments, during operation S550, assuming the average answering time in the time-series feature values ​​is 45 seconds, and the first preset threshold (here, the average answering time threshold) corresponding to the average answering time is 20 seconds, the guessing coefficient corresponding to the average answering time is calculated using a linear mapping formula based on the difference between the average answering time and its corresponding first preset threshold. Further, the linear mapping formula corresponding to the average answering time can be:

[0085]

[0086] in, This represents the guessing coefficient corresponding to the average answering time. , Let represent the average answering time and the average answering time threshold, respectively. Using a similar method, the guessing coefficients corresponding to other time-series feature values ​​and the global feature value can be obtained. For example, if the switching frequency in the time-series feature value is 5 times / 10 minutes and the preset switching frequency is 2 times / 10 minutes, the guessing coefficient corresponding to the switching frequency can be obtained based on the difference between these two switching frequencies using the linear mapping formula corresponding to the switching frequency. Similarly, if the option randomness index in the global feature value is 0.95 and the preset option randomness index threshold is 0.8, the guessing coefficient corresponding to the option randomness index can be obtained using the linear mapping formula corresponding to the option randomness index. Finally, these guessing coefficients are weighted and summed to obtain the final required guessing coefficient. Furthermore, the guessing coefficient ranges from [0,1]. The guessing coefficient can represent the degree of guessing by the target object (also known as the degree of speculative answering) and is used to adjust the credibility of the evaluation.

[0087] According to embodiments of this application, by obtaining a second score and a guessing coefficient, it is possible to assess the target object's guessing answers and the cognitive level demonstrated during the target object's answering process, thereby improving the accuracy and dimensionality of the assessment.

[0088] In some embodiments, the second score is corrected based on the guessing coefficient, including: correcting the second score according to the guessing coefficient, the attenuation coefficient of the guessing coefficient relative to the second score, the risk coefficient, and the violation operation parameter; wherein, the attenuation coefficient characterizes the degree of influence of the guessing coefficient on the second score, the risk coefficient characterizes the degree of violation risk of the customer data involved in the target object's answer, and the violation operation parameter characterizes the degree of violation operation generated by the target object during the answering process.

[0089] Furthermore, the second score can be corrected using a correction formula, which is as follows:

[0090]

[0091] in, The revised second score. The second score is the uncorrected score. , , and

[0092] These represent the decay coefficient of the guessing coefficient relative to the second score, the final guessing coefficient, the risk coefficient, and the violation operation parameter, respectively. If the customer data in the generated test question has a violation risk, the risk coefficient is greater than 0 and less than or equal to 1. If there is no customer data with a violation risk in the generated test question, the risk coefficient is equal to 0. If the target object commits a violation during the answering process, the violation operation parameter is 1. If the target object does not commit a violation during the answering process, the violation operation parameter is 0. For example, when the target object answers the question "Customer Data Usage", the guessing coefficient is 0.7, the violation operation parameter is 1, the risk coefficient is 0.2, and the uncorrected second score is 4. According to the correction formula above, the corrected second score is 2.32.

[0093] According to embodiments of this application, by using a guessing coefficient to correct the second score, the accuracy of assessing the cognitive level of the target object during the answering process can be improved.

[0094] In some embodiments, a gating fusion mechanism is used to obtain a comprehensive score based on a first score, a guessing coefficient, and a corrected second score. This includes: using the gating fusion mechanism, the comprehensive score is calculated using a comprehensive score calculation formula based on the first score, the guessing coefficient, and the corrected second score; further, the comprehensive score calculation formula is as follows: ,in, For the overall score, For first place, , , These represent the weights of the first score, the corrected second score, and the guess coefficient, respectively. Furthermore, based on statistical analysis methods, for common business scenarios, we can... Set to 0.6, Set to 0.4, Setting it to 0.5 allows for adjustments in high-risk scenarios, such as credit risk, operational compliance, and transaction risk identification. For example, Setting it to 0.6 reduces the tolerance for guessing, thereby decreasing the probability of risk occurring.

[0095] In some embodiments, a preset training competency level table is shown in Table 1:

[0096] Table 1

[0097]

[0098] Based on the first score, the guessing coefficient, the corrected second score, and the overall score, the training ability level of the target subject can be obtained from Table 1. Table 1 is based on the historical scoring results of each target subject obtained through statistical methods. For example, if the target subject's first score is greater than or equal to 85, the corrected second score is less than or equal to 2, the guessing coefficient is less than or equal to 0.3, and the overall score is greater than or equal to 0.85, then the target subject's training ability level is proficient; if the target subject's first score is between 70 and 85, the corrected second score is between 2 and 3, the guessing coefficient is between 0.3 and 0.5, and the overall score is between 0.7 and 0.85, then the target subject's training ability level is skilled; if the target subject's first score is between 55 and 70, the corrected second score is between 3 and 4, the guessing coefficient is between 0.5 and 0.7, and the overall score is between 0.85 and 0.85, then the target subject's training ability level is proficient. If the score is between 0.55 and 0.7, the target's training ability level is considered qualified. If the target's first score is less than 55, the corrected second score is greater than or equal to 4, the guessing coefficient is greater than or equal to 0.7, and the overall score is less than 0.55, then the target's training ability level is considered unqualified. Furthermore, for target individuals with a proficient training ability level, they can be involved in more complex business operations or serve as internal training instructors. For target individuals with a skilled training ability level, they can be assigned high-value clients and receive regular training. For target individuals with a qualified training ability level, they can be provided with specialized skills training, such as training in the use of structured query language or customer relationship management. For target individuals with an unqualified training ability level, they can be provided with comprehensive knowledge training or assigned a mentor for one-on-one guidance.

[0099] Figure 6 A flowchart illustrating the optimization of training strategies according to an embodiment of this application is shown.

[0100] like Figure 6 As shown, the training strategy optimization steps include operations S610 to S650.

[0101] When operating S610, subsequent training strategies are obtained from the preset training strategy table based on the training capability level; the preset training strategy table is obtained by statistical analysis of historical training strategies and the historical training performance of each target group.

[0102] When operating the S620, a new training strategy is generated using a greedy strategy based on the subsequent training strategy.

[0103] When operating S630, the reward values ​​of subsequent training strategies and new training strategies are predicted separately through a hierarchical decision network, and the training strategy with the highest reward value is selected as the final training strategy.

[0104] When operating the S640, the target object is trained according to the final training strategy, and the first score, the corrected second score, and the guess coefficient of the target object are re-obtained.

[0105] In operating S650, repeat the operations of acquiring training ability level, final training strategy, first score, corrected second score, and guess coefficient until the target object's first score, corrected second score, and guess coefficient reach their respective preset scores.

[0106] In some embodiments, during operation S610, when the training ability level is qualified in the preset training strategy table, if the first score is less than the first preset score, it indicates that the target object has a poor grasp of the basic knowledge in the domain. Therefore, the subsequent training strategy provided to them can be training methods that can consolidate basic knowledge, such as learning basic knowledge courses. According to statistical methods, the first preset score can be set to 60 points. When the training ability level is qualified or unqualified, if the fluctuation value of the first score is greater than the preset score fluctuation threshold, it is necessary to provide the target object with specialized reinforcement training. For example, providing the target object with a chain of micro-courses related to weak knowledge points. For instance, assuming that the target object is relatively weak in data analysis, micro-courses on data cleaning, feature engineering, and model training can be provided to the target object. These micro-courses are all closely related to data analysis business. Furthermore, based on statistical methods, the preset score fluctuation threshold can be set to 20%. When the training ability level is qualified or proficient, if the first score is greater than or equal to the second preset score and the corrected second score is greater than the third preset score, it indicates that the target audience understands the knowledge points but not deeply. Therefore, scenario simulation training can be provided to the target audience, such as various simulated cases, such as training on cross-border remittance compliance review processes and credit risk monitoring processes. Furthermore, based on statistical methods, the second preset score can be set to 70 points and the third preset score can be set to 2. When the training ability level is unqualified or the target audience's operation involves violations, it is necessary to provide the target audience with training on regulatory rules and develop test questions for them.

[0107] In some embodiments, during operation S620, for an evaluation result where the first score is less than a first preset score, the second score is the maximum score, and the guessing coefficient is equal to the first preset guessing coefficient, where the first preset guessing coefficient can be 0.3, this evaluation result indicates that the target object's basic knowledge is weak. The new training strategy can be to provide the target object with basic knowledge training, for example, to let the target object learn relevant knowledge according to basic knowledge concepts, business processing procedures, and case studies. For an evaluation result where the first score is equal to a fourth preset score, the second score is 4, and the guessing coefficient is equal to the second preset guessing coefficient, where the fourth preset score can be 75, and the second preset guessing coefficient can be 0.2, this evaluation result indicates that the target object's explicit score (first score) is inflated. The new training strategy can be to provide the target object with case studies for business simulation, for example, to let the target object simulate cross-border remittance operations, credit card application operations, etc. For an evaluation result where the first score is equal to a fifth preset score, the second score is 4, and the second score is 4, the new training strategy can be to provide the target object with case studies for business simulation, for example, to let the target object simulate cross-border remittance operations, credit card application operations, etc. The evaluation result is divided into three parts, with the guessing coefficient equal to the third preset guessing coefficient. The fifth preset score is 85, and the third preset guessing coefficient is 0.8. This evaluation result indicates that the target's answer was a guess. Therefore, the new training strategy could be to have the target learn the regulatory rules, then conduct another training test, reminding them not to guess, and recording their answers in a log. For the evaluation result where the first score equals the sixth preset score, the second score equals 2, and the guessing coefficient equals the fourth preset guessing coefficient, the sixth preset score could be 90, and the fourth preset guessing coefficient could be 0.1. This evaluation result indicates that the target has a basic and comprehensive grasp of the knowledge points in the corresponding business scenario. In this case, the new training strategy could be to provide the target with cross-business scenario training programs, such as training from credit risk control to big data credit scoring, or training from retail fraud to auto insurance claims.

[0108] In some embodiments, if a target employee in a teller position receives a first score of 55 during suspicious transaction monitoring training, with errors primarily concentrated on the "time limit for reporting large transactions," a second score of 5 (indicating significant potential knowledge gaps), and a guessing coefficient of 0.2 (indicating high reliability of the training capability assessment), based on the obtained assessment results, after calculating the reward value for each training strategy, the final training strategy adopted is to first push a micro-course on suspicious transaction monitoring to the target employee, then have the target employee simulate cash deposit and withdrawal operations, forcing them to perform cash deposit and withdrawal operations in the order of identity verification, system entry, and report submission, and finally have the target employee answer compliance questions, which include questions on "time limit for reporting large transactions." Regarding the "easy report time limit" issue, after the target audience completes the training, their first score, second score, and guess coefficient are recalculated. It is found that the first score has increased, while the second score and guess coefficient have both decreased. Then, the training strategy is determined based on the newly obtained first score, second score, and guess coefficient. This process is repeated until the target audience's first score, second score, and guess coefficient each reach their respective preset values. In this business scenario, after two cycles, the first score can be increased to 78, and the second score reduced to 3. When the target audience begins suspicious transaction monitoring training, the total training path length is 20 steps; after two cycles, the total training path length is 12. Step 1: Assuming the target audience is account managers, during customer value quantification training, the account managers received a first score of 75, with errors concentrated on the correlation between high-value customers and supply chain finance products. Their second score was 4, and they switched modules more than 5 times per 10 minutes during the test, with a guessing coefficient of 0.3, which is considered low. Based on the obtained evaluation results, after calculating the reward values ​​for each training strategy, the final training strategy adopted was to have the target audience learn about real-world cases of high-value customer churn warnings and design retention strategies using customer value quantification tools. After the target audience completed this training, the account managers' first and second scores and guessing coefficients were recalculated, revealing an increase in the first score. Both the second score and the guessing coefficient decreased. Then, the training strategy was determined based on the newly obtained first score, second score, and guessing coefficient. This process was repeated until the target's first score, second score, and guessing coefficient reached their respective preset values. In this business scenario, after 3 cycles, the first score could be increased to 88, and the second score reduced to 2. Assuming the target is a data analyst, when the target receives coding training, their first score is 90, indicating that the code they wrote is basically correct. Their second score is 2, indicating that their operation is relatively smooth. However, during the question-and-answer process, the module switching entropy value is greater than 2, and the guessing coefficient is 0.9. This indicates an abnormally chaotic switching path; the target participant frequently switched to external platforms during the quiz, violating regulatory rules. Based on the obtained assessment results, after calculating the reward values ​​for each training strategy, the final training strategy adopted was to force the target participant to learn the regulatory rules and retake the test. A "closed-book programming without reference materials" component was also added. After the target participant completed this training, their first score, second score, and guessing coefficient were recalculated. The first score increased, while the second score and guessing coefficient decreased. Then, the training strategy was determined again based on the newly obtained first score, second score, and guessing coefficient. This process was repeated until the target participant's first score, second score, and guessing coefficient reached their respective preset values. In this business scenario, after two iterations, the first score was 60, and the guessing coefficient decreased to 0.3. Only then could the assessment result accurately and objectively evaluate the target participant's training ability.

[0109] According to the embodiments of this application, by obtaining the training strategy with the highest reward value, the purpose of optimizing the training strategy can be achieved, which is conducive to carrying out targeted training for different target objects, thereby significantly improving the business level of the target objects; by continuously adjusting the training strategy based on the target object's first score, second score and guessing coefficient, the training strategy can be updated in real time, improving the timeliness of the training strategy.

[0110] In some embodiments, a hierarchical decision network is used to predict the reward values ​​of subsequent training strategies and new training strategies, respectively. This includes: constructing a state vector at the current moment based on the target object's first score, the corrected second score, the guessing coefficient, and the knowledge graph at the current moment; inputting the state vector at the current moment into the hierarchical decision network so that the hierarchical decision network can predict the reward values ​​of subsequent training strategies and new training strategies under the state vector at the current moment based on the knowledge points in the knowledge graph at the current moment whose relevance to the current business scenario meets a preset relevance threshold; furthermore, the hierarchical decision network can calculate the reward value of each training strategy through a Markov decision process.

[0111] For example, suppose the business scenario is transaction risk identification. The target object's initial score at the current moment is 70 points, the corrected second score is 2.5, and the guessing coefficient is 0.2. The knowledge point in the knowledge graph with the highest relevance to this business scenario at the current moment is the customer's transaction history over the past year. The reward values ​​for training strategies related to the customer's transaction history over the past year in the acquired subsequent training strategies and the reward values ​​for training strategies related to the customer's transaction history over the past year in the new training strategies are calculated separately. The training strategy with the highest reward value is selected as the final training strategy for the target object. After the target object is trained, the initial score, the corrected second score, and the guessing coefficient are calculated again. If the initial score increases, the corrected second score... If both the first score and the guessing coefficient decrease, it indicates that the training ability of the target audience has been improved. Assuming the business scenario is a deposit and withdrawal operation, the knowledge point in the knowledge graph with the highest relevance to this business scenario at the current moment is the customer's account balance. Calculate the reward value of the training strategies related to the customer's account balance in the subsequent training strategies and the reward value of the training strategies related to the customer's account balance in the new training strategies. Select the training strategy with the highest reward value as the final training strategy to train the target audience. After the target audience is trained, recalculate the target audience's first score, the corrected second score, and the guessing coefficient. If the first score increases and the corrected second score and the guessing coefficient decrease, it indicates that the training ability of the target audience has been improved.

[0112] In some embodiments, the reward value of the training strategy is calculated using the following reward function:

[0113]

[0114] in, , and These represent the increase in the first score, the decrease in the corrected second score, and the decrease in the guess coefficient, respectively, after adopting the acquired training strategy. , , These are the coefficients for increasing the first score, decreasing the corrected second score, and decreasing the guessing coefficient. Indicates the compliance coefficient. This represents the compliance reward value; if the acquired training strategy complies with regulatory rules, then... Assign a positive value if the acquired training strategy does not comply with regulatory rules. Assign a negative value.

[0115] According to the embodiments of this application, by predicting the reward value of each training strategy and determining the training strategy, the training strategy can be optimized, which is conducive to carrying out targeted training for different target groups, thereby significantly improving the business level of the target groups.

[0116] Based on the above-mentioned method for assessing the training capacity of the target audience, this application also provides a device for assessing the training capacity of the target audience. The following will be combined with... Figure 7 The device is described in detail.

[0117] Figure 7 A schematic block diagram of a target training capability assessment device according to an embodiment of this application is shown.

[0118] like Figure 7 As shown, the target training ability assessment device 700 of this embodiment includes an explicit score acquisition module 710, an implicit score acquisition module 720, an implicit score correction module 730, a comprehensive score acquisition module 740, and a training ability assessment module 750.

[0119] The explicit score acquisition module 710 is used to obtain the target object's first score based on the target object's answer results in the business training system and the knowledge points in the knowledge graph at the current moment. In one embodiment, the explicit score acquisition module 710 can be used to perform the operation S210 described above, which will not be repeated here.

[0120] The implicit score acquisition module 720 is used to analyze the behavioral characteristics of the target object during the answering process using a time series analysis model to obtain the target object's second score and guessing coefficient; wherein, the guessing coefficient represents the degree of guessing in the target object's answer. In one embodiment, the implicit score acquisition module 720 can be used to perform the operation S220 described above, which will not be repeated here.

[0121] The implicit score correction module 730 is used to correct the second score based on the guessed coefficients to obtain the corrected second score. In one embodiment, the implicit score correction module 730 can be used to perform the operation S230 described above, which will not be repeated here.

[0122] The comprehensive score acquisition module 740 is used to obtain a comprehensive score based on the first score, the guessed coefficient, and the corrected second score using a gating fusion mechanism. In one embodiment, the comprehensive score acquisition module 740 can be used to perform the operation S240 described above, which will not be repeated here.

[0123] The training capability assessment module 750 is used to obtain the training capability level corresponding to the comprehensive score from a preset training capability level table, and use the training capability level as the assessment result of the target object's training capability. In one embodiment, the training capability assessment module 750 can be used to perform the operation S250 described above, which will not be repeated here.

[0124] In some embodiments, the explicit score acquisition module 710 is specifically used to: generate test questions corresponding to the business scenario on the business training system according to the business scenario; calculate the similarity between the answer result of the target object and the knowledge point corresponding to the test question in the knowledge graph at the current time; and obtain the first score from the preset first score table according to the similarity; wherein the preset first score table is obtained by using a machine learning algorithm based on historical answer data.

[0125] In some embodiments, the explicit score acquisition module 710 is further configured to:

[0126] The knowledge graph is constructed as follows: Domain knowledge is categorized according to business type; entity recognition is performed on knowledge points within each business type to obtain entity relationships between knowledge points; each case in the case library is decomposed according to preset decomposition elements, including business objectives, data sources, data analysis methods, and conclusions; the decomposed data is associated with entity relationships to obtain the knowledge graph; the knowledge graph is continuously updated using a knowledge graph incremental update algorithm.

[0127] In some embodiments, the implicit score acquisition module 720 is specifically used for: extracting temporal features and global features of behavioral features using a temporal analysis model; wherein, the temporal features represent the local changes of behavioral features within a first preset time range, and the global features represent the overall changes of behavioral features within a second preset time range, the first preset time range being smaller than the second preset time range; processing the temporal features using a first sub-network in a hybrid neural network to obtain temporal feature values; wherein, the temporal feature values ​​represent the statistical values ​​of the temporal features; processing the global features using a second sub-network in a hybrid neural network to obtain global feature values; wherein, the global feature values ​​represent the statistical values ​​of the global features; fusing the temporal feature values ​​and global feature values ​​through a fusion layer of the hybrid neural network to obtain a second score; and calculating a guessing coefficient based on the difference between the temporal feature values ​​and the first preset threshold corresponding to the temporal feature values, and the difference between the global feature values ​​and the second preset threshold corresponding to the global feature values.

[0128] In some embodiments, the implicit score correction module 730 is specifically used to: correct the second score based on the guessing coefficient, the attenuation coefficient of the guessing coefficient relative to the second score, the risk coefficient, and the violation operation parameter; wherein, the attenuation coefficient represents the degree of influence of the guessing coefficient on the second score, the risk coefficient represents the degree of violation risk of the customer data involved in the target object's answer, and the violation operation parameter represents the degree of violation operation generated by the target object during the answering process.

[0129] In some embodiments, the device 700 is further configured to: obtain subsequent training strategies from a preset training strategy table based on the training ability level; wherein the preset training strategy table is obtained by statistical analysis of historical training strategies and the historical training performance of each target object; generate new training strategies using a greedy strategy based on the subsequent training strategies; predict the reward values ​​of the subsequent training strategies and the new training strategies respectively through a hierarchical decision network, and select the training strategy with the highest reward value as the final training strategy; train the target object according to the final training strategy, and re-obtain the target object's first score, the corrected second score, and the guessing coefficient; repeat the operations of obtaining the training ability level, obtaining the final training strategy, and obtaining the first score, the corrected second score, and the guessing coefficient until the target object's first score, the corrected second score, and the guessing coefficient reach their respective preset scores.

[0130] In some embodiments, the device 700 is further configured to: construct a state vector at the current moment based on the target object's first score at the current moment, the corrected second score, the guessing coefficient, and the knowledge graph at the current moment; input the state vector at the current moment into a hierarchical decision network, so that the hierarchical decision network can predict the reward values ​​of subsequent training strategies and new training strategies under the state vector at the current moment based on the knowledge points in the knowledge graph at the current moment whose correlation with the current business scenario meets a preset correlation threshold.

[0131] According to embodiments of this application, the device 700 can assess the target's guessing behavior and cognitive level during the answering process, thereby comprehensively and multidimensionally assessing the target's training ability and improving the accuracy of the assessment.

[0132] According to embodiments of this application, any multiple modules among the explicit score acquisition module 710, implicit score acquisition module 720, implicit score correction module 730, comprehensive score acquisition module 740, and training ability assessment module 750 can be combined into one module, or any one of these modules can be split into multiple modules. Alternatively, at least some of the functions of one or more of these modules can be combined with at least some of the functions of other modules and implemented in one module. According to embodiments of this application, at least one of the explicit score acquisition module 710, implicit score acquisition module 720, implicit score correction module 730, comprehensive score acquisition module 740, and training ability assessment module 750 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or any other reasonable means of integrating or packaging the circuitry, or implemented in any one of the three methods of software, hardware, and firmware, or in a suitable combination of any of these. Alternatively, at least one of the explicit score acquisition module 710, implicit score acquisition module 720, implicit score correction module 730, comprehensive score acquisition module 740, and training ability assessment module 750 can be at least partially implemented as a computer program module, which can perform corresponding functions when the computer program module is run.

[0133] Figure 8 A block diagram schematically illustrates an electronic device suitable for implementing a method for assessing the training capabilities of a target audience, according to an embodiment of this application.

[0134] like Figure 8 As shown, an electronic device 800 according to an embodiment of this application includes a processor 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage portion 808 into a random access memory (RAM) 803. The processor 801 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 801 may also include onboard memory for caching purposes. The processor 801 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of this application.

[0135] RAM 803 stores various programs and data required for the operation of electronic device 800. Processor 801, ROM 802, and RAM 803 are interconnected via bus 804. Processor 801 executes various operations of the method flow according to embodiments of this application by executing programs in ROM 802 and / or RAM 803. It should be noted that the programs may also be stored in one or more memories other than ROM 802 and RAM 803. Processor 801 may also execute various operations of the method flow according to embodiments of this application by executing programs stored in said one or more memories.

[0136] According to embodiments of this application, the electronic device 800 may further include an input / output (I / O) interface 805, which is also connected to a bus 804. The electronic device 800 may also include one or more of the following components connected to the input / output (I / O) interface 805: an input section 806 including a keyboard, mouse, etc.; an output section 807 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the input / output (I / O) interface 805 as needed. A removable medium 811, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 810 as needed so that computer programs read from it can be installed into the storage section 808 as needed.

[0137] This application also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of this application.

[0138] According to embodiments of this application, the computer-readable storage medium can be a non-volatile computer-readable storage medium, such as including but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this application, the computer-readable storage medium may include ROM 802 and / or RAM 803 and / or one or more memories other than ROM 802 and RAM 803 described above.

[0139] Embodiments of this application also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code enables the computer system to implement the target training capability assessment method provided in the embodiments of this application.

[0140] When the computer program is executed by the processor 801, it performs the functions defined in the system / apparatus of this application embodiment. According to the embodiments of this application, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0141] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 809, and / or installed from a removable medium 811. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0142] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 809, and / or installed from the removable medium 811. When the computer program is executed by the processor 801, it performs the functions defined in the system of this application embodiment. According to the embodiments of this application, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0143] According to embodiments of this application, program code for executing the computer programs provided in the embodiments of this application can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0144] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0145] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application.

Claims

1. A method for assessing the training capabilities of a target audience, characterized in that, The method includes: Based on the target object's answer results in the business training system and the knowledge points in the knowledge graph at the current moment, obtain the target object's first score; The behavioral characteristics of the target object during the question-answering process are analyzed using a time series analysis model to obtain the target object's second score and guessing coefficient; wherein, the guessing coefficient represents the degree of guessing in the target object's answer. The second score is corrected based on the guessing coefficient to obtain the corrected second score; Based on the first score, the guessed coefficient, and the corrected second score, a gating fusion mechanism is used to obtain a comprehensive score. The training ability level corresponding to the comprehensive score is obtained from the preset training ability level table, and the training ability level is used as the evaluation result of the training ability of the target object.

2. The method according to claim 1, characterized in that, The step of obtaining the target object's first score based on the target object's answer results in the business training system and the knowledge points in the knowledge graph at the current moment includes: Based on the business scenario, generate test questions corresponding to the business scenario on the business training system; Calculate the similarity between the target object's answer and the knowledge point corresponding to the question in the knowledge graph at the current moment; The first score is obtained from a preset first score table based on the similarity; wherein the preset first score table is obtained using a machine learning algorithm based on historical answer data.

3. The method according to claim 1, characterized in that, The knowledge graph is constructed in the following way: The domain knowledge is classified according to business type, and entity recognition is performed on the knowledge points in each business type to obtain the entity relationships between the knowledge points. Each case in the case library is decomposed according to preset decomposition elements; wherein, the preset decomposition elements include business objectives, data sources, data analysis methods, and conclusions; The decomposed data is associated with the entity relationships to obtain the knowledge graph; wherein, the knowledge graph is continuously updated through a knowledge graph incremental update algorithm.

4. The method according to claim 1, characterized in that, The step of using a time-series analysis model to analyze the behavioral characteristics of the target object during the question-answering process, and obtaining the target object's second score and guessing coefficient, includes: The temporal and global features of the behavioral characteristics are extracted using the temporal analysis model; wherein the temporal features represent the local changes of the behavioral characteristics within a first preset time range, and the global features represent the overall changes of the behavioral characteristics within a second preset time range, wherein the first preset time range is shorter than the second preset time range; The temporal features are processed using the first sub-network in the hybrid neural network to obtain temporal feature values; wherein, the temporal feature values ​​represent the statistical values ​​of the temporal features; The global features are processed using the second sub-network in the hybrid neural network to obtain global feature values; wherein, the global feature values ​​represent the statistical values ​​of the global features; The second score is obtained by fusing the temporal feature values ​​and the global feature values ​​through the fusion layer of the hybrid neural network; The guessing coefficient is calculated based on the difference between the time-series feature value and the first preset threshold corresponding to the time-series feature value, and the difference between the global feature value and the second preset threshold corresponding to the global feature value.

5. The method according to claim 1, characterized in that, The step of correcting the second score based on the guessing coefficient includes: The second score is corrected based on the guessing coefficient, the attenuation coefficient of the guessing coefficient relative to the second score, the risk coefficient, and the violation operation parameter; wherein, the attenuation coefficient represents the degree of influence of the guessing coefficient on the second score, the risk coefficient represents the degree of violation risk of the customer data involved when the target object answers the question, and the violation operation parameter represents the degree of violation operation that occurred during the target object's answering of the question.

6. The method according to claim 1, characterized in that, The method further includes: Based on the training capability level, subsequent training strategies are obtained from a preset training strategy table; wherein, the preset training strategy table is obtained by statistical analysis of historical training strategies and the historical training performance of each target object; Based on the aforementioned subsequent training strategy, a new training strategy is generated using a greedy strategy. The reward values ​​of the subsequent training strategy and the new training strategy are predicted by a hierarchical decision network, and the training strategy with the highest reward value is selected as the final training strategy. The target object is trained according to the final training strategy, and the first score, the corrected second score, and the guess coefficient of the target object are obtained again. The process of repeatedly obtaining training ability levels, final training strategies, and first scores, corrected second scores, and guessing coefficients is repeated until the target object's first score, corrected second score, and guessing coefficient each reach their respective preset scores.

7. The method according to claim 6, characterized in that, The step of predicting the reward values ​​of the subsequent training strategy and the new training strategy using a hierarchical decision network includes: Based on the target object's first score, corrected second score, guessing coefficient, and knowledge graph at the current moment, construct the state vector at the current moment; The current state vector is input into the hierarchical decision network so that the hierarchical decision network can predict the reward values ​​of the subsequent training strategy and the new training strategy under the current state vector based on the knowledge points in the knowledge graph at the current time that have a correlation with the current business scenario that meet a preset correlation threshold.

8. A device for assessing the training ability of a target audience, characterized in that, The device includes: The explicit score acquisition module is used to obtain the first score of the target object based on the target object's answer results in the business training system and the knowledge points in the knowledge graph at the current moment; The implicit score acquisition module is used to analyze the behavioral characteristics of the target object during the answering process using a time series analysis model, and obtain the target object's second score and guessing coefficient; wherein, the guessing coefficient represents the degree of guessing of the target object's answer; The implicit score correction module is used to correct the second score based on the guessing coefficient to obtain the corrected second score; The comprehensive score acquisition module is used to obtain a comprehensive score based on the first score, the guessed coefficient, and the corrected second score using a gating fusion mechanism. The training capability assessment module is used to obtain the training capability level corresponding to the comprehensive score from a preset training capability level table, and use the training capability level as the assessment result of the target object's training capability.

9. An electronic device, comprising: One or more processors; Memory, used to store one or more computer programs. The characteristic feature is that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 7.

11. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 7.

Images