Professional skill evaluation method and system based on artificial intelligence large model

Through an adaptive assessment method based on artificial intelligence large model, combined with multiple algorithms and data mining technologies, the existing evaluation system is solved, and efficient and personalized skill assessment and career development suggestions are achieved. Adapt to changing needs of the industry.

CN118917738APending Publication Date: 2024-11-08DMAI (GUANGZHOU) CO LTD
View PDF 0 Cites 5 Cited by

Patent Information

Application Number
CN202411100705.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-12
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

The existing artificial intelligence evaluation system has problems such as low efficiency, strong subjectivity, and difficulty in adapting to industry changes in talent skills evaluation, and poor code reusability, making it difficult to meet the evaluation needs in different scenarios.

Method used

Adopting professional skills assessment methods based on artificial intelligence large models, selecting suitable text comprehension and generative models for customized training, combining Bayesian optimization, computer adaptive testing, enhanced learning and multi-arm slot machine algorithms to realize adaptive assessment Strategy, dynamically adjust the difficulty and type of questions, integrate data mining and natural language processing algorithms to generate personalized reports.

Benefits of technology

A more objective, efficient, personalized and accurate skill assessment has been achieved, helping enterprises better select talents and formulate career development plans, identify skills blind spots and provide targeted suggestions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118917738A_ABST
    Figure CN118917738A_ABST
Patent Text Reader

Abstract

The invention provides a professional skill evaluation method based on an artificial intelligence large model, and the method comprises the following steps: selecting the artificial intelligence large model suitable for text understanding and generation, and carrying out the customized training, and obtaining an optimized model; based on the optimized model, professional skill evaluation is carried out in combination with an adaptive evaluation strategy, and a professional skill evaluation result is obtained; the adaptive evaluation strategy integrates a Bayesian optimization algorithm, a computer adaptive test algorithm, a reinforcement learning algorithm and a dobby machine algorithm; and carrying out graphical display on the professional skill evaluation result to generate a professional skill improvement report. According to the method, more objective, efficient, personalized and accurate skill assessment can be provided, so that enterprises are helped to better perform talent selection, training and individual occupational development planning.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and more specifically to a professional skills assessment method and system based on an artificial intelligence large model. Background Art

[0002] In the current digital age, the advancement of artificial intelligence (AI) technology has greatly changed the operating mode of many industries, especially in human resource management and career development. Traditional talent skill assessment methods mainly rely on manual assessment and interviews. Although this method can provide deep interpersonal interaction and intuitive judgment, it also has some obvious limitations.

[0003] Manual evaluation usually requires a lot of manpower and time, especially when the scale of evaluation is large, the efficiency problem is particularly prominent. As the demand for talent in enterprises grows, this inefficient method has gradually become a bottleneck. And it is often affected by factors such as the personal bias and experience level of the evaluator, resulting in a strong subjectivity in the evaluation results. This subjectivity may lead to unfair evaluation, which in turn affects the selection of suitable candidates.

[0004] In addition, existing AI evaluation systems are often limited to fixed evaluation indicators and standards, making it difficult to adapt to changing work needs and industry development trends. They often require customized algorithm development and coding, which is not only time-consuming and labor-intensive, but also has poor code reusability and is difficult to meet evaluation needs in different scenarios.

[0005] Therefore, how to design a professional skills assessment method based on a large artificial intelligence model that can cope with more complex talent skill assessment tasks and provide more objective, efficient, personalized and accurate skill assessment is an issue that technical personnel in this field urgently need to solve. Summary of the invention

[0006] In view of this, the present invention provides a professional skills assessment method based on an artificial intelligence big model, which can provide a more objective, efficient, personalized and accurate skills assessment, thereby helping enterprises to better carry out talent selection, training and personal career development planning.

[0007] In order to achieve the above object, the present invention adopts the following technical solution:

[0008] In a first aspect, the present invention provides a method for evaluating professional skills based on an artificial intelligence large model, comprising the following steps:

[0009] S1. Select a large AI model suitable for text understanding and generation for customized training to obtain an optimized model;

[0010] S2. Based on the optimized model, the professional skills evaluation is performed in combination with an adaptive evaluation strategy to obtain the professional skills evaluation results; the adaptive evaluation strategy integrates a Bayesian optimization algorithm, a computer adaptive testing algorithm, a reinforcement learning algorithm, and a multi-armed bandit algorithm;

[0011] S3. Graphically display the professional skills assessment results and generate a professional skills improvement report.

[0012] Furthermore, the S1 includes:

[0013] S11. Obtain professional literature, textbooks and practical cases in related industries to build a professional knowledge dataset;

[0014] S12, preprocessing the professional knowledge data set; the preprocessing includes: data cleaning, error information removal and data labeling;

[0015] S13. Use the preprocessed professional knowledge dataset to conduct customized training to enhance the understanding of professional terminology and actual workflow;

[0016] S14. Based on the customized trained model, construct an evaluation logic; the evaluation logic can score according to the input content of the person being evaluated.

[0017] Furthermore, in S2, the Bayesian optimization algorithm is used for initial question selection and optimization, including:

[0018] Select an initial set of questions covering different difficulty levels;

[0019] Construct a probability model to predict the performance of the test takers on questions of different difficulty levels;

[0020] Based on the performance of the test takers on questions of different difficulty levels, medium-difficulty questions are selected as initial questions, and the probability model parameters are updated.

[0021] Furthermore, in S2, the computer adaptive test algorithm is used to dynamically adjust the difficulty of the questions, including:

[0022] The evaluation begins with the initial questions selected based on the Bayesian optimization algorithm;

[0023] Adjust the difficulty of the questions in real time based on whether the person being tested answered the previous question correctly and the time it took to answer;

[0024] Update the ability estimate of the person being assessed based on their responses to each question;

[0025] Choose questions that are at a difficulty level that is closest to the estimated current ability of the person being assessed.

[0026] Furthermore, in S2, the reinforcement learning algorithm is used to optimize the evaluation process, including:

[0027] Building an intelligent agent that can learn from the performance of the person being assessed in real time during the assessment process and make the best decision to choose the difficulty and type of the next question based on the performance;

[0028] Based on the performance of the person being evaluated on the current question, combined with the selection strategy, it is determined whether to increase or decrease the difficulty of the question; the selection strategy is optimized by maximizing the long-term reward.

[0029] Furthermore, in S2, the multi-armed bandit algorithm is used to balance the question pool, including:

[0030] At the beginning of the assessment, a certain number of questions are randomly selected to assess the initial responses of the assessees;

[0031] Dynamically adjust the topic pool by balancing the effectiveness of known topics with the potential value of exploring new topics;

[0032] Based on the topic pool, topics that effectively distinguish the ability levels of the persons being evaluated are identified and prioritized.

[0033] Furthermore, in S3, a professional skills improvement report is generated, including:

[0034] S31. Collecting evaluation data of the person being evaluated; the evaluation data includes language usage habits, comprehension ability and expression ability;

[0035] S32, using data mining algorithms to analyze the assessment data to identify skill blind spots and learning needs; including: using cluster analysis to identify performance differences of the assessed personnel in different skill areas; using association rule learning to discover potential correlations between the skills of the assessed personnel; using time series analysis to track the changing trends of the assessed personnel's skills over time;

[0036] S33. Generate personalized learning and improvement suggestions through NLP algorithm, combined with industry trends, career goals and personal interests of the person being evaluated.

[0037] In a second aspect, the present invention provides a professional skills assessment system based on an artificial intelligence large model, comprising:

[0038] Customized training module: used to select a large AI model suitable for text understanding and generation for customized training to obtain an optimized model;

[0039] Adaptive evaluation module: used to evaluate the skills of professionals based on the optimized model and in combination with an adaptive evaluation strategy to obtain the professional skills evaluation results; the adaptive evaluation strategy integrates Bayesian optimization algorithm, computer adaptive testing algorithm, reinforcement learning algorithm and multi-armed bandit algorithm;

[0040] Report generation module: used to graphically display the professional skills assessment results and generate professional skills improvement reports.

[0041] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned professional skills assessment method when executing the computer program.

[0042] In a fourth aspect, the present invention provides a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned professional skills assessment method is implemented.

[0043] The descriptions of the second to fourth aspects of the present invention can refer to the detailed description of the first aspect; and the beneficial effects of the descriptions of the second to fourth aspects can refer to the analysis of the beneficial effects of the first aspect, which will not be repeated here.

[0044] It can be seen from the above technical solution that compared with the prior art, the present invention has the following beneficial effects:

[0045] 1. The system improves the model's ability to understand professional fields by collecting professional knowledge datasets in specific fields and fine-tuning the BERT model. It can not only better understand professional terms, but also more accurately assess the professional skill level of the person being evaluated.

[0046] 2. By integrating multiple machine learning algorithms, Bayesian optimization, computer adaptive testing (CAT), reinforcement learning and multi-armed bandit algorithms, it achieves personalized and efficient assessment of the skill level of the person being assessed. It can dynamically adjust the difficulty and type of questions according to the performance of the person being assessed to ensure the accuracy and efficiency of the assessment.

[0047] 3. The report generation module not only provides a graphical display of the assessment results, but also uses data mining technology and natural language processing (NLP) algorithms to generate personalized career development suggestions. Through in-depth analysis of the assessment data, the system can identify the skills blind spots of the assessed personnel and provide targeted learning resources and practice opportunities to help them develop effective personal career development plans. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0049] Figure 1 A flow chart of a professional skills assessment method based on an artificial intelligence large model provided by an embodiment of the present invention;

[0050] Figure 2 A flow chart of a model after optimization provided by an embodiment of the present invention;

[0051] Figure 3 A flow chart of generating a professional skills improvement report provided by an embodiment of the present invention;

[0052] Figure 4 A framework diagram of a professional skills assessment system based on an artificial intelligence large model provided by an embodiment of the present invention;

[0053] Figure 5 A schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] The evaluation method provided in the embodiment of the present application can be applied to an evaluation server. The above-mentioned evaluation server can be hardware or software. When the evaluation server is hardware, it can be implemented as a distributed server cluster that provides evaluation services, or it can be implemented as a single server. When the evaluation server is software, it can be installed in the servers listed above. It can be implemented as multiple software or software modules, or it can be implemented as a single software or software module, which is not specifically limited here.

[0056] Embodiment 1;

[0057] like Figure 1 As shown, this embodiment provides a professional skills assessment method based on an artificial intelligence large model, comprising the following steps:

[0058] S1. Select a large AI model suitable for text understanding and generation for customized training to obtain an optimized model;

[0059] S2. Based on the optimized model, the professional skills evaluation is performed in combination with an adaptive evaluation strategy to obtain the professional skills evaluation results; the adaptive evaluation strategy integrates a Bayesian optimization algorithm, a computer adaptive testing algorithm, a reinforcement learning algorithm, and a multi-armed bandit algorithm;

[0060] S3. Graphically display the professional skills assessment results and generate a professional skills improvement report. This method can provide a more objective, efficient, personalized and accurate skills assessment, thereby helping enterprises to better select talents, train and plan personal career development.

[0061] The above steps and related features are further described in detail below:

[0062] The artificial intelligence model can choose OpenAI's GPT-3 or Google's BERT as the core framework for customized training. This embodiment selects the BERT model, such as Figure 2 As shown, this embodiment S1 includes:

[0063] S11. Obtain professional literature, textbooks and practical cases in related industries to build a professional knowledge dataset;

[0064] S12, preprocessing the professional knowledge data set; the preprocessing includes: data cleaning, error information removal and data labeling;

[0065] S13. Use the preprocessed professional knowledge dataset to perform customized training of the BERT model to enhance the understanding of professional terminology and actual workflow;

[0066] S14. Based on the customized trained BERT model, construct an evaluation logic; the evaluation logic can score according to the input content of the person being evaluated.

[0067] The BERT model is a pre-trained language model that uses a bidirectional Transformer encoder that can read text from front to back and from back to front at the same time. Each word can be encoded based on its context to better understand the meaning of the word.

[0068] In the pre-training stage, BERT uses the masked language model task, which randomly masks a portion of the words in the input text and then trains the model to predict these masked words. This mechanism helps the model learn how to predict missing information based on the context. In addition to MLM, in the pre-training stage, BERT also uses the NSP task to train the model to understand the relationship between two sentences. This helps the model learn sentence-level representations.

[0069] On this basis, this step involves the entire process from collecting professional information to building professional knowledge data sets, to customized training of the model, and finally integrating evaluation logic, aiming to improve the model's understanding and evaluation capabilities in the professional field.

[0070] In this embodiment S2, the Bayesian optimization algorithm is used for initial question selection and optimization, including:

[0071] Select an initial set of questions covering different difficulty levels;

[0072] Construct a probability model to predict the performance of the test takers on questions of different difficulty levels;

[0073] Based on the performance of the test takers on questions of different difficulty levels, medium-difficulty questions are selected as initial questions, and the probability model parameters are updated.

[0074] Computer adaptive testing algorithms are used for dynamic question difficulty adjustment, including:

[0075] The evaluation begins with the initial questions selected based on the Bayesian optimization algorithm;

[0076] Adjust the difficulty of the questions in real time based on whether the person being tested answered the previous question correctly and the time it took to answer;

[0077] Update the ability estimate of the person being assessed based on their responses to each question;

[0078] Choose questions that are at a difficulty level that is closest to the estimated current ability of the person being assessed.

[0079] Reinforcement learning algorithms are used to optimize the evaluation process, including:

[0080] Building an intelligent agent that can learn from the performance of the person being assessed in real time during the assessment process and make the best decision to choose the difficulty and type of the next question based on the performance;

[0081] Based on the performance of the person being evaluated on the current question, combined with the selection strategy, it is determined whether to increase or decrease the difficulty of the question; the selection strategy is optimized by maximizing the long-term reward.

[0082] The multi-armed bandit algorithm is used to balance the question pool, including:

[0083] At the beginning of the assessment, a certain number of questions are randomly selected to assess the initial responses of the assessees;

[0084] Dynamically adjust the topic pool by balancing the effectiveness of known topics with the potential value of exploring new topics;

[0085] Based on the topic pool, topics that effectively distinguish the ability levels of the persons being evaluated are identified and prioritized.

[0086] The adaptive assessment strategy in this embodiment achieves efficient and personalized assessment of professional skills by integrating multiple advanced algorithms. First, a set of initial questions covering different difficulty levels is selected through the Bayesian optimization algorithm, and a probability model is constructed to predict the performance of the person being assessed on questions of different difficulty levels. Based on the initial performance of the person being assessed on these questions, a medium-difficulty question is selected as the initial question, and the probability model parameters are updated. Then, the computer adaptive test algorithm adjusts the difficulty of the question in real time according to the person being assessed's answer to the previous question and the time taken to answer, and updates its ability estimate based on the person being assessed's answer to each question, and selects the question with the difficulty level closest to the current ability estimate of the person being assessed. Subsequently, the reinforcement learning algorithm constructs an intelligent agent, learns the performance of the person being assessed in real time during the assessment process, and makes the best decision to select the difficulty and type of the next question based on this, optimizing the assessment process by maximizing long-term rewards. Finally, the multi-armed bandit algorithm dynamically adjusts the question pool by balancing the effect of known questions and exploring the potential value of new questions, identifying questions that effectively distinguish the ability levels of the person being assessed and giving priority.

[0087] The entire assessment process in this step, from initial question selection to question difficulty adjustment, and then to assessment process optimization and question pool balance, forms a complete set of adaptive assessment strategies to ensure the personalization and efficiency of the assessment.

[0088] like Figure 3 As shown, in this embodiment S3, a professional skills improvement report is generated, including:

[0089] S31. Collecting evaluation data of the person being evaluated; the evaluation data includes language usage habits, comprehension ability and expression ability;

[0090] Language usage: Analyze the grammatical accuracy, vocabulary richness, and discourse coherence of the test taker in the test. Comprehension ability: Assess the test taker's understanding of complex concepts, such as problem-solving ability, reading comprehension, and reasoning skills. Expression ability: Examine the test taker's ability to express ideas clearly, including written and oral communication skills. Specific sources include: Answer records in the test. Results of interactive tasks, such as simulated dialogues, writing samples, etc. Observation records, including non-verbal behavior (such as body language).

[0091] S32, using data mining algorithms to analyze the assessment data to identify skill blind spots and learning needs; including: using cluster analysis to identify performance differences of the assessed personnel in different skill areas; using association rule learning to discover potential correlations between the skills of the assessed personnel; using time series analysis to track the changing trends of the assessed personnel's skills over time;

[0092] S33. Generate personalized learning and improvement suggestions through NLP algorithms, combined with industry trends, career goals, and personal interests of the person being assessed. Generate targeted learning path suggestions, including recommended courses, books, practical projects and other resources. Provide customized training plans to help the person being assessed fill in the skill gaps and strengthen their strengths.

[0093] By integrating data analysis and machine learning technology, it realizes personalized assessment and improvement guidance of the professional skills of the evaluated personnel. It can not only accurately locate the blind spots of individual skills, but also generate customized learning paths based on personal characteristics and industry trends, and combine dynamic feedback mechanisms to help the evaluated personnel efficiently improve their professional capabilities, thereby enhancing their competitiveness in the workplace.

[0094] Embodiment 2;

[0095] In this embodiment, software developers are the test subjects, and the professional skills assessment method based on the BERT model is used to perform professional skills assessment, which specifically includes:

[0096] In this embodiment S1: building and training a customized BERT model;

[0097] S11: Obtain professional literature, textbooks and practical cases in related industries to build professional knowledge datasets

[0098] Dataset construction: Specifically include: collecting books, tutorials, papers, technical blogs and open source project documents about Python programming. Including: Python language basics, data structure, object-oriented programming, Web development framework (such as Django, Flask), database operations, unit testing, etc. You can also collect some actual project cases, such as data analysis projects, machine learning application cases, etc.

[0099] S12: Preprocess the professional knowledge dataset;

[0100] Data preprocessing: remove HTML tags, special characters, non-text content, etc. Identify and correct spelling errors, grammatical errors, etc. Use experts or crowdsourcing to annotate the content in the dataset, such as marking code snippets, comments, function definitions, etc. And introduce tags, such as: CODE: indicates a code paragraph. COMMENT: indicates a comment. FUNCTION: indicates a function definition. EXAMPLE: indicates an example code.

[0101] S13: Use the preprocessed professional knowledge dataset to perform customized training of the BERT model;

[0102] Based on the preprocessed dataset, fine-tune the BERT model to better understand the professional terms and context in the field of Python programming. And define one or more specific tasks. Question answering task: answer questions about Python programming. Code generation task: generate the corresponding Python code based on the question description. Code evaluation task: evaluate the quality of a given code snippet.

[0103] Training process: Divide the data set into training set, validation set and test set. Select appropriate hyperparameters for training. Use cross-validation method to optimize model performance. Monitor the change of loss function during training to prevent overfitting.

[0104] S14: Build evaluation logic based on the customized trained BERT model;

[0105] Input content understanding: Use a customized BERT model to understand the answers submitted by the test takers.

[0106] Define scoring rules: Check if the answer meets the expected results. Evaluate the readability, maintainability, and efficiency of the code. Evaluate if the answer has innovative solutions.

[0107] Define the scoring function and calculate the score according to the above scoring rules. Use the BERT model to extract key information and compare it with the standard answer. Give the corresponding score based on the difference.

[0108] Through the above steps, a customized BERT model for Python programming was built, which can understand and evaluate Python code, and thus provide strong evaluation logic support for the evaluation system. This customized model can help the evaluation system more accurately evaluate the professional skills of the evaluated personnel and provide valuable feedback.

[0109] In this embodiment S2, the professional skills assessment is performed in combination with the adaptive assessment strategy;

[0110] S21: Initial question selection and optimization (Bayesian optimization algorithm);

[0111] The topic pool includes: Beginner topic: Explain what list comprehensions are in Python? Intermediate topic: Write a program that uses list comprehensions to calculate the squares of all even numbers in a list. Advanced topic: Write a program that uses list comprehensions to calculate the squares of all even numbers in a two-dimensional list and return a new two-dimensional list.

[0112] Initial topic selection: Participant A was first assigned a medium-level topic: "Write a program that uses list comprehensions to calculate the squares of all even numbers in a list." Participant A explained how to create a new list that contains the squares of all even numbers in the original list.

[0113] The BERT model evaluates that the response of test subject A is correct, grammatically clear, and consistent with the expected answer. The probability model updates the ability estimate of test subject A based on his response, showing that he performs well at the intermediate level.

[0114] S22: Dynamic question difficulty adjustment (computer adaptive testing algorithm);

[0115] Since test subject A performed well on the intermediate difficulty level, he was given a question of advanced difficulty: "Write a program that uses list comprehension to calculate the squares of all even numbers in a two-dimensional list and return a new two-dimensional list."

[0116] Subject A described an algorithmic step that uses list comprehension to calculate the squares of all even numbers in a two-dimensional list and return a new two-dimensional list.

[0117] The BERT model evaluated that the answer of test subject A was correct, but pointed out that there was room for improvement in the coding style, such as adding comments and naming variables more standardized. The estimated ability of test subject A was further improved, but at the same time it was suggested that there was room for improvement.

[0118] S23: Evaluation process optimization (reinforcement learning algorithm);

[0119] The intelligent agent decided to keep the difficulty level of the next question high, but add more detail requirements, such as code comments and variable naming.

[0120] Topic: "Write a program that uses list comprehensions to compute the squares of all even numbers in a two-dimensional list and return a new two-dimensional list. Make sure your code has proper comments and meaningful variable names."

[0121] Participant A described in detail how to create a function that takes a two-dimensional list and returns a new list containing the squares of all the even numbers in the original list. He also explained how to add comments and choose meaningful variable names.

[0122] The BERT model evaluates that the answer of test subject A is correct and the coding style has been significantly improved. Combined with long-term reward maximization: The intelligent agent records this improvement and prioritizes the coding style requirements in similar future evaluations.

[0123] S24: Question Pool Balance (Multi-Armed Bandit Algorithm)

[0124] After several rounds of testing, the system found that test subject A performed very well in list derivation, so it reduced the proportion of such questions in the question pool and added other types of questions, such as function definition, exception handling, etc.

[0125] Next, test subject A was assigned a function definition question: "Define a function that receives a string parameter and returns a reversed version of the string."

[0126] Test subject A answers correctly, and the system continues to evaluate and adjust the question pool.

[0127] In this embodiment S3, the test subject A performs well in list derivation, but needs to be strengthened in function definition and exception handling. Based on the performance of the test subject A, he is advised to further learn the knowledge points of function definition and exception handling, and relevant learning resource links are provided.

[0128] Through this embodiment, the evaluation method can automatically adjust the difficulty of the questions according to the actual performance of the testee, and ensure the effectiveness and pertinence of the evaluation process through multiple algorithms. It helps the testee to accurately understand his or her skill level and obtain personalized learning suggestions to improve his or her professional skills.

[0129] Embodiment 3;

[0130] like Figure 4 As shown, this embodiment provides a professional skills assessment system based on an artificial intelligence big model, including:

[0131] Customized training module: used to perform customized training on the BERT model to obtain an optimized BERT model;

[0132] Adaptive evaluation module: Based on the optimized BERT model, the adaptive evaluation strategy is combined to evaluate the professional skills and obtain the professional skills evaluation results; the adaptive evaluation strategy integrates Bayesian optimization algorithm, computer adaptive testing algorithm, reinforcement learning algorithm and multi-armed bandit algorithm;

[0133] Report generation module: graphically displays the professional skills assessment results and generates a professional skills improvement report.

[0134] Embodiment 4;

[0135] like Figure 5 As shown, this embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the professional skills assessment method of the above embodiment is implemented.

[0136] Embodiment 5;

[0137] This embodiment provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the professional skills assessment method of the above embodiment is implemented.

[0138] In the above embodiments provided in the present application, it should be understood that the disclosed methods, systems, devices and media can be implemented in other ways. The above described methods, systems, devices and media embodiments are only schematic. For example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation. Each functional unit can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0139] The units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0140] The computer program includes computer program code, which may be in source code form, object code form, executable file or some intermediate form, etc. Computer readable media may include: any entity or device capable of carrying computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory, random access memory, electric carrier signal, telecommunication signal and software distribution medium, etc.

[0141] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A professional skills assessment method based on an artificial intelligence large model, characterized in that: The following steps are involved: S1. Select a large AI model suitable for text understanding and generation for customized training to obtain an optimized model; S2. Based on the optimized model, the professional skills evaluation is performed in combination with an adaptive evaluation strategy to obtain the professional skills evaluation results; the adaptive evaluation strategy integrates a Bayesian optimization algorithm, a computer adaptive testing algorithm, a reinforcement learning algorithm, and a multi-armed bandit algorithm; S3. Graphically display the professional skills assessment results and generate a professional skills improvement report.

2. A professional skills assessment method based on an artificial intelligence large model according to claim 1, characterized in that: Said S1 comprises: S11. Obtain professional literature, textbooks and practical cases in related industries to build a professional knowledge dataset; S12, preprocessing the professional knowledge data set; the preprocessing includes: data cleaning, error information removal and data labeling; S13. Use the preprocessed professional knowledge dataset to conduct customized training to enhance the understanding of professional terminology and actual workflow; S14. Based on the customized trained model, construct an evaluation logic; the evaluation logic can score according to the input content of the person being evaluated.

3. A professional skills assessment method based on artificial intelligence big model according to claim 1, characterized in that: In S2, the Bayesian optimization algorithm is used for initial question selection and optimization, including: Select an initial set of questions covering different difficulty levels; Construct a probability model to predict the performance of the test takers on questions of different difficulty levels; Based on the performance of the test takers on questions of different difficulty levels, medium-difficulty questions are selected as initial questions, and the probability model parameters are updated.

4. The method for evaluating professional skills based on an artificial intelligence large model according to claim 1 is characterized in that: In S2, the computer adaptive test algorithm is used to dynamically adjust the difficulty of the questions, including: The evaluation begins with the initial questions selected based on the Bayesian optimization algorithm; Adjust the difficulty of the questions in real time based on whether the person being tested answered the previous question correctly and the time it took to answer; Update the ability estimate of the person being assessed based on their responses to each question; Choose questions that are at a difficulty level that is closest to the estimated current ability of the person being assessed.

5. The method for evaluating professional skills based on an artificial intelligence large model according to claim 1 is characterized in that: In S2, the reinforcement learning algorithm is used to optimize the evaluation process, including: Building an intelligent agent that can learn from the performance of the person being assessed in real time during the assessment process and make the best decision to choose the difficulty and type of the next question based on the performance; Based on the performance of the person being evaluated on the current question, combined with the selection strategy, it is determined whether to increase or decrease the difficulty of the question; the selection strategy is optimized by maximizing the long-term reward.

6. The method for evaluating professional skills based on an artificial intelligence large model according to claim 1 is characterized in that: In S2, the multi-armed bandit algorithm is used to balance the question pool, including: At the beginning of the assessment, a certain number of questions are randomly selected to assess the initial responses of the assessees; Dynamically adjust the topic pool by balancing the effectiveness of known topics with the potential value of exploring new topics; Based on the topic pool, topics that effectively distinguish the ability levels of the persons being evaluated are identified and prioritized.

7. The method for evaluating professional skills based on an artificial intelligence large model according to claim 1, characterized in that: In S3, a professional skills improvement report is generated, including: S31. Collecting evaluation data of the person being evaluated; the evaluation data includes language usage habits, comprehension ability and expression ability; S32, using data mining algorithms to analyze the assessment data to identify skill blind spots and learning needs; including: using cluster analysis to identify performance differences of the assessed personnel in different skill areas; using association rule learning to discover potential correlations between the skills of the assessed personnel; using time series analysis to track the changing trends of the assessed personnel's skills over time; S33. Generate personalized learning and improvement suggestions through NLP algorithm, combined with industry trends, career goals and personal interests of the person being evaluated.

8. A professional skills assessment system based on an artificial intelligence large model, characterized in that: include: Customized training module: used to select a large AI model suitable for text understanding and generation for customized training to obtain an optimized model; Adaptive evaluation module: used to evaluate the skills of professionals based on the optimized model and in combination with an adaptive evaluation strategy to obtain the professional skills evaluation results; the adaptive evaluation strategy integrates Bayesian optimization algorithm, computer adaptive testing algorithm, reinforcement learning algorithm and multi-armed bandit algorithm; Report generation module: used to graphically display the professional skills assessment results and generate professional skills improvement reports.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the professional skills assessment method as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the professional skills assessment method as described in any one of claims 1 to 7 is implemented.

Citation Information

Cited By

  • AI-based enterprise diagnosis report generation method and system

    CN119228107A

  • An AI-based enterprise diagnostic report generation method and system

    CN119228107B

  • Task matching method and system

    CN119272070A

  • Personalized training and self-adaptive evaluation method and system for operating personnel based on AI (artificial intelligence) large model

    CN120634492A

  • Job personnel individualized training and adaptive evaluation method and system based on AI large model

    CN120634492B