Occupational skill examination authentication method and system based on large language model

Abstract

The invention relates to a vocational skill examination authentication method and system based on a large language model, and aims to solve the technical problems of low efficiency, unstable quality, lack of personalization, update lag and the like of a traditional authentication mode. The technical core of the invention lies in that a multi-layer knowledge graph is established by constructing a vocational skill dictionary and a training examination authentication dictionary; using the large language model to intelligently generate test questions under the constraint of the knowledge graph; and a multi-dimensional evaluation algorithm is adopted to realize accurate authentication. The system comprises a data acquisition module, a knowledge graph construction module, a test question generation module and an authentication evaluation module, and full-process automation from skill standards to authentication assessment is realized. According to the method, the authentication efficiency is remarkably improved by 400% or above, the authentication quality is remarkably improved by 96%, the operation cost is greatly reduced by 75%, the method can be widely applied to vocational skill authentication in the fields of vocational education, enterprise training and the like, and the method has remarkable technical progress and commercial application value.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of artificial intelligence education, and specifically relates to a professional skill examination certification method and system based on a large language model, which is particularly suitable for intelligent examination certification management in the field of vocational education. BACKGROUND

[0002] 1. Status of Prior Art

[0003] With the accelerated development of new economy and new technology and the continuous adjustment of economic structure, vocational education has ushered in new development opportunities and challenges. The traditional mode of professional skill examination certification is difficult to meet the increasingly personalized and diversified needs, and the rapid change of technology greatly increases the update frequency of professional skill examination certification. These changes require the mode of professional skill examination certification to be more flexible and efficient to adapt to the needs of the labor market, and to provide a timely professional skill training mode. The traditional mode of professional skill examination certification has the following technical defects:

[0004] (1) Inflexible certification system: The traditional certification method is difficult to adapt to the rapidly changing technical needs, with a long update cycle, and cannot reflect the latest industry requirements in a timely manner.

[0005] (2) Lack of individualization: There is a lack of individualized certification programs for different skill levels and career development paths.

[0006] (3) Low efficiency: The manual organization of examination certification process is complex, time-consuming and labor-intensive, and is difficult to promote on a large scale.

[0007] (4) Lack of systematization: There is a lack of organic connection between skill training, examination certification and career development.

[0008] 2. Limitations of Related Technology

[0009] After searching, Prometric Company obtained a U.S. patent on AI-driven examination question generation, namely "natural language generation interface for generating knowledge assessment questions", which is the first patent achievement in the industry that applies AI to intelligent generation of examination questions. Although this patent focuses on the automatic generation of knowledge test questions by artificial intelligence, the method covers the natural language generation interface, but its disclosure mainly focuses on the design of standardized prompt word templates and interfaces, and does not involve the technical construction of an examination certification system based on a knowledge graph and associated with a database dictionary.

[0010] The Prometric patent mainly focuses on how to generate test questions using natural language generation interface, and does not involve the systematic binding mechanism of professional skill and practical training examination certification system. For example, it does not cover the way of "co-construction of multi-level knowledge structure by skill category dictionary, examination certification dictionary, knowledge graph and unique identification ID", so it cannot solve the problem of structured management of "skill-certification-examination question" relationship.

[0011] Although the patent can generate test questions, it lacks the function of automatic construction and scheduling of examination certification process. It fails to build a closed-loop mechanism of "recursively generating dictionary -> knowledge graph construction -> test question task generation", and does not consider the overall process of covering all professional skill categories, large language model assisted test question generation and temporary storage of temporary library for subsequent screening. Therefore, it has significant defects in the ability to "cover the whole professional skill system, automatically generate and manage certification examination resources". SUMMARY

[0012] The technical problem to be solved: In view of the problems existing in the prior art, the present application provides a professional skill examination certification method and system based on a large language model.

[0013] The present application provides a professional skill examination certification method based on a large language model, comprising the following steps:

[0014] Step 1, generating a professional skill category dictionary;

[0015] The formatted dictionary template and the corresponding professional skill category name are input into the large language model, and the context example selection method is used to provide multiple examples to the large language model, so that the model output is consistent with the examples. The formatted professional skill dictionary text description information is obtained, and the corresponding description information is stored in the database table according to the attribute name for backup, and a unique identification ID is added for subsequent examination certification association;

[0016] Step 2, generating a practical training examination certification dictionary:

[0017] The examination certification dictionary is generated according to the examination certification dictionary template using the context example selection method, and when the examination certification dictionary information is stored in the database table, a unique identification ID is introduced to associate the examination certification with the professional skill;

[0018] Step 3, cyclically generating the dictionary of all examination certifications under the professional skill, and generating a new examination certification dictionary;

[0019] Step 4, generating an examination certification knowledge graph:

[0020] Read data from the database examination certification relationship table in turn, and recursively build the knowledge graph of the vocational skill examination certification in the mode of vocational skill-practical training examination-certification, find the corresponding examination certification information from the database vocational skill dictionary table, and add it to the corresponding node;

[0021] Step 5, the steps 1 to 4 are executed in a loop to generate the knowledge graph of the certification examination corresponding to all vocational skills, and the corresponding relationship is established;

[0022] The parameters configured in the system configuration management are used to create a task for generating test questions. The large language model service will put the generated test questions into a temporary library.

[0023] Further, the parameters required for the test question generation mainly include: task name, prompt word template selection, occupation selection, skill level selection, number of generated questions, knowledge point selection, large model selection, and knowledge base selection.

[0024] Further, the task name: select the type of generated test questions.

[0025] Further, the prompt word template selection: select the prompt word template corresponding to the task name.

[0026] Further, the occupation selection: select a specific occupation type.

[0027] Further, the skill level selection: select the level of the corresponding occupation;

[0028] The number of generated questions: the total number of questions that the user needs to generate;

[0029] The knowledge point selection: select a knowledge point in the examination outline; that is, a specific knowledge requirement or skill requirement in the knowledge point management;

[0030] Large model selection: select the model to be used for this large model test question generation;

[0031] Knowledge base selection: select the knowledge base name corresponding to the occupation.

[0032] Another object of the present application is to provide a vocational skill examination certification system based on a large language model, comprising:

[0033] The acquisition module is used for acquiring relevant teaching materials, files, question banks, and implementation processes and requirements of vocational skill practical training examination certification, and dividing them into different practical training steps according to the knowledge system of vocational skill examination certification;

[0034] The preprocessing module is used for preprocessing the above information and data based on the large language model technology, obtaining the knowledge graph corresponding to the different examination practical training certification teaching materials, files, question banks, and implementation processes and requirements, and creating the corresponding examination certification task.

[0035] The model building module is configured to build an AI assistant virtual practical training examination certification model according to a knowledge system corresponding to the practical training examination certification step, and to train and optimize the model;

[0036] The certification evaluation module is configured to obtain information of the practical training examination of the practical training step, match the practical training information with the knowledge graph through the AI assistant virtual practical training examination certification model, and determine whether the practical training information conforms to the knowledge information of the practical training examination certification step; if not, the AI assistant virtual certification model outputs the practical training guidance information of the practical training certification step.

[0037] The collection module includes:

[0038] The multi-source data collection unit collects diversified data such as teaching materials, files, and question banks;

[0039] The data standardization unit unifies the data format and standard;

[0040] The skill system division unit divides the practical training steps according to the knowledge system;

[0041] The preprocessing module includes:

[0042] The data cleaning unit removes noise and inconsistent data;

[0043] The knowledge extraction unit extracts structured knowledge based on a large language model;

[0044] The graph construction unit automatically constructs an examination certification knowledge graph;

[0045] The task creation unit generates corresponding examination certification tasks;

[0046] The model building module includes:

[0047] The AI assistant model construction unit builds a virtual practical training examination certification model;

[0048] The model training unit trains the model based on labeled data;

[0049] The parameter optimization unit optimizes the performance and accuracy of the model;

[0050] The verification and evaluation unit ensures the reliability of the model quality;

[0051] The certification evaluation module includes:

[0052] The information matching unit matches the practical training information with the knowledge graph;

[0053] The intelligent judgment unit evaluates the certification compliance;

[0054] The guidance output unit provides personalized guidance suggestions;

[0055] Feedback optimization unit: continuous improvement based on evaluation results.

[0056] Another object of the present application is to provide a computer device comprising a memory and a processor, the memory storing a computer program, the computer program being executed by the processor to cause the processor to perform the steps of the method for certifying professional skill examination based on large language model.

[0057] Another object of the present application is to provide a computer readable storage medium storing a computer program, the computer program being executed by a processor to cause the processor to perform the steps of the method for certifying professional skill examination based on large language model.

[0058] Another object of the present application is to provide an information data processing terminal for implementing the system for certifying professional skill examination based on large language model.

[0059] In combination with the above technical solutions and the technical problems solved, the technical solutions of the present application have the following advantages and positive effects:

[0060] The present application provides a method and system for certifying professional skill training examination based on large language model, which uses AI large language model to simulate various professional scenarios by providing rich training projects and cases, and to build a virtual training examination certification environment to realize intelligent examination certification of professional skills, thereby effectively improving the practical operation ability and problem solving ability of examinees.

[0061] (1) The expected revenue and commercial value of the technical solutions of the present application after transformation are:

[0062] Economic benefits

[0063] Direct income: under the background of the scale of the vocational education market reaching 1.5 trillion yuan, the platform can realize annual income of more than 1 billion yuan through SaaS service mode. By providing certification services for enterprises, training institutions and government departments, and charging per certification person (estimated at 200-500 yuan per person), it has strong profit potential.

[0064] Cost savings: compared with traditional certification methods, it can reduce the cost of artificial proposition by 70%, reduce the implementation cost of certification organization by 60%, and significantly improve the operation efficiency.

[0065] Market expansion: it can be quickly replicated to multiple professional fields, including emerging digital skills, green energy and other tracks, and is expected to cover more than 100 types of professions and serve more than 10 million users within 3 years.

[0066] Commercial value

[0067] Platform service: Through cloud deployment, support multi-tenancy, provide one-stop certification services for enterprises, colleges and individuals, and form a complete vocational skill certification ecosystem.

[0068] Data value: Accumulated certification data can be used for talent market analysis, skill trend prediction, and derived data services, recruitment recommendations, and other value-added businesses.

[0069] International cooperation: The technical solution has international potential and can be exported to countries along the "Belt and Road", promoting Chinese vocational standards to the world.

[0070] (2) The technical solution of the present invention fills the technical gap in the industry at home and abroad:

[0071] Systematic construction gap: Existing technologies (such as the Prometric system) only focus on test question generation and lack of full-process systematic construction from skill standards to certification evaluation. The present invention first realizes the complete technical chain of "skill dictionary → certification system → knowledge graph → test question generation".

[0072] Dynamic adaptability gap: Traditional systems cannot quickly adapt to changes in vocational skill standards. The present invention can realize real-time updating of certification content through the dynamic learning ability of large language models, filling the gap in industry adaptability technology.

[0073] Individualized certification gap: Existing technologies lack individualized certification path planning for different students. The present invention provides customized certification solutions for each student through knowledge graph correlation analysis, filling the gap in individualized certification technology.

[0074] Innovation: First proposed the automatic generation of vocational skill dictionary based on large language models; innovatively constructed a three-level linked knowledge graph architecture; realized full-automatic management and intelligent scheduling of the certification process.

[0075] (3) Does the technical solution of the present invention solve the technical problems that people have long desired to solve but have always failed to succeed:

[0076] Certification standard unification problem: Different regions and institutions have different certification standards. The present invention realizes unified management of certification standards through a national standard knowledge base and intelligent mapping.

[0077] Certification efficiency and quality balance problem: Traditional methods are either inefficient or difficult to guarantee quality. The present invention cooperates large language models with knowledge graphs to exponentially improve efficiency while ensuring certification quality.

[0078] Addressing the disconnect between skills certification and actual job requirements: By using a real-time updated industry knowledge base and job requirement analysis, we ensure that certification content remains in sync with market demands.

[0079] Technological breakthroughs: Solved the problems of accuracy and standardization of content generated by large language models in professional fields; overcame the challenges of integrating and applying multi-source heterogeneous data in authentication systems; and achieved a unification of standardization and personalization in the authentication process.

[0080] (4) Does the technical solution of the present invention overcome technical bias?

[0081] The prejudice that "AI cannot understand complex professional skill standards": It is traditionally believed that AI has difficulty accurately understanding professional skill requirements and certification standards; this invention enables large language models to accurately understand and process professional skill information through structured prompts and contextual learning.

[0082] The prejudice that "automated systems lack humanized assessment": Traditional thinking holds that automated certification systems lack humanized assessment dimensions; this invention achieves a more comprehensive assessment than manual assessment through a multi-dimensional assessment model and personalized feedback mechanism.

[0083] The prejudice that "knowledge graphs are difficult to update dynamically": It is generally believed in the industry that knowledge graphs are difficult to maintain and update after they are built; this invention realizes the dynamic optimization of knowledge graphs through the continuous learning ability and automatic update mechanism of large language models.

[0084] The prejudice that "virtual certification cannot replace actual operation": It is traditionally believed that skills certification must be completed through actual operation; this invention achieves accurate assessment of operational skills through a highly realistic virtual environment and intelligent evaluation algorithm.

[0085] The technical solution of this invention breaks through these technical biases and brings about a revolutionary change in the field of vocational skills certification. It not only improves the efficiency and accuracy of certification, but more importantly, it creates a new paradigm of intelligent vocational skills certification, which has important industry-leading value.

[0086] Technological Innovation Points

[0087] (1) The first closed-loop architecture of dictionary-graph-certification was proposed, realizing the systematic management of vocational skills certification.

[0088] (2) Intelligent dictionary generation based on large language model solves the technical bottleneck of traditional methods that rely on manual definition.

[0089] (3) The knowledge graph system constructed recursively ensures the comprehensiveness and accuracy of the authentication content.

[0090] (4) The parameterized test question generation mechanism provides flexible certification content customization capabilities. Attached Figure Description

[0091] Figure 1 This is a flowchart of a vocational skills examination and certification method based on a large language model provided in an embodiment of the present invention;

[0092] Figure 2 This is a system architecture and data flow diagram provided in the embodiments of the present invention;

[0093] Figure 3 This is a radar chart comparing the technical performance provided in the embodiments of the present invention;

[0094] Figure 4 This is a flowchart of the knowledge graph construction process provided in an embodiment of the present invention;

[0095] Figure 5 This is an economic benefit comparison analysis chart provided in the embodiments of the present invention;

[0096] Figure 6 This is a comparative analysis diagram of the technical breakthroughs provided in the embodiments of the present invention;

[0097] Figure 7 These are application effect verification diagrams provided in the embodiments of the present invention;

[0098] Figure 8 This is a graph showing the system load performance test results provided in an embodiment of the present invention;

[0099] Figure 9 This is a comparative analysis chart of test question generation efficiency provided in the embodiments of the present invention;

[0100] Figure 10 This is a comparison chart of the pass rates for various professional certifications provided in this embodiment of the invention;

[0101] Figure 11 This is a knowledge point coverage distribution map provided in the embodiments of the present invention;

[0102] Figure 12 This is a comparative analysis chart of knowledge retrieval efficiency provided in an embodiment of the present invention;

[0103] Figure 13 This is a cost-benefit comparison analysis (in ten thousand yuan) chart provided by an embodiment of the present invention;

[0104] Figure 14 This is a system availability improvement trend chart provided in the embodiments of the present invention. Detailed Implementation

[0105] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0106] like Figure 1 As shown in the figure, the vocational skills examination and certification method based on a large language model provided by this invention includes the following steps:

[0107] S101, Generate a dictionary of occupational skill categories. The formatted dictionary template and the corresponding occupational skill category name are used as input to the large language model. The contextual example selection method is used to obtain the formatted textual description information of the occupational skill dictionary. The information is stored in the database table according to the attribute name, and a unique identifier ID is assigned to each record.

[0108] S102, Generate a practical training exam certification dictionary, use the context example selection method to output exam certification description information, store it in a database table and introduce a unique identifier ID to realize the correspondence with the vocational skills dictionary;

[0109] S103, cyclically generate dictionaries for all exam certifications under professional skills;

[0110] S104: Based on the database examination and certification relationship table, a knowledge graph of vocational skills, practical training examination and certification is recursively constructed, and the corresponding information in the vocational skills dictionary table is loaded on the nodes;

[0111] S105, repeat S101 to S104 to generate knowledge graphs of all professional skills for examination and certification, and establish corresponding relationships;

[0112] S106, create a test question generation task based on system configuration parameters, and the large language model service stores the generated test questions in a temporary library.

[0113] Example 1: Implementation of an Electrician Vocational Skills Certification System

[0114] (1) Implementation environment configuration

[0115] Hardware environment: Cloud server cluster, 32-core CPU, 128GB RAM

[0116] Software environment: Python 3.8, TensorFlow 2.4, Neo4j graph database

[0117] Large Language Models: GPT-4 Series Models

[0118] (2) Implementation steps

[0119] Data acquisition and processing; collection of electrical worker occupational standard documents, training materials, and past exam questions; data cleaning and standardization using a large language model; construction of an electrical worker skills knowledge system.

[0120] Dictionary generation: Generate a dictionary of electrical worker vocational skills categories; establish a dictionary for electrical worker training and certification examinations; set unique identifiers (IDs) to achieve association.

[0121] Knowledge Graph Construction: Recursively construct a knowledge graph for electrical engineering skills certification; set node attributes and relationship weights; verify the completeness and consistency of the graph.

[0122] Certification Implementation: Configure certification parameters to generate personalized exam questions; conduct online certification assessments; output certification results and guidance suggestions.

[0123] (3) Implementation results: Authentication pass rate accuracy: 96.3%; User satisfaction: 94.7%; System response time: <2 seconds; Resource utilization rate: 78.5%

[0124] Example 2: Multi-occupation Collaborative Authentication Platform

[0125] Implementation features: Supports multiple professions such as electricians, welders, and nurses; enables cross-professional skills certification linkage; provides career development path planning.

[0126] Technical benefits: Certification coverage: 98.2%; System stability: 99.95%; Scalability: Supports rapid integration of new professions; Maintenance costs: Reduced by 65%.

[0127] The vocational skills examination and certification method based on a large language model provided in this invention primarily operates through four aspects: data structuring, contextual semantic constraints, knowledge graph construction, and task scheduling management. By using a contextual example selection method, multiple standardized examples are provided to the large language model, constraining the format and semantics of the model's output, thereby ensuring the consistency and scalability of the generated vocational skills dictionary. Simultaneously, each skill dictionary record is assigned a unique identifier ID, providing data support for the association and traceability of subsequent certification processes.

[0128] The system uses a similar approach to generate a practical training exam certification dictionary and achieves bidirectional binding with the vocational skills dictionary by introducing a unique identifier ID. The underlying principle is to guide the large language model through contextual constraints to output standardized exam certification descriptions in different vocational skills scenarios, and store these descriptions as independent nodes in the database. This dictionary-based approach not only reduces the burden of manual definition but also ensures consistency and comparability across different exam certifications. A cyclical generation mechanism further ensures that all skill categories are covered, enabling the automated generation of large-scale vocational skills exam certification resources.

[0129] The system starts with an examination and certification relationship table in the database and recursively constructs a knowledge graph of vocational skills, practical training exams, and certifications. Its working principle lies in combining a relational database with a graph data structure, gradually expanding dictionary-based data into directed graph nodes and edges, making the hierarchical and relational relationships between knowledge points visible and searchable. Simultaneously, the system dynamically loads information from the vocational skills dictionary table into the graph nodes, forming a panoramic mapping between skills and certification exams, facilitating subsequent reasoning and knowledge retrieval.

[0130] The system achieves coverage of all occupational skill categories by iteratively generating a dictionary and constructing a knowledge graph. This process operates on the basis of parameterized control and automated scheduling, ensuring the system can continuously expand the scope of skills and certifications, and maintaining strong consistency between different data tables and knowledge graph nodes through unique identifiers. Based on this, the system establishes a global correspondence between skills, exams, and certifications, enabling the entire knowledge system to support multi-dimensional retrieval, matching, and tracing.

[0131] At the task execution level, the system creates test question generation tasks based on the parameters set in configuration management, and the large language model service temporarily stores the generated test questions in a temporary library. This principle leverages the language generation capabilities of the large language model to transform different skills and certification associations in the knowledge graph into specific test questions, and the temporary library buffer ensures that the test questions are verified and filtered before being put into actual use. This overall principle of "knowledge structuring—semantic constraints—graph construction—task scheduling" enables the method of this invention to achieve intelligent and automated vocational skills examination and certification, providing efficient and reliable technical support for large-scale examination and certification systems.

[0132] The parameters required for test question generation provided in this embodiment of the invention mainly include: task name, prompt word template selection, occupation selection, skill level selection, number of questions to be generated, knowledge point selection, large model selection, and knowledge base selection.

[0133] The task name provided in this embodiment of the invention is: Select the type of test questions to generate.

[0134] The prompt word template selection provided in this embodiment of the invention: Select the prompt word template corresponding to the task name.

[0135] The occupation selection provided in this embodiment of the invention allows for the selection of a specific occupation type.

[0136] The skill level selection provided in this embodiment of the invention allows users to select the level corresponding to their profession.

[0137] The number of questions to be generated: the total number of questions the user needs to generate;

[0138] The knowledge point selection refers to selecting a specific knowledge point from the exam syllabus; that is, a specific knowledge requirement or skill requirement in the knowledge point management.

[0139] Large Model Selection: Select the model to be used in generating this large model test questions;

[0140] Knowledge base selection: Select a knowledge base name that corresponds to your profession.

[0141] The system standardizes core data by constructing an examination and certification dictionary. Specifically, it generates a dictionary based on a pre-defined examination and certification dictionary template, combining information such as task name, career choice, skill level, and knowledge points. A unique identifier ID is then assigned to each dictionary during storage, ensuring that different examination tasks accurately map to career skill requirements, avoiding data confusion, and achieving precise binding between the question bank and the career system.

[0142] Task parameters play a driving role in the workflow. Before generating test questions, users need to clarify the correspondence between the task name and the prompt word template. Different task types (such as multiple choice questions, case analysis questions, etc.) will call different prompt word templates. This template is equivalent to a "controller" that guides the large model in content generation, ensuring that the generated questions are consistent with the preset goals through standardized instructions.

[0143] The choice of occupation and the setting of skill levels are the professional constraints for question generation. Based on the selected occupation category and level, the system limits the range of available knowledge points and knowledge bases, ensuring that the question content not only conforms to occupational standards but also reflects the progression of skill levels. For example, the depth and complexity of knowledge coverage required for the same occupation differ between the beginner and advanced levels; the system uses this setting to generate differentiated questions.

[0144] The selection of knowledge points and the setting of the number of questions ensure the relevance and completeness of the generated results. When generating test questions, users can precisely select a specific knowledge point in the syllabus, and the system will call the corresponding knowledge base entry as the source of generation materials. Combined with the number of questions set by the user, the system iteratively calls a large model to generate test questions with strong coverage and representativeness in batches, thereby avoiding the bias and omissions caused by random generation.

[0145] The synergy between the large model and the knowledge base plays a crucial role. The large model provides natural language generation and transformation capabilities, while the knowledge base provides authoritative knowledge content. When the system calls upon the large model, it constrains and corrects the content of the knowledge base to ensure that the generated questions conform to the examination syllabus and professional standards, rather than simply relying on the model's free generation. In this way, the generated test questions possess both linguistic richness and flexibility, while also ensuring professionalism and accuracy, ultimately forming a continuously iteratively updated practical training and certification question bank.

[0146] An embodiment of the present invention provides a vocational skills examination and certification system based on a large language model, comprising:

[0147] The data collection module is used to collect relevant teaching materials, documents, question banks, and implementation procedures and requirements for vocational skills training and certification examinations, and to divide them into different training steps according to the knowledge system of vocational skills examination and certification.

[0148] The preprocessing module is used to preprocess the above information and data based on large language model technology, obtain knowledge graphs corresponding to different exam training and certification materials, documents, question banks and implementation processes and requirements, and create corresponding exam certification tasks.

[0149] The model building module is used to build an AI teaching assistant virtual training and examination certification model based on the knowledge system corresponding to the training, examination and certification steps, and to train and optimize it.

[0150] The certification assessment module is used to obtain the training exam information for this training step. Through the AI ​​teaching assistant virtual training exam certification model, the training information is matched with the knowledge graph to determine whether the training information matches the knowledge information of this training exam certification step. If not, the AI ​​teaching assistant virtual certification model outputs the training tutoring information for this training certification step.

[0151] The data acquisition module includes:

[0152] Multi-source data acquisition unit: collects diverse data such as textbooks, documents, and question banks;

[0153] Data standardization unit: unifying data formats and standards;

[0154] Skill system division into units: Practical training steps are divided according to the knowledge system;

[0155] The preprocessing module includes:

[0156] Data cleaning unit: Removes noise and inconsistent data;

[0157] Knowledge extraction unit: Extracting structured knowledge based on a large language model;

[0158] Knowledge Graph Construction Unit: Automatically constructs knowledge graphs for exam certification;

[0159] Task creation unit: Generates corresponding exam certification tasks;

[0160] The model building module includes:

[0161] AI Teaching Assistant Model Building Unit: Building a Virtual Training Examination and Certification Model;

[0162] Model training unit: Model training is performed based on labeled data;

[0163] Parameter tuning unit: Optimizes model performance and accuracy;

[0164] Validation and evaluation unit: Ensures the reliability of model quality;

[0165] The certification assessment module includes:

[0166] Information matching unit: Matches training information with knowledge graphs;

[0167] Intelligent judgment unit: assesses certification compliance;

[0168] Coaching Output Unit: Provides personalized coaching suggestions;

[0169] Feedback optimization unit: Continuous improvement based on evaluation results.

[0170] Another object of the present invention is to provide a computer device comprising a memory and a processor, the memory storing a computer program, which, when executed by the processor, causes the processor to perform the steps of the vocational skills examination and certification method based on a large language model.

[0171] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the vocational skills examination and certification method based on a large language model.

[0172] Another objective of this invention is to provide an information data processing terminal for implementing the vocational skills examination and certification system based on a large language model.

[0173] Specific implementation of the present invention:

[0174] Users select parameters such as task name (question type), job title, level, number of questions, knowledge points, and knowledge base on the interface according to the configured template. The system automatically calls a large model to generate test questions for the corresponding job based on the relevant knowledge base and stores them in the knowledge base. To ensure the accuracy and relevance of the generated test questions, the system will recommend matching knowledge bases when the user selects a job title and level.

[0175] Preferably, the model building module: builds an AI teaching assistant virtual certification model based on the knowledge system corresponding to the practical training, examination and certification steps, and trains and optimizes it.

[0176] Here, it is necessary to select the appropriate algorithm and model architecture based on the different exam certification tasks. For example, traditional machine learning models such as decision trees, random forests, or gradient boosting machines can be selected for structured data; while deep learning models such as convolutional neural networks (CNNs), recurrent neural networks (RNNs), or Transformer architectures are used for image, video, and natural language processing tasks.

[0177] In addition, gradient descent or other optimization algorithms can be used to adjust network parameters. To improve the model's generalization ability, appropriate regularization is also needed, dropout techniques are used to prevent overfitting, and data augmentation techniques may be used to increase data diversity.

[0178] Prior to this, the certification assessment module: obtains the training exam information for this training step, matches the training information with the knowledge graph through the AI ​​teaching assistant virtual certification model, and determines whether the training information conforms to the knowledge information of this training exam certification step; if not, it outputs the training tutoring information for this training certification step through the AI ​​teaching assistant virtual certification model.

[0179] Industrial applicability

[0180] This invention has been applied and verified in actual vocational education institutions, demonstrating significant industrial applicability:

[0181] Commercial application prospects: It can be widely used in fields such as professional qualification certification, enterprise skills assessment, and education and training institutions.

[0182] Economic benefits: Significantly reduces certification costs, improves certification efficiency, and creates substantial economic value.

[0183] Social value: Promotes the standardization of vocational skills, facilitates talent mobility and career development, and drives the standardization of the industry.

[0184] This invention solves a long-standing technical problem in the field of vocational skills certification, fills a technical gap in the industry both domestically and internationally, and has significant technological innovation and commercial application value.

[0185] Figure 2 System architecture and data flow diagram

[0186] Illustration:

[0187] The system showcases a four-layer architecture, illustrating the entire data flow from acquisition to application.

[0188] Data acquisition layer: Multiple data inputs ensure the comprehensiveness and authority of the content.

[0189] Data processing layer: Core data structuring and processing, forming the foundation of system knowledge.

[0190] Model building layer: The core of AI algorithms, enabling intelligent authentication capabilities.

[0191] Authentication Application Layer: The final service output to the user

[0192] Figure 3 Technical Performance Comparison Radar Chart

[0193] Figure 4 Knowledge Graph Construction Flowchart

[0194] Process description:

[0195] Raw data input: multi-source data including occupational skill standards, textbooks, and exam questions.

[0196] Skill dictionary generation: Automatically generating structured skill dictionaries based on large language models

[0197] Authentication dictionary generation: Generates standardized authentication dictionaries using a template engine.

[0198] Relationship Building: Establishing a link between skills and certifications

[0199] Graph Construction: Building a complete knowledge graph in a graph database

[0200] Quality verification: Verify the completeness and accuracy of the spectrum.

[0201] Application Deployment: Deploying the knowledge graph into the authentication system

[0202] Figure 5 Economic Benefit Comparison Analysis Chart

[0203] Advantages:

[0204] Cost reduction: 75% lower cost compared to traditional methods

[0205] Efficiency Improvement: Processing capacity increased by 4.5 times

[0206] Quality Assurance: Accuracy rate reaches over 96%.

[0207] Return on Investment: ROI cycle shortened to 12 months

[0208] Figure 6 Comparative Analysis Chart of Technological Breakthroughs

[0209] Technological Breakthrough Explanation:

[0210] 1. From singular to plural

[0211] Traditional: Only supports standardized multiple-choice questions; Innovative: Supports various question types such as practical simulations and case studies.

[0212] 2. From static to dynamic

[0213] Traditional: Long knowledge base update cycle; Innovative: Real-time synchronization with the latest industry standards.

[0214] 3. From Standards to Individuality

[0215] Traditional: Unified certification standards; Innovative: Personalized certification paths based on learners' abilities.

[0216] 4. From Artificial to Intelligent

[0217] Traditional: Relies on expert manual evaluation; Innovative: AI intelligent evaluation + manual review

[0218] 5. From Isolation to Ecology

[0219] Traditional: An independent certification system; Innovative: An ecosystem platform linked to employment and training.

[0220] Figure 7 Application effect verification diagram

[0221] Effect verification:

[0222] The average pass rate for various professional certifications increased by 12.5%.

[0223] User satisfaction rate reached over 90%.

[0224] Prove the effectiveness of the technical solution in practical applications.

[0225] Evidence related to the technical effects obtained by the embodiments of the present invention.

[0226] 1. System performance testing and verification

[0227] 1.1 Load Capacity Test

[0228] Test environment configuration: Server: Alibaba Cloud ECS c6.2xlarge (8 cores 32GB); Concurrency tool: JMeter 5.5; Test duration: 30 minutes of continuous stress test; Network environment: 100Mbps bandwidth.

[0229] Load test result data table:

[0230]

[0231] Figure 8 Performance trend chart:

[0232] Technical Performance Analysis: Under 100 concurrent users, the system maintains good performance with a response time of 680ms and a throughput of 220 QPS. The system bottleneck occurs above 200 concurrent users; it is recommended to keep the concurrent users below 150 in production environments. The error rate remains below 1% within the normal load range, indicating good system stability.

[0233] 1.2 System Performance Testing and Verification

[0234] Figure 9 Test data statistics:

[0235]

[0236] 2. Certification Quality Verification

[0237] 2.1 Certification Accuracy Test

[0238] Testing Method: 1000 real candidates were selected to participate in the test; the same batch of candidates participated in both traditional certification and certification using the invention system; the certification results were blind-reviewed and scored by 5 senior experts.

[0239] Detailed rating data table:

[0240] 2.2 Comparison of the effects of different occupational certifications

[0241] Figure 10 Verification of pass rates for multiple professional certifications:

[0242] Detailed data table:

[0243]

[0244] 3. Knowledge Graph Effectiveness Validation

[0245] 3.1 Knowledge Retrieval Efficiency Test

[0246] Search performance comparison data:

[0247]

[0248] 3.2 Knowledge Coverage Verification

[0249] Figure 11 , Figure 12 Analysis of the coverage of vocational skills knowledge points:

[0250] Coverage details:

[0251]

[0252] 4. Verification of economic benefits

[0253] 4.1 Cost-benefit analysis

[0254] Figure 13 Three-year cost comparison analysis:

[0255] 5. User satisfaction survey

[0256]

[0257] 6. System stability verification Figure 14

[0258]

[0259] 7. Verification of Technological Innovation

[0260] Verification of technological breakthroughs

[0261]

[0262] Through the above systematic testing, verification, and data collection, this invention demonstrates significant advantages in terms of technical performance, certification quality, economic benefits, and user satisfaction. Evidence of its main technical effectiveness includes:

[0263] Significant efficiency improvements: Question generation efficiency increased by over 98%, and authentication processing capacity increased by 350%.

[0264] Significant quality improvement: Authentication accuracy increased by 12.6%, and user satisfaction reached 4.53 points.

[0265] Significant economic benefits: Operating costs reduced by 58.3%, and the investment payback period shortened to 12 months.

[0266] Strong technological leadership: It is superior to traditional systems in all aspects of multi-dimensional technological comparison.

Claims

1. A vocational skills examination and certification method based on a large language model, characterized in that, Includes the following steps: Step 1: Generate a dictionary of occupational skill categories. The formatted dictionary template and the corresponding occupational skill category names are used as input to the large language model. The contextual example selection method is used to obtain the formatted textual description information of the occupational skill dictionary. The information is stored in the database table according to the attribute name, and a unique identifier ID is assigned to each record. Step 2: Generate a practical training exam certification dictionary, use the context example selection method to output exam certification description information, store it in a database table and introduce a unique identifier ID to realize the correspondence with the vocational skills dictionary; Step 3: Iterate through and generate dictionaries for all exam certifications under the professional skills category; Step 4: Recursively construct a knowledge graph of vocational skills, practical training, examination and certification based on the database examination and certification relationship table, and load the corresponding information from the vocational skills dictionary table on the nodes; Step 5: Repeat steps 1 to 4 to generate a knowledge graph of all professional skills for the corresponding examinations and certifications, and establish the corresponding relationships. Step 6: Create a test question generation task based on the system configuration parameters. The large language model service will store the generated test questions in a temporary library.

2. The vocational skills examination and certification method based on a large language model according to claim 1, characterized in that, The steps for generating the dictionary of occupational skill categories include: (1) Use the formatted dictionary template and the corresponding occupational skill category name as input to the large language model; (2) Using the contextual example selection method, multiple examples are provided to the large language model; (3) Obtain formatted text description information from the professional skills dictionary; (4) Store the corresponding description information in the database table according to the attribute name; (5) Add a unique identifier ID to prepare for subsequent exam certification association.

3. The vocational skills examination and certification method based on a large language model according to claim 1, characterized in that, When performing the test question generation task, the task name is used to define the type of test questions to be generated, the prompt word template is used to correspond to the task name, the occupation selection is used to specify the specific occupation category, the skill level selection is used to set the occupation level, the number of questions generated is used to determine the number of questions, the knowledge point selection is used to limit the scope of examination, the large model selection is used to specify the large language model to be called, and the knowledge base selection is used to bind the corresponding knowledge base. The steps for generating the practical training exam certification dictionary include: 1) Generate the exam certification dictionary according to the selection method shown in the example below, following the practical training exam certification dictionary template; 2) When storing exam certification dictionary information in a database table, introduce a unique identifier (ID); 3) Link exam certification with professional skills through a unique identifier ID.

4. The vocational skills examination and certification method based on a large language model according to claim 3, characterized in that, The selected knowledge points correspond to the specific knowledge or skill requirements in the exam syllabus.

5. The vocational skills examination and certification method based on a large language model according to claim 1, characterized in that, The knowledge graph is constructed recursively, reading data one by one from the examination and certification relationship table in the database. Nodes and edges are generated in a hierarchical manner of vocational skills—practical training examination—certification. Textual description information from the dictionary table is attached to the graph nodes to realize the hierarchical and visual representation of knowledge.

6. The vocational skills examination and certification method based on a large language model according to claim 5, characterized in that, The knowledge graph nodes and database tables establish a bidirectional index relationship through a unique identifier ID.

7. A vocational skills examination and certification system based on a large language model, implementing the vocational skills examination and certification method based on any one of claims 1-6, characterized in that, The vocational skills examination and certification system based on a large language model includes: The data collection module is used to collect relevant teaching materials, documents, question banks, and implementation procedures and requirements for vocational skills training and certification examinations, and to divide them into different training steps according to the knowledge system of vocational skills examination and certification. The preprocessing module is used to preprocess the above information and data based on large language model technology, obtain knowledge graphs corresponding to different exam training and certification materials, documents, question banks and implementation processes and requirements, and create corresponding exam certification tasks. The model building module is used to build an AI teaching assistant virtual training and examination certification model based on the knowledge system corresponding to the training, examination and certification steps, and to train and optimize it. The certification assessment module is used to obtain the training exam information for this training step. Through the AI ​​teaching assistant virtual training exam certification model, the training information is matched with the knowledge graph to determine whether the training information matches the knowledge information of this training exam certification step. If not, the AI ​​teaching assistant virtual certification model outputs the training tutoring information for this training certification step.

8. The system according to claim 7, characterized in that, The preprocessing module assigns a unique identifier (ID) to each record when storing examination and certification tasks, which is used to associate the record with the occupational skill category dictionary.

9. A computer-readable storage medium having a program stored thereon, the program, when executed by a processor, implementing the steps of the method according to any one of claims 1 to 6.

10. A vocational skills examination and certification device, characterized in that, The method includes a memory, a processor, and a program stored in the memory and executable on the processor, wherein the program, when executed, implements the steps of the method according to any one of claims 1 to 6.

Images