Interview question generation method and device based on large language model and storage medium

Abstract

The present application relates to the technical field of artificial intelligence assisted education, and more particularly to an interview question generation method and device based on a large language model and a storage medium.The method comprises the following steps: obtaining professional information and learning stage information of a target student, and constructing a student professional ability development graph; obtaining an intended employment direction selected by the target student, retrieving core ability requirements and constructing a post ability demand graph; generating an ability coverage correlation matrix; calculating the reachable mastery degree of each post knowledge point according to the ability coverage correlation matrix, and dividing the difficulty level of the knowledge point; forming a set of knowledge points to be examined; constructing a prompt and inputting a large language model to generate an interview simulation question with each post knowledge point in the set of knowledge points to be examined as the core, and performing learning stage adaptability verification.The present application can dynamically adjust the knowledge point coverage range and difficulty structure of the interview questions according to the actual situation of different students, and realizes individualized interview simulation training.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence-assisted education technology, and in particular to a method, apparatus and storage medium for generating interview questions based on a large language model. Background Technology

[0002] As competition for jobs among college graduates intensifies, more and more students hope to improve their interview skills and test-taking abilities through mock interviews before formally seeking employment. However, most existing mock interview systems are designed for job seekers with existing work experience, and their question generation is mainly based on matching job descriptions with resume information, neglecting the important characteristic of college students who are "still in the professional learning stage and have not yet acquired complete work experience."

[0003] For college students, their resumes often lack practical project experience and complete work history, making existing methods for generating interview questions based on matching job descriptions with resumes difficult to apply. Existing technologies mostly use a static job-skill matching approach when generating interview questions, failing to fully consider the dynamic relationship between students' learning stages and their professional training programs. This results in generated questions that exceed students' knowledge scope and are disconnected from their actual learning content. Furthermore, a two-way mapping relationship between the professional curriculum system and job competency requirements has not been established, making it difficult to achieve precise interview question generation for college students. Summary of the Invention

[0004] The purpose of this invention is to provide a method, apparatus and storage medium for generating interview questions based on a large language model, in order to solve the problem that the existing technology does not have a two-way mapping relationship between professional curriculum system and job competency requirements, resulting in inaccurate generation of interview questions for college students.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A method for generating interview questions based on a large language model, comprising:

[0007] Obtain the target students' major information and learning stage information, construct the students' professional ability development map based on the prerequisite dependencies between courses in the professional training program, and assign mastery weights to each node based on the grades of the courses already completed.

[0008] Obtain the target students' intended employment directions, retrieve the core competency requirements, and construct a job competency requirement map;

[0009] Semantically map students' professional competence development map with job competence requirement map to generate a competence coverage association matrix;

[0010] The attainability of knowledge points for each position is calculated based on the ability coverage correlation matrix, and the difficulty level of knowledge points is divided according to the attainability.

[0011] Obtain the proportion of questions at each difficulty level set by the target students, and extract job-related knowledge points from each knowledge point set according to the differentiated extraction strategy to form a set of knowledge points to be assessed.

[0012] The system constructs prompts based on the knowledge points for each position in the set of knowledge points to be assessed, inputs them into a large language model to generate mock interview questions, and then performs a learning phase adaptability check on the generated mock interview questions.

[0013] Furthermore, based on courses and their prerequisite dependencies, a directed acyclic graph structure is constructed for the development of students' professional competence. In the development graph, each node represents a professional course, and the directed edges between nodes point from prerequisite courses to subsequent courses. For course nodes that have been completed, their mastery weight is calculated based on their grades. For course nodes that are currently being studied, their mastery weight is temporarily set to a preset experience weight. For course nodes that have not yet been studied, their mastery weight is recorded as a null value.

[0014] Furthermore, based on the target students' intended employment direction, the core competency requirements of real positions corresponding to the intended employment direction are retrieved from a pre-set industry job knowledge base. Based on the core competency requirements, a job competency requirement graph is constructed. The job competency requirement graph is a weighted directed graph structure, where each node represents a knowledge point or competency item required for a position, and the directed edges between nodes represent the logical relationships between knowledge points. The association strength weight of each directed edge is determined based on the co-occurrence frequency of the two knowledge points connected by the directed edge in the real job description text of the target position. Job requirement weights are assigned according to the importance of knowledge points in the job competency model. The logical relationships include inclusion relationships, progressive relationships, and parallel relationships.

[0015] Furthermore, for the association strength weight, the actual job description text corresponding to the intended employment direction is selected, and the number of times each knowledge point pair appears together in the same text is counted. The number of times they appear together is normalized to the range of 0 to 1, which is used as the association strength weight of the association edge between the knowledge point pair.

[0016] For each knowledge point node, the job demand weight is determined as follows: perform word frequency statistics on the descriptive text corresponding to the intended employment direction, count the frequency of each knowledge point name in the entire text, and normalize the frequency to the range of 0 to 1 as the job demand weight of that knowledge point.

[0017] Furthermore, for each job knowledge point node in the job competency requirement graph, all course knowledge nodes and their subordinate sub-knowledge nodes in the student professional competency development graph are traversed to determine the semantic similarity between them. If the semantic similarity between a course knowledge node and a job knowledge point node exceeds the preset first similarity threshold, the course knowledge node is marked as the associated course node of the job knowledge point node, and its semantic similarity value is recorded as the mapping confidence. By traversing all node pairs, a mapping table from job knowledge points to associated course nodes is established.

[0018] After completing all mappings, a competency coverage association matrix is ​​generated by combining the mastery weights of each course knowledge node. Each job knowledge node in the behavioral job competency requirement map of the competency coverage association matrix is ​​listed as a course knowledge node in the student professional competency development map. The matrix element values ​​are calculated as follows: if a mapping relationship exists, the element value is the product of the mapping confidence and the mastery weight; if no mapping relationship exists, the element value is empty.

[0019] Furthermore, the semantic similarity is obtained by comprehensively matching three dimensions: name matching degree, description text matching degree, and domain affiliation matching degree, and then weighted and summed according to a preset matching weight ratio:

[0020] For name matching, after segmenting the names of the two nodes, the proportion of the number of shared words to the union of the total number of words in the names of the two nodes is calculated.

[0021] For the description text matching degree, the cosine similarity is calculated after vectorizing the attribute description texts of the two nodes in their respective graphs as the description text matching degree.

[0022] For domain affiliation matching, it determines whether two nodes belong to the same parent domain category. If they belong to the same domain, the value is 1; otherwise, the value is 0.

[0023] Furthermore, a row-by-row analysis of the competency coverage correlation matrix is ​​performed. For each job knowledge point node in the job competency requirement map, all course knowledge nodes with mapping relationships and their corresponding element values ​​are extracted from the row corresponding to that job knowledge point node. The sum of the element values ​​in that row is divided by the sum of the confidence scores of each mapping to obtain a weighted average value, which is used as the attainability and mastery of that job knowledge point under the current student's knowledge structure. The attainability and mastery of each job knowledge point is compared with the mastery threshold to classify the difficulty level of the job knowledge points.

[0024] Furthermore, each job-related knowledge point in the set of knowledge points to be assessed is used as the core examination point of the question. Prompts for question generation are constructed sequentially and input into a pre-defined large language model to generate mock interview questions. After the mock interview questions are generated, a learning stage adaptability check is performed. All job-related knowledge points involved in the mock interview questions are analyzed, and each job-related knowledge point involved in each mock interview question is matched with course knowledge nodes in the student's professional ability development map to determine whether the mock interview question passes the adaptability check. If there are course knowledge nodes in the student's professional ability development map that match the job-related knowledge points involved in the mock interview questions, and the proportion of nodes exceeding the preset tolerance threshold, the current mock interview question is considered to have passed the adaptability check. Mock interview questions that fail the adaptability check are regenerated until they pass the adaptability check, and all mock interview questions that pass the adaptability check are output.

[0025] On the other hand, the present invention also provides an interview question generation device based on a large language model, comprising:

[0026] The professional analysis module is used to obtain the target students' professional information and learning stage information, construct the students' professional ability development map based on the prerequisite dependencies between courses in the professional training program, and assign mastery weights to each node based on the grades of the courses already completed.

[0027] The intention analysis module is used to obtain the intended employment directions selected by target students, retrieve core competency requirements, and construct a job competency requirement map.

[0028] The association mapping module is used to semantically map students' professional ability development map with job ability requirement map to generate an ability coverage association matrix;

[0029] The difficulty classification module is used to calculate the attainability of knowledge points for each position based on the ability coverage correlation matrix, and classify the difficulty level of knowledge points according to the attainability.

[0030] The assessment analysis module is used to obtain the proportion of questions at each difficulty level set by the target students, and to extract job knowledge points from each knowledge point set according to a differentiated extraction strategy to form a set of knowledge points to be assessed.

[0031] The question generation module is used to construct prompts based on the knowledge points of each position in the set of knowledge points to be assessed, input them into a large language model to generate mock interview questions, and perform learning stage adaptability verification on the generated mock interview questions.

[0032] On the other hand, the present invention also provides a storage medium characterized in that it stores instructions that, when run on a computer, cause the computer to execute the above-described method for generating interview questions based on a large language model.

[0033] The beneficial effects of this invention are as follows: By constructing a student professional ability development map and a job competency requirement map, and establishing a semantic mapping between the two, the problem of existing technologies failing to dynamically generate interview questions based on the actual learning stage of students is solved. The difficulty level of knowledge points is divided based on the degree of mastery, and differentiated extraction is performed in conjunction with the proportion set by the students, ensuring that the scope of questions is closely related to the students' completed and ongoing course system. Simultaneously, after generating questions using a large language model, the learning stage adaptability is checked, filtering out questions that exceed the students' knowledge range, avoiding a disconnect between questions and learning content, thus realizing personalized interview simulation training and improving the accuracy and practicality of question generation. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is a flowchart of the interview question generation method based on a large language model in this embodiment.

[0036] Figure 2 This is a schematic diagram of the interview question generation device based on a large language model in this embodiment. Detailed Implementation

[0037] The following detailed description, in conjunction with the accompanying drawings and specific embodiments, provides a more comprehensive overview of the interview question generation method, apparatus, and storage medium based on a large language model disclosed in this invention. It should be noted that the technical features or combinations of technical features described in the following embodiments should not be considered isolated; they can be combined to achieve better technical results. In the accompanying drawings of the following embodiments, the same reference numerals in each drawing represent the same features or components, which can be applied to different embodiments. Therefore, once an item is defined in one drawing, it does not need to be further discussed in subsequent drawings.

[0038] It should be noted that the structures, proportions, sizes, etc., illustrated in the accompanying drawings are merely for illustrative purposes and to aid those skilled in the art in understanding and reading the invention. They are not intended to limit the conditions under which the invention can be implemented. Any modifications to the structure, changes in proportions, or adjustments to size, provided they do not affect the effectiveness or purpose of the invention, should fall within the scope of the technical content disclosed in the invention. The scope of the preferred embodiments of the present invention includes other implementations, wherein functions may be performed not in the order stated or discussed, including substantially simultaneously or in reverse order, depending on the functions involved. This should be understood by those skilled in the art to which the embodiments of the present invention pertain.

[0039] Techniques, methods, and apparatus known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and apparatus should be considered part of the specification. In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0040] In the description of the embodiments of this application, " / " means "or", and "and / or" is used to describe the relationship between related objects, indicating that there can be three relationships. For example, "A and / or B" means: A and B exist alone, B exists alone, and A and B exist simultaneously. In the description of the embodiments of this application, "multiple" refers to two or more embodiments.

[0041] Example 1

[0042] This embodiment uses a third-year student majoring in computer science and technology at a university as an example to illustrate the complete implementation process of the method of the present invention.

[0043] Please see Figure 1 As shown, it is a flowchart of the interview question generation method based on a large language model in this embodiment, including:

[0044] Step S1: Obtain the target student's major information and learning stage information, construct the student's professional ability development map based on the prerequisite dependencies between courses in the professional training program, and assign mastery weights to each node based on the grades of the courses already completed.

[0045] The basic information of the target student is as follows: major name is "Computer Science and Technology", enrollment year is September 2023, current year is third year of university, current semester is sixth semester. The student's academic record is retrieved from the school's academic affairs management system to obtain all courses that have been completed and their grades.

[0046] In this embodiment, the courses that the current target student has completed include: Advanced Mathematics (86 points), Linear Algebra (79 points), Probability Theory and Mathematical Statistics (82 points), Fundamentals of Programming (91 points), Data Structures and Algorithms (88 points), Computer Organization (76 points), Operating Systems (81 points), Database System Principles (84 points), and Computer Networks (79 points). At the same time, the list of courses that the current target student is currently taking this semester is obtained, including: Software Engineering and Compiler Principles.

[0047] The student professional competence development map was determined based on the undergraduate curriculum of the computer science and technology major at the university.

[0048] Specifically, the curriculum clearly stipulates the prerequisite courses for each professional course. For example, the prerequisite courses for: Fundamentals of Programming, Fundamentals of Programming, Computer Organization, Computer Organization, Database Systems, Software Engineering, and Compiler Principles are all prerequisite courses for both Data Structures and Algorithms and Computer Organization.

[0049] Based on the above courses and their prerequisite dependencies, a directed acyclic graph structure is constructed for the development of students' professional abilities. In the development graph, each node represents a professional course, and the directed edges between nodes point from the prerequisite courses to the subsequent courses. For example, there exists a directed edge from the "Fundamentals of Programming" node to the "Data Structures and Algorithms" node, indicating that "Fundamentals of Programming" is a prerequisite for "Data Structures and Algorithms".

[0050] For course nodes that have been completed, the mastery weight is calculated based on the grades. The analysis method for the mastery weight is to divide the percentage grade by 100 to obtain the mastery weight value in the range of 0 to 1.

[0051] Specifically, the mastery weights are as follows: Advanced Mathematics 0.86, Linear Algebra 0.79, Probability Theory and Mathematical Statistics 0.82, Fundamentals of Programming 0.91, Data Structures and Algorithms 0.88, Computer Organization Principles 0.76, Operating Systems 0.81, Database System Principles 0.84, and Computer Networks 0.79.

[0052] For course nodes that are currently being studied, since students have not yet completed the course and have not obtained the final grade, their mastery weight is temporarily set to a preset experience weight. In this embodiment, the preset experience weight is set to 0.4. The basis for this value is that students usually take the mid-term exam when they have completed about 40% of the course. The average mastery level at the mid-term stage is about 40%. This is a common experience value in this field. In other embodiments, the preset experience weight can also be dynamically adjusted according to the proportion of the course time that has been taught to the total class time. For example, if the proportion of the course time that has been taught to the total class time in the current semester is 50%, then the mastery weight of the course being studied can be taken as 0.50. For course nodes that have not yet been studied, their mastery weight is recorded as a null value, indicating that the student has not yet been exposed to this part of the knowledge.

[0053] For each completed course, the course outline is also extracted from the curriculum plan, recording the main knowledge units covered by the course. For example, the outline of the Data Structures and Algorithms course includes: linear structures, tree structures, graph structures, sorting algorithms, and search algorithms. These knowledge units are sub-knowledge nodes attached to the course nodes.

[0054] This step involves acquiring information about students' majors and learning stages, constructing a directed acyclic graph based on course prerequisites, and combining previously completed grades with the pre-set weights of courses in progress to accurately quantify students' current knowledge mastery. This provides a structured data foundation for subsequent ability mapping and ensures that the generated questions closely align with students' actual learning progress.

[0055] Step S2: Obtain the intended employment direction selected by the target students, retrieve the core competency requirements, and construct a job competency requirement graph. In the job competency requirement graph, nodes represent job knowledge points, and edges represent the logical relationships between knowledge points. Each node and edge is assigned a job requirement weight and a relationship strength weight, respectively.

[0056] In this embodiment, the target student's intended employment direction is: industry category "Internet / IT" and job category "backend development engineer".

[0057] Based on the target students' intended employment direction, the core competency requirements for backend development engineer positions are retrieved from a pre-designed industry job knowledge base. The industry job knowledge base is constructed by collecting and analyzing a large number of real job description texts published by companies, extracting the common knowledge points, tools, and technology stacks required for various positions, and having them reviewed and confirmed by domain experts.

[0058] In this embodiment, the core competency requirements for the backend development engineer position obtained through retrieval include:

[0059] Essential professional knowledge includes: programming languages ​​such as Java, Python, and C++; data structures and algorithms; database principles and SQL; operating system principles; computer networks; software engineering methods; and design patterns.

[0060] Commonly used tools include: basic Linux operating system operations, Git version control tool, MySQL database management system, Redis caching system, and Docker containerization tool.

[0061] Based on core competency requirements, a job competency requirement graph is constructed. The job competency requirement graph is a weighted directed graph structure, where each node represents a knowledge point or competency item required for a job. The directed edges between nodes represent the logical relationships between knowledge points. The association strength weight of each directed edge is determined based on the co-occurrence frequency of the two knowledge points connected by the directed edge in the actual job description text of the target job. The job requirement weight is assigned according to the importance of the knowledge points in the job competency model.

[0062] Specifically, the "Data Structures and Algorithms" node includes several sub-knowledge point nodes such as "Arrays", "Linked Lists", "Stacks and Queues", "Trees and Binary Trees", "Sorting Algorithms", "Search Algorithms", and "Graph Algorithms"; the "Database Principles and SQL" node includes sub-knowledge point nodes such as "Relational Models", "SQL Query Language", "Index Mechanisms", "Transactions and Concurrency Control", and "Database Design Paradigms"; and the "Operating System Principles" node includes sub-knowledge point nodes such as "Processes and Threads", "Scheduling Algorithms", "Memory Management", and "File Systems".

[0063] Logical relationships include:

[0064] Containment relationships, for example, if the node "Data Structures and Algorithms" has a directed edge pointing to the child node "Sorting Algorithms", it means that "Sorting Algorithms" is a component of "Data Structures and Algorithms".

[0065] A progressive relationship, for example, the "Database Principles" node has a directed edge pointing to the "Database Optimization" node, which means that mastering basic database knowledge is a prerequisite for mastering database optimization techniques;

[0066] Parallel relationships, such as a bidirectional edge between a "quicksort" node and a "mergesort" node, represent that they belong to different branches of the same level.

[0067] Specifically, for the association strength weight, 100 real job description texts for backend development engineers are selected, and the number of times each job knowledge point pair appears in the same text is counted. The number of times they appear in the same text is divided by 100 and normalized to the range of 0 to 1. This number is used as the association strength weight of the association edge between the knowledge point pair. For example, "data structure and algorithm" and "sorting algorithm" appear in 95 descriptions, and their association strength weight is 0.95.

[0068] For each knowledge point node, the job requirement weight is determined as follows: perform word frequency statistics on 100 backend development engineer job description texts, count the frequency of each knowledge point name in all texts, and normalize the frequency to the range of 0 to 1 as the job requirement weight of that knowledge point.

[0069] This step involves retrieving the core competency requirements of desired positions, constructing a weighted directed graph of job competency requirements, and statistically analyzing the correlation strength and requirement weights based on real job texts. This ensures that the organization of job knowledge points closely resembles actual work scenarios, providing a reliable job-side reference for subsequent accurate mapping and question generation.

[0070] Step S3 involves semantically mapping the student's professional competence development map with the job competence requirement map to generate a competence coverage association matrix.

[0071] Specifically, for each job knowledge point node in the job competency requirement graph, all course knowledge nodes and their subordinate sub-knowledge nodes in the student professional competency development graph are traversed to determine the semantic similarity between them. If the semantic similarity between a course knowledge node and a job knowledge point node exceeds a preset first similarity threshold, the course knowledge node is marked as an associated course node of the job knowledge point node, and its semantic similarity value is recorded as the mapping confidence. By traversing all node pairs, a mapping table from job knowledge points to associated course nodes is established. In this embodiment, the preset first similarity threshold is set to 0.7.

[0072] The determination of the first similarity threshold is based on the following: 200 pairs of different course-job knowledge point combinations are selected, and a review panel composed of 5 university professional teachers with more than 10 years of teaching experience and 5 recruitment experts with more than 10 years of industry experience independently judges whether a mapping relationship should be established for each combination. Based on the consensus of at least 6 people, the mapping accuracy rate under different similarity thresholds is calculated, and the similarity threshold corresponding to the highest mapping accuracy rate is taken as the first similarity threshold. In other embodiments, this threshold can be selected between 0.65 and 0.75.

[0073] The semantic similarity is obtained by comprehensively matching three dimensions: name matching degree, description text matching degree, and domain affiliation matching degree, and then weighted and summed according to a preset matching weight ratio.

[0074] For name matching, after segmenting the names of two nodes, the proportion of the number of shared words to the union of the total words of the two node names is calculated. For example, "sorting algorithm" is segmented into "sorting / algorithm", and "data structure and algorithm" is segmented into "data / structure / and / algorithm". The intersection is "algorithm", which is 1 word. The union is "sorting / algorithm / data / structure", which is 4 words. The name matching degree is 1 / 4 = 0.25.

[0075] For the description text matching degree, the cosine similarity is calculated after vectorizing the attribute description text of the two nodes in their respective graphs. The attribute description text comes from the knowledge unit description in the course syllabus and the knowledge point definition description in the job knowledge base.

[0076] For domain affiliation matching degree, it is determined whether two nodes belong to the same higher-level domain category, such as whether they belong to the same algorithm category or the same database category. If they belong to the same domain, the value is 1, otherwise the value is 0.

[0077] In this embodiment, the preset matching weight ratio is set to 0.5:0.3:0.2. The name is the most direct indicator for judging the correspondence of knowledge points, so it is given the highest weight of 0.5. The description text can reflect the deeper meaning of the knowledge points, so it is given a weight of 0.3. Domain affiliation is used as an auxiliary judgment dimension, so it is given a weight of 0.2.

[0078] For example, taking the job knowledge point "sorting algorithm" as an example, traversing all course knowledge nodes: the name matching degree with the "sorting algorithm" sub-knowledge node under the "Data Structures and Algorithms" course is calculated to be 0.95, the description text matching degree is 0.88, the domain affiliation matching degree is 1, and the weighted sum is 0.95×0.5+0.88×0.3+1×0.2=0.939, which exceeds 0.7, so the mapping confidence is 0.939; the name matching degree with the node of the "Fundamentals of Programming" course is 0.2, the description text matching degree is 0.35, the domain affiliation matching degree is 0.5, and the weighted sum is 0.2×0.5+0.35×0.3+0.5×0.2=0.305, which does not exceed 0.7, so no mapping relationship is established.

[0079] After completing all mappings, a competency coverage association matrix is ​​generated by combining the mastery weights of each course knowledge node. Each job knowledge node in the behavioral job competency requirement map of the competency coverage association matrix is ​​listed as a course knowledge node in the student professional competency development map. The matrix element values ​​are calculated as follows: if a mapping relationship exists, the element value is the product of the mapping confidence and the mastery weight; if no mapping relationship exists, the element value is empty.

[0080] This step uses multi-dimensional semantic similarity calculation to finely map course knowledge nodes with job knowledge points, generating a capability coverage association matrix. This establishes a quantitative bridge between what students learn and job requirements, solving the problem of the disconnect between the curriculum and job requirements, and providing a basis for assessing the mastery of knowledge points.

[0081] Step S4: Calculate the attainability of knowledge points for each position based on the ability coverage correlation matrix, and classify the difficulty level of knowledge points according to the attainability. The difficulty level includes four levels: basic, intermediate, advanced, and extended.

[0082] Specifically, the ability coverage correlation matrix is ​​analyzed row by row. For each job knowledge point node in the job ability requirement map, all course knowledge nodes with mapping relationships and their corresponding element values ​​are extracted from the row corresponding to the job knowledge point node. The sum of the element values ​​in the row is divided by the sum of the confidence scores of each mapping to obtain a weighted average value, which is used as the attainable mastery degree of the job knowledge point under the current student knowledge structure.

[0083] For example, for the knowledge point of sorting algorithm job: there is only one related course, Data Structures and Algorithms, with an element value of 0.826, a mapping confidence of 0.939, and a mastery level of 0.826 / 0.939=0.88;

[0084] For SQL query language job knowledge points: there is only one related course, Database System Principles, with an element value of 0.777, a mapping confidence level of 0.925, and a mastery level of 0.777 / 0.925=0.84;

[0085] For the knowledge point related to transaction and concurrency control: there is only one related course, Database System Principles, with an element value of 0.655, a mapping confidence level of 0.780, and an attainability level of 0.655 / 0.78=0.84. However, since this part of the content belongs to a relatively difficult chapter in the "Database System Principles" course, students have a low accuracy rate in the relevant quizzes for this chapter. Therefore, the attainability level of this knowledge point is independently adjusted based on the chapter quiz scores. Specifically, the accuracy rate of the chapter quiz is used as the direct attainment level of the knowledge point. In this example, the accuracy rate of student 00 in this chapter is 0.62, so the attainability level of this knowledge point is adjusted to 0.62.

[0086] The attainable mastery level of each job knowledge point is compared with the mastery level threshold. Job knowledge points whose attainable mastery level exceeds the first mastery level threshold are classified into the first category of knowledge points, and job knowledge points whose attainable mastery level is between the second mastery level threshold and the first mastery level threshold are classified into the second category of knowledge points. The first category of knowledge points represents the basic difficulty level, and the second category of knowledge points represents the medium difficulty level. In this embodiment, the first mastery level threshold is set to 0.75 and the second mastery level threshold is set to 0.5.

[0087] It is worth noting that for job knowledge points that do not have any related course nodes in the student's professional competence development map with a mastery level exceeding the second mastery threshold, but are partially covered in the current courses, such as design patterns having two class hours of design pattern overview content in the syllabus of the software engineering course, and meeting the above conditions, the job knowledge point is classified into the third category of knowledge points.

[0088] For job knowledge points that exist in the job competency requirement map but have no corresponding courses in the students' professional training programs, such as computer science majors that arrange to teach distributed systems at the master's level, the job knowledge points should be classified into the fourth category of knowledge points.

[0089] The third set of knowledge points represents the advanced difficulty level, and the fourth set of knowledge points represents the extended difficulty level.

[0090] The determination of the first and second mastery thresholds is based on the following: statistical analysis of the course grade distribution of students in each major of the school over the past three years, calculation of the first full sample average, and the corresponding mastery weight is the first full sample average / 100, which is used as the dividing threshold between "fully mastered" and "partially mastered". At the same time, the distribution of failing grades of students is statistically analyzed, and the second full sample average is calculated. The corresponding mastery weight is the second full sample average / 100, which is used as the dividing threshold between "partially mastered" and "basically not mastered". In other embodiments, the first mastery threshold can be adjusted between 0.7 and 0.8 according to the overall teaching level of the school, and the second mastery threshold can be adjusted between 0.45 and 0.55.

[0091] This step calculates the attainability of job-related knowledge points and adjusts them by combining fine-grained information such as chapter quizzes. It scientifically divides the knowledge points into four difficulty levels: basic, intermediate, advanced, and extended. This difficulty division reflects students' true mastery level while also taking into account the inherent hierarchical structure of job-related knowledge.

[0092] Step S5: Obtain the proportion of questions at each difficulty level set by the target students, and extract job knowledge points from each knowledge point set according to the differentiated extraction strategy to form a set of knowledge points to be assessed.

[0093] Specifically, the system obtains the proportion of questions at each difficulty level set by the target student through the interactive interface, and determines the number of knowledge points to be extracted at each difficulty level based on the proportion of questions at each difficulty level. In this embodiment, the parameters set by the target student are: a total of 10 simulated interview questions, and the preset proportions of questions at each difficulty level are: 40% for basic difficulty, 30% for intermediate difficulty, 20% for advanced difficulty, and 10% for extended difficulty, corresponding to 4 questions at basic difficulty, 3 questions at intermediate difficulty, 2 questions at advanced difficulty, and 1 question at extended difficulty. The proportion of questions at each difficulty level can also be automatically recommended by the system based on the student's grade level. For example, for a third-year student who has taken courses covering most of the core professional courses, the system will automatically recommend the above proportions.

[0094] Specifically, four job-related knowledge points are extracted from the first set of knowledge points, and sorted from high to low according to the job requirement weight of each knowledge point. The first four job-related knowledge points are then extracted in sequence.

[0095] Three job-related knowledge points are extracted from the second set of knowledge points. The knowledge points are sorted from low to high according to their mastery level, and the top three job-related knowledge points are extracted in turn.

[0096] Two job-related knowledge points are extracted from the third set of knowledge points. The knowledge points are sorted from high to low according to their job requirement weights, and the first two job-related knowledge points are extracted in turn.

[0097] Extract one job knowledge point from the fourth category of knowledge points set. In the job competency requirement graph, find job knowledge points with extended difficulty levels that have directed edge association with the extracted third category of knowledge points. Select the job knowledge point with the highest edge association strength weight.

[0098] The above four types of extraction results are combined to obtain a set of 10 knowledge points to be assessed.

[0099] It is understandable that the extraction of the first and third sets of knowledge points is to select job knowledge points with higher job requirement weights, the extraction of the second set of knowledge points is to select job knowledge points with lower attainability and mastery, and the extraction of the fourth set of knowledge points is to select job knowledge points with higher relevance strength weights.

[0100] This step involves obtaining the proportion of questions at each difficulty level customized by the students and adopting a differentiated extraction strategy for different categories of knowledge points. This ensures that the extracted knowledge points meet the students' individual needs in terms of difficulty distribution, making the final assessment set both targeted and challenging.

[0101] Step S6: Construct prompts based on the knowledge points of each position in the set of knowledge points to be assessed, input them into a large language model to generate mock interview questions, and perform learning stage adaptability verification on the generated mock interview questions.

[0102] Specifically, each job-related knowledge point in the set of knowledge points to be assessed is used as the core examination point of the question. Question generation prompts are constructed sequentially and input into a preset large language model to generate mock interview questions. After the mock interview questions are generated, a learning stage adaptability check is performed. All job-related knowledge points involved in the mock interview questions are analyzed, and each job-related knowledge point involved in the mock interview questions is matched with course knowledge nodes in the student's professional ability development map to determine whether the mock interview questions pass the adaptability check. If there are course knowledge nodes in the student's professional ability development map that match the job-related knowledge points involved in the mock interview questions, and the percentage of nodes exceeding the preset tolerance threshold, the current mock interview question is considered to have passed the adaptability check. The percentage exceeding the threshold is the ratio of the number of job-related knowledge points involved in the mock interview questions that failed to match to the total number of job-related knowledge points involved in the mock interview questions. In this embodiment, the preset tolerance threshold is 30%.

[0103] Taking "sorting algorithm" as an example, the prompt content is: "The core examination point is sorting algorithm. The knowledge point of this position belongs to the basic difficulty level. The context of this knowledge point in the job competency requirement map is: it belongs to the core sub-knowledge point under the data structure and algorithm module. Common examination dimensions include the basic principles of various sorting algorithms, time complexity analysis, stability judgment and applicable scenario selection. Please generate an interview simulation question. The question should include a specific scenario description to examine the candidate's understanding and application ability of sorting algorithms." After being generated by the big language model, the following question is obtained: "Suppose you need to sort a list containing 100,000 random integers. There are two schemes available: quick sort and merge sort. Please answer: (1) What is the basic principle of quick sort? (2) In the worst case, what is the time complexity of quick sort? Under what circumstances will the worst case occur? (3) If the range of integers in the list is known to be 1 to 1000 Given integers between 0 and 1, is there a more efficient sorting method than quicksort and mergesort? Please explain your reasoning. Semantic analysis reveals that this question involves the following knowledge points: "Basic Principles of Quicksort," "Time Complexity Analysis," "Worst-Case Analysis," and "Counting Sort." "Basic Principles of Quicksort," "Time Complexity Analysis," and "Worst-Case Analysis" belong to the "Sorting Algorithms" chapter of the "Data Structures and Algorithms" course; "Counting Sort" belongs to the "Linear Time Sort" chapter of the same course. While this chapter is an elective in the school's curriculum, it is covered in the course syllabus. Matching the aforementioned job-related knowledge points with the course knowledge nodes in the student's professional competency development map, "Data Structures and Algorithms" is a course the student has already taken, with a mastery weight of 0.88. Therefore, all knowledge points involved in this mock interview question fall within the scope of the student's already taken courses, and the mock interview question passes the suitability check.

[0104] The large language model used in this embodiment is a language generation model that has been pre-trained in a general domain. Its input is a text-formatted prompt, and its output is the question text corresponding to the prompt.

[0105] For example, taking the knowledge point "microservice architecture" to be assessed as an example, after constructing the prompt, the large language model generates the question: "Please briefly describe the core idea of ​​microservice architecture and its main advantages and disadvantages compared with monolithic architecture."

[0106] The suitability of this question was verified: Analysis of the job knowledge points involved in the question included "microservice architecture," "monolithic architecture," and "architectural evolution." Matching these job knowledge points with the student's professional competency development map revealed that "microservice architecture" was not included in any completed or currently enrolled courses. This indicates that the university's undergraduate computer science curriculum does not include courses related to distributed systems and microservice architecture. This content is outside the scope of undergraduate teaching. The proportion of uncompleted course knowledge in the question's total knowledge exceeds the preset tolerance threshold. Therefore, this mock interview question failed the suitability verification. The tolerance threshold allows questions to include a small amount of extended knowledge that students have not yet learned as challenging content, but it should not exceed one-third of the total knowledge; otherwise, students will be unable to answer, rendering the mock interview unproductive.

[0107] Regenerate the question and add the following constraint to the large language model: "Please generate a mock interview question focusing on 'design patterns,' at an advanced difficulty level. The knowledge covered in the question is limited to the overview and related content of design patterns already taught in software engineering courses." This constraint is added as an additional instruction in the input prompt.

[0108] The regenerated question is: "Please list three design patterns you are familiar with, explain the core idea of ​​each design pattern, and analyze how to choose the appropriate design pattern in a specific software development scenario." The regenerated question undergoes an adaptability check: Analysis of the knowledge points involved includes "design pattern concepts," "design pattern classifications," and "design pattern application scenarios." Matching these job-related knowledge points with the student's professional competency development map reveals that "design pattern concepts" and "design pattern classifications" are covered in the "Overview of Design Patterns" chapter of the "Software Engineering" syllabus. Since "Software Engineering" is a currently enrolled course, its mastery weight is 0.4, indicating that the student is currently encountering this knowledge but has not yet fully mastered it. The job-related knowledge points involved in the question do not exceed the student's current learning scope, therefore the regenerated question passes the adaptability check.

[0109] The above-mentioned adaptability verification process is executed sequentially for generating interview simulation questions for all 10 job knowledge points. For interview simulation questions that fail the adaptability verification, they are regenerated until they pass the adaptability verification, and all interview simulation questions that pass the adaptability verification are output.

[0110] This step generates mock interview questions using a large language model and combines them with learning stage adaptability checks. Questions that are out of scope are regenerated under constraints until they pass the checks, ensuring that each question in the final output falls within the scope of the courses that students have taken or are taking, effectively guaranteeing the answerability and training value of the questions.

[0111] Example 2

[0112] This embodiment uses a second-year student majoring in finance at a university as an example to further illustrate the applicability of the present invention in different professional backgrounds.

[0113] The student's basic information is as follows: major is "Finance", current year is sophomore, courses taken include: Microeconomics (83 points), Macroeconomics (79 points), Fundamentals of Accounting (78 points), Introduction to Finance (72 points); currently taking courses include: Corporate Finance, Statistics, intended employment direction: industry category "Finance", job category "Investment Analyst";

[0114] Perform each step according to the same procedure as in Example 1:

[0115] The mastery weights for each course completed were calculated based on the student's grades: Microeconomics 0.83, Macroeconomics 0.79, Fundamentals of Accounting 0.78, Introduction to Finance 0.72, and the mastery weights for the currently enrolled courses "Corporate Finance" and "Statistics" were tentatively set at 0.4.

[0116] The core competency requirements for investment analysis positions were retrieved from an industry job knowledge base, including: financial statement analysis (interpretation of balance sheets, income statements, cash flow statements, and financial ratios), macroeconomic analysis, industry analysis methods, company valuation methods (price-to-earnings ratio method, discounted cash flow method), and portfolio theory. A corresponding competency requirement map was constructed, and the job requirement weights for each knowledge point were calculated.

[0117] The semantic similarity between the "Fundamentals of Accounting" course and the knowledge points for the "Financial Statement Analysis" job is 0.86, which is greater than 0.7, and the mapping confidence level is 0.86; the mapping confidence level between the "Introduction to Finance" course and "Macroeconomic Analysis" is 0.82; the mapping confidence level between the "Corporate Finance" course and "Corporate Valuation Methods" is 0.85; establish the above mapping relationships and generate a capability coverage association matrix;

[0118] For the job knowledge point "Financial Statement Analysis", the associated course is "Basic Accounting". The element value is 0.86×0.78=0.671, the mapping confidence level is 0.86, and the attainability is 0.671 / 0.86=0.78. Since the attainability is greater than the first attainability threshold, it is classified into the first type of knowledge point set.

[0119] For the job knowledge point "financial ratio analysis", the related course is "fundamentals of accounting", with a mastery level of 0.78, and it is also classified into the first category of knowledge points;

[0120] For the job knowledge point "Company Valuation Methods", the associated course is "Corporate Finance", with an element value of 0.85 × 0.4 = 0.34, a mapping confidence level of 0.85, and an attainability level of 0.340 / 0.85 = 0.4. This attainability level is less than the first attainability threshold and less than the second attainability threshold. Furthermore, "Corporate Finance" is a course currently being taken, so it is classified into the third category of knowledge points.

[0121] For the job knowledge point "derivative pricing", this job knowledge point exists in the job competency requirement map, but there is no related course knowledge node in the student professional competency development map. Since there is a related course node but the course is a course that has not been taken, it is classified into the third type of knowledge point set.

[0122] The student set the following target parameters for the mock interview: a total of 8 questions, with 50% basic difficulty, 25% medium difficulty, 25% advanced difficulty, and 0% extended difficulty. For each difficulty level, the following questions were selected: 4 basic questions, 2 medium difficulty questions, 2 advanced difficulty questions, and 0 extended difficulty questions.

[0123] Four knowledge points are extracted from the first set of knowledge points according to the job requirement weight from high to low. Two knowledge points are extracted from the second set of knowledge points according to the attainability from low to high. Two knowledge points are extracted from the third set of knowledge points according to the job requirement weight from high to low. These are combined to obtain eight knowledge points to be assessed. The subsequent question generation and adaptability verification process is the same as in Example 1, and will not be repeated here.

[0124] Please see Figure 2 As shown, this is the interview question generation device based on a large language model in this embodiment, including:

[0125] The professional analysis module is used to obtain the target students' professional information and learning stage information, construct the students' professional ability development map based on the prerequisite dependencies between courses in the professional training program, and assign mastery weights to each node based on the grades of the courses already completed.

[0126] The intention analysis module is used to obtain the intended employment directions selected by target students, retrieve core competency requirements, and construct a job competency requirement map.

[0127] The association mapping module is used to semantically map students' professional ability development map with job ability requirement map to generate an ability coverage association matrix;

[0128] The difficulty classification module is used to calculate the attainability of knowledge points for each position based on the ability coverage correlation matrix, and classify the difficulty level of knowledge points according to the attainability.

[0129] The assessment analysis module is used to obtain the proportion of questions at each difficulty level set by the target students, and to extract job knowledge points from each knowledge point set according to a differentiated extraction strategy to form a set of knowledge points to be assessed.

[0130] The question generation module is used to construct prompts based on the knowledge points of each position in the set of knowledge points to be assessed, input them into a large language model to generate mock interview questions, and perform learning stage adaptability verification on the generated mock interview questions.

[0131] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0132] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0133] In the above description, the disclosure of this invention is not intended to limit itself to these aspects. Rather, within the scope of the objectives of this disclosure, components can be selectively and operationally combined in any number. Furthermore, terms such as “comprising,” “encompassing,” and “having” should be interpreted by default as inclusive or open-ended, rather than exclusive or closed, unless explicitly defined as such. All technical, scientific, or other terms are to be understood by those skilled in the art, unless defined as such. Public terms found in dictionaries should not be interpreted in the context of the relevant technical documents in an overly idealistic or impractical manner, unless explicitly defined as such in this disclosure. Any modifications or alterations made by those skilled in the art based on the foregoing disclosure are within the scope of the claims.

Claims

1. A method for generating interview questions based on a large language model, characterized in that, include: Obtain the target students' major information and learning stage information, construct the students' professional ability development map based on the prerequisite dependencies between courses in the professional training program, and assign mastery weights to each node based on the grades of the courses already completed. Obtain the target students' intended employment directions, retrieve the core competency requirements, and construct a job competency requirement map; Semantically map students' professional competence development map with job competence requirement map to generate a competence coverage association matrix; The attainability of knowledge points for each position is calculated based on the ability coverage correlation matrix, and the difficulty level of knowledge points is divided according to the attainability. Obtain the proportion of questions at each difficulty level set by the target students, and extract job-related knowledge points from each knowledge point set according to a differentiated extraction strategy to form a set of knowledge points to be assessed. The system constructs prompts based on the knowledge points for each position in the set of knowledge points to be assessed, inputs them into a large language model to generate mock interview questions, and then performs a learning phase adaptability check on the generated mock interview questions.

2. The interview question generation method based on a large language model according to claim 1, characterized in that, Based on courses and their prerequisite dependencies, a directed acyclic graph structure is constructed for the development of students' professional competence. In the development graph, each node represents a professional course, and the directed edges between nodes point from prerequisite courses to subsequent courses. For course nodes that have been completed, their mastery weight is calculated based on their grades. For course nodes that are currently being studied, their mastery weight is temporarily set to a preset experience weight. For course nodes that have not yet been studied, their mastery weight is recorded as a null value.

3. The interview question generation method based on a large language model according to claim 1, characterized in that, Based on the target students' intended employment directions, the core competency requirements of real positions corresponding to those directions are retrieved from a pre-defined industry job knowledge base. Based on these core competency requirements, a job competency requirement graph is constructed. This graph is a weighted directed graph structure, where each node represents a knowledge point or competency required for a position. Directed edges between nodes represent logical relationships between knowledge points. The strength weight of each directed edge is determined based on the co-occurrence frequency of the two knowledge points connected by that edge in the actual job description text of the target position. Job requirement weights are assigned based on the importance of knowledge points in the job competency model. These logical relationships include inclusion, progression, and parallel relationships.

4. The interview question generation method based on a large language model according to claim 3, characterized in that, For the association strength weight, select the real job description text corresponding to the intended employment direction, count the number of times each knowledge point pair appears in the same text, normalize the number of times they appear to the interval between 0 and 1, and use it as the association strength weight of the association edge between the knowledge point pair. For each knowledge point node, the job demand weight is determined as follows: perform word frequency statistics on the descriptive text corresponding to the intended employment direction, count the frequency of each knowledge point name in the entire text, and normalize the frequency to the range of 0 to 1 as the job demand weight of that knowledge point.

5. The interview question generation method based on a large language model according to claim 1, characterized in that, For each job knowledge point node in the job competency requirement graph, traverse all course knowledge nodes and their subordinate sub-knowledge nodes in the student professional competency development graph to determine the semantic similarity between them. If the semantic similarity between a course knowledge node and a job knowledge point node exceeds the preset first similarity threshold, then mark the course knowledge node as the associated course node of the job knowledge point node and record its semantic similarity value as the mapping confidence. By traversing all node pairs, establish a mapping table from job knowledge points to associated course nodes. After completing all mappings, a competency coverage association matrix is ​​generated by combining the mastery weights of each course knowledge node. Each job knowledge node in the behavioral job competency requirement map of the competency coverage association matrix is ​​listed as a course knowledge node in the student professional competency development map. The matrix element values ​​are calculated as follows: if a mapping relationship exists, the element value is the product of the mapping confidence and the mastery weight; if no mapping relationship exists, the element value is empty.

6. The interview question generation method based on a large language model according to claim 5, characterized in that, The semantic similarity is obtained by comprehensively matching three dimensions: name matching degree, description text matching degree, and domain affiliation matching degree, and then weighted and summed according to a preset matching weight ratio. For name matching, after segmenting the names of the two nodes, the proportion of the number of shared words to the union of the total number of words in the names of the two nodes is calculated. For the description text matching degree, the cosine similarity is calculated after vectorizing the attribute description texts of the two nodes in their respective graphs as the description text matching degree. For domain affiliation matching, it determines whether two nodes belong to the same parent domain category. If they belong to the same domain, the value is 1; otherwise, the value is 0.

7. The interview question generation method based on a large language model according to claim 1, characterized in that, A row-by-row analysis of the competency coverage correlation matrix is ​​performed. For each job knowledge point node in the job competency requirement map, all course knowledge nodes with mapping relationships and their corresponding element values ​​are extracted from the row corresponding to that job knowledge point node. The sum of the element values ​​in that row is divided by the sum of the confidence scores of each mapping to obtain a weighted average value, which is used as the attainability and mastery of the job knowledge point under the current student's knowledge structure. The attainability and mastery of each job knowledge point is compared with the mastery threshold to classify the difficulty level of the job knowledge point.

8. The interview question generation method based on a large language model according to claim 1, characterized in that, Each job-related knowledge point in the set of knowledge points to be assessed is used as the core examination point of the question. Prompts for question generation are constructed sequentially and input into a pre-defined large language model to generate mock interview questions. After the mock interview questions are generated, a learning stage adaptability check is performed. All job-related knowledge points involved in the mock interview questions are analyzed, and each job-related knowledge point involved in each mock interview question is matched with course knowledge nodes in the student's professional ability development map to determine whether the mock interview question passes the adaptability check. If there are course knowledge nodes in the student's professional ability development map that match the job-related knowledge points involved in the mock interview questions, and the proportion of nodes outside the scope exceeds a pre-defined tolerance threshold, the current mock interview question is considered to have passed the adaptability check. Mock interview questions that fail the adaptability check are regenerated until they pass the adaptability check, and all mock interview questions that pass the adaptability check are output.

9. An interview question generation device based on a large language model, applied to the interview question generation method based on a large language model as described in any one of claims 1-8, characterized in that, include: The professional analysis module is used to obtain the target students' professional information and learning stage information, construct the students' professional ability development map based on the prerequisite dependencies between courses in the professional training program, and assign mastery weights to each node based on the grades of the courses already completed. The intention analysis module is used to obtain the intended employment directions selected by target students, retrieve core competency requirements, and construct a job competency requirement map. The association mapping module is used to semantically map students' professional ability development map with job ability requirement map to generate an ability coverage association matrix; The difficulty classification module is used to calculate the attainability of knowledge points for each position based on the ability coverage correlation matrix, and classify the difficulty level of knowledge points according to the attainability. The assessment analysis module is used to obtain the proportion of questions at each difficulty level set by the target students, and to extract job knowledge points from each knowledge point set according to a differentiated extraction strategy to form a set of knowledge points to be assessed. The question generation module is used to construct prompts based on the knowledge points of each position in the set of knowledge points to be assessed, input them into a large language model to generate mock interview questions, and perform learning stage adaptability verification on the generated mock interview questions.

10. A storage medium, characterized in that, The system stores instructions that, when executed on a computer, cause the computer to perform the interview question generation method based on a large language model as described in any one of claims 1-8.

Images