Multi-modal online test question recommendation method and system based on end-cloud cooperation, and medium

Abstract

The application discloses a kind of multi-modal online test question recommendation method, system and medium based on end cloud cooperation.The method obtains the test record of user, question attribute and expression information in the process of answering by obtaining the device end, carries out cognitive diagnosis to obtain user knowledge radar chart diagnosis result in the device end, and realizes adaptive test question recommendation according to diagnosis result.To solve the limitation of device end recommendation question bank, cloud end question selection strategy and device end question selection strategy are designed, and a variety of questions can be selected from the angle of question type and quality according to user demand, which are downloaded to the user by the cloud, and the redundant questions in the device end are removed.The method of the application realizes adaptive examination under the cooperation of end and cloud, accurately diagnoses by fusing user expression information, and realizes personalized test question recommendation according to diagnosis result, solves the problem that traditional method cannot be applied to large-scale question bank, and has the advantages of good recommendation effect, accurate diagnosis and the like.

Description

Technical Field

[0001] This invention belongs to the field of intelligent education with online information recommendation, and in particular relates to a multimodal online test question recommendation method, system and medium based on edge-cloud collaboration. Background Technology

[0002] Designing effective tests to assess test-takers' knowledge is a fundamental task in many real-world scenarios, including education, gaming, and job recruitment. Traditionally, teachers assess students' learning by administering paper-and-pencil tests, carefully selecting questions for them to answer. While this simple approach is effective, it provides all test-takers with the same environment, making it difficult to ensure the appropriateness of the selected questions. Therefore, Computer Adaptive Testing (CAT) emerged. This form of testing aims to personalize and adapt the test according to each test-taker's knowledge level, progressively customizing questions to improve the test's effectiveness. In fact, CAT offers many advantages, including improved accuracy, enhanced security, and increased test-taker engagement, and has been adopted in numerous examinations, such as the Graduate Management Admission Test (GMAT) and the Graduate Record Examination (GRE).

[0003] A typical CAT system usually comprises two key components: (1) a cognitive diagnostic model (CDM), which estimates the candidate's knowledge state based on their performance; and (2) a selection strategy, which selects questions from a question pool for the candidate to answer. Specifically, the CAT system can establish an interactive testing process when the candidate begins the test. In the first step, the system presents a question (e.g., q_t), which the candidate reads and answers. Upon receiving the result (correct or incorrect), the CDM estimates the current state and selects a new question q for the next round based on this. t+1 This process is repeated multiple times until a termination condition is met, such as reaching the maximum test length, to assess the candidate's mastery of the learned knowledge points (e.g., "functions" in mathematics). In this way, even starting with the same questions, personalized assessments can be achieved. Therefore, the key issue is how to build a CAT system capable of selecting appropriate questions to achieve more effective knowledge assessment.

[0004] Traditional CAT systems exhibit insufficient linear retrieval speed when faced with massive question banks, resulting in excessively long processing times in practical applications, failing to meet real-world requirements. This problem is particularly pronounced when considering factors such as communication overhead between the device and the cloud, as well as the time cost of computationally complex question selection strategies. Consequently, the practical application performance of existing CAT systems in the context of large-scale question banks is unsatisfactory.

[0005] In today's big data era, building an efficient computer adaptive testing system, especially when dealing with large question banks, has become an urgent problem to solve. How to effectively coordinate question banks from the device to the cloud while ensuring real-time performance, and how to handle information gaps, timeliness issues, and redundant questions in large question banks, are all challenges currently faced by CAT systems. Summary of the Invention

[0006] The purpose of this invention is to solve the problems existing in the prior art and to provide a multimodal online test question recommendation method, system and medium based on edge-cloud collaboration, so as to recommend personalized test questions to different users, efficiently diagnose users' learning levels and provide adaptive testing services for teaching.

[0007] The specific technical solution adopted in this invention is as follows:

[0008] In a first aspect, the present invention provides a multimodal online test question recommendation method based on edge-cloud collaboration, comprising the following steps:

[0009] S1. The cloud retrieves the knowledge points tested by the questions and the difficulty level of the questions from the pre-built global question bank, constructs a two-dimensional knowledge point mapping matrix to map the relationship between each question and the knowledge points tested by each question, constructs a two-dimensional difficulty level matrix to map the relationship between each question and the difficulty level of each question, and sends the two-dimensional knowledge point mapping matrix and the two-dimensional difficulty level matrix to the device.

[0010] The difficulty level of the question is either easy or hard. For the current question in the global question bank, the number of times each user answered correctly and the number of times they answered incorrectly in the history are counted. If the number of correct answers is greater than or equal to the number of incorrect answers, the difficulty level of the current question is set to easy; otherwise, the difficulty level of the current question is set to hard.

[0011] S2. Before the user on the device starts answering questions, the cloud randomly selects questions from the global question bank to build a recommended question bank and sends it to the device. For the current set of questions in the recommended question bank, the device takes a picture of the user answering the current set of questions to obtain the facial image corresponding to each user. The device inputs the collected facial images into a trained expression recognition classifier model to classify the expression information and obtain the classification result of the user's expression. After the user completes the current set of questions, the device obtains the user's answer record information, which includes the user's answer result and the answer time.

[0012] S3. The device inputs the user ID of the recommended question bank, the user's answer record information, the user's expression classification results, and the constructed two-dimensional knowledge point mapping matrix and two-dimensional difficulty level matrix into the cognitive diagnostic model trained with the previous set of questions, and obtains the user knowledge radar map diagnostic results and sends them to the cloud.

[0013] The cognitive diagnostic model is deployed on the device. After a user completes the current set of questions, the current set of questions is deleted from the recommended question bank. The user ID of the user who answered the recommended question bank, the user's answer record information, the classification results of the user's facial expressions, the constructed two-dimensional knowledge point mapping matrix and two-dimensional difficulty level matrix, and the user's knowledge radar chart diagnostic results are re-inputted into the cognitive diagnostic model for training.

[0014] S4. The device determines whether there are any remaining questions in the recommended question bank: If not, the cloud automatically selects a recommended question bank that matches the user's current knowledge status from the global question bank based on the pre-built cloud question selection strategy and the user's knowledge radar chart diagnostic results; if there are, the device selects a set of questions from the recommended question bank for the user based on the pre-built device question selection strategy.

[0015] S5. On the device side, iterate through the distribution time of each question in the recommended question bank and determine whether the difference between the current time during the iteration and the distribution time is greater than or equal to the preset expiration time. If it is satisfied, remove the corresponding question from the recommended question bank on the device side. If it is not satisfied, keep the corresponding question in the recommended question bank on the device side.

[0016] Preferably, in step S1, the two-dimensional knowledge point mapping matrix and the two-dimensional difficulty level matrix are constructed in the following way:

[0017] S11. Number all questions in the global question bank, number the knowledge points tested by the questions, and initialize the two-dimensional knowledge point mapping matrix and the two-dimensional difficulty level matrix respectively. After initialization, the value of each element in the two-dimensional knowledge point mapping matrix and the two-dimensional difficulty level matrix is ​​0.

[0018] S12. For the i-th question in the global question bank, if the i-th question involves the j-th knowledge point, set the value of the element in the i-th row and j-th column of the two-dimensional knowledge point mapping matrix to 1; if the i-th question is easy, set the value of the element in the i-th row and 1-th column of the two-dimensional difficulty level matrix to 1; if the i-th question is difficult, set the value of the element in the i-th row and 2-th column of the two-dimensional difficulty level matrix to 1; where i represents the index of the question and j represents the index of the knowledge point.

[0019] S13. After all questions in the global question bank have been traversed, the constructed two-dimensional knowledge point mapping matrix and two-dimensional difficulty level matrix are obtained.

[0020] Preferably, in step S2, the classification results of the user's facial expressions include a total of 6 types, namely anger, tension, confusion, focus, confidence, and joy.

[0021] Preferably, in step S2, the facial expression recognition classifier uses a 5-layer fully connected layer.

[0022] Preferably, during the training process of the cognitive diagnostic model in step S3, the user's answer to the question is used as the true label. If the answer is correct, the label value is 1; if the answer is incorrect, the label value is 0. The cross-entropy loss between the probability of the user answering the question correctly and the true label is calculated. The parameters of the cognitive diagnostic model are updated based on minimizing the cross-entropy loss until the preset number of iterations is reached, at which point the cognitive diagnostic model converges, and a cognitive diagnostic model trained for the current set of questions is obtained.

[0023] Preferably, in step S4, the device-side topic selection strategy adopts the Search-Efficient Computerized Adaptive Testing (SECAT) framework.

[0024] As a preferred option, the specific process of constructing the cloud-based topic selection strategy in step S4 is as follows:

[0025] S41. The device uploads the user ID of the recommended question bank and the cognitive diagnostic model trained using the previous set of questions to the cloud. After receiving the cloud, the i-th question q from the global question bank Q is uploaded. i The candidate ID ∈Q and the candidate ID who answered the i-th question are input together into the cognitive diagnostic model received in the cloud to obtain the gradient change EMC of the cognitive diagnostic model, where 1≤i≤|Q|, |Q| represents the number of questions in the global question bank, and i represents the index of the question in the global question bank;

[0026] S42. The change in knowledge point coverage for each question in the global question bank in cloud computing, and the i-th question in the global question bank q i Changes in knowledge point coverage IWKC(q) i The calculation method for ) is as follows:

[0027]

[0028] Where J represents the set of knowledge point indices, w j Additional importance weights for the j-th knowledge point; IncCov(j,q) i () represents the increment function, which is calculated as follows:

[0029]

[0030] Wherein, cnt(j,q) i The expression indicates whether the question related to the j-th knowledge point contains the i-th question q in the global question bank. i If it contains, then cnt(j,q) i If the value is 1, then cnt(j,q) is not valid. i The value is 0;

[0031] S43. The cloud uses gradient change EMC as the primary key and knowledge point coverage change IWKC for each question as the secondary key. It sorts the global question bank Q based on the primary and secondary keys, and then selects the top Q from the sorted global question bank. k A set of questions, serving as a recommended question bank tailored to the user's current knowledge level, are distributed to the user on the device, with the time of distribution recorded; among them, Q... k It is a preset positive integer.

[0032] Secondly, this invention provides a multimodal online test question recommendation system based on edge-cloud collaboration, comprising:

[0033] The data acquisition module is used to obtain the knowledge points tested by the questions and the difficulty level of the questions from a pre-built global question bank through the cloud, construct a two-dimensional knowledge point mapping matrix that maps the relationship between each question and the knowledge points tested by each question, construct a two-dimensional difficulty level matrix that maps the relationship between each question and the difficulty level of each question, and send the two-dimensional knowledge point mapping matrix and the two-dimensional difficulty level matrix to the device.

[0034] The difficulty level of the question is either easy or hard. For the current question in the global question bank, the number of times each user answered correctly and the number of times they answered incorrectly in the history are counted. If the number of correct answers is greater than or equal to the number of incorrect answers, the difficulty level of the current question is set to easy; otherwise, the difficulty level of the current question is set to hard.

[0035] The cognitive diagnosis module is used to construct a recommended question bank by randomly selecting questions from a global question bank in the cloud before the user starts answering questions on the device, and then sending it to the device. For the current set of questions in the recommended question bank, the module takes a picture of the user answering the current set of questions to obtain the corresponding facial image of each user. The collected facial images are input into a trained facial expression recognition classifier model to classify facial expression information and obtain the classification result of the user's facial expression. After the user completes the current set of questions, the module obtains the user's answer record information, which includes the user's answer result and the answer time.

[0036] The diagnostic results acquisition module is used to input the user ID of the answer recommendation question bank, the user's answer record information, the user's expression classification results, as well as the constructed two-dimensional knowledge point mapping matrix and two-dimensional difficulty level matrix into the cognitive diagnostic model trained using the previous set of questions, to obtain the user knowledge radar chart diagnostic results and send them to the cloud.

[0037] The cognitive diagnostic model is deployed on the device. After a user completes the current set of questions, the current set of questions is deleted from the recommended question bank. The user ID of the user who answered the recommended question bank, the user's answer record information, the classification results of the user's facial expressions, the constructed two-dimensional knowledge point mapping matrix and two-dimensional difficulty level matrix, and the user's knowledge radar chart diagnostic results are re-inputted into the cognitive diagnostic model for training.

[0038] The adaptive question selection module is used to determine whether there are any remaining questions in the recommended question bank on the device: if not, the cloud automatically selects a recommended question bank that matches the user's current knowledge status from the global question bank based on the pre-built cloud question selection strategy and the user's knowledge radar chart diagnostic results; if there are remaining questions, the device selects a set of questions from the recommended question bank for the user based on the pre-built device question selection strategy.

[0039] The question bank filtering and processing module is used to traverse the distribution time of each question in the device's recommended question bank and determine whether the difference between the current time during the traversal and the distribution time is greater than or equal to the preset expiration time. If it is satisfied, the corresponding question is removed from the device's recommended question bank; otherwise, the corresponding question is retained in the device's recommended question bank.

[0040] Thirdly, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the multimodal online test recommendation method based on end-to-cloud collaboration as described in the first aspect above.

[0041] Fourthly, the present invention provides a computer electronic device, including a memory and a processor;

[0042] The memory is used to store computer programs;

[0043] The processor is configured to, when executing the computer program, implement the multimodal online test question recommendation method based on edge-cloud collaboration as described in the first aspect above.

[0044] Compared with the prior art, the present invention has the following advantages:

[0045] This invention provides a multimodal online test question recommendation method based on edge-cloud collaboration. This method addresses the problem that the linear complexity of question selection strategies in traditional computer adaptive testing is unsuitable for large question banks by distributing a portion of the global question bank to the device. Simultaneously, considering the information gaps inherent in the device-side recommended question bank, a cloud-based question selection strategy and a device-side question selection strategy are designed. This allows for the selection of diverse questions based on user needs in terms of question type and quality. The cloud adaptively requests these questions to be distributed to the user, while redundant questions are removed from the device. The cloud-based question selection strategy is redesigned from the perspectives of type and quality, thus constructing a novel and efficient computer adaptive testing paradigm. Extensive experiments have been conducted, and the results show that the method of this invention has excellent effectiveness and efficiency. Compared with traditional CAT methods, this invention has a greater advantage when dealing with large question banks. Attached Figure Description

[0046] Figure 1 This is a flowchart of the steps of the present invention;

[0047] Figure 2 This is a scene diagram of a traditional CAT system;

[0048] Figure 3 This is a CAT system scenario diagram that integrates user response expressions in the context of edge-cloud collaboration. Detailed Implementation

[0049] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. Technical features in the various embodiments of the present invention can be combined accordingly without mutual conflict.

[0050] In the description of this invention, it should be understood that the terms "first" and "second" are used only for descriptive purposes and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first" and "second" may explicitly or implicitly include at least one of those features.

[0051] This invention can extract and analyze features from users' question-answering text information, and adaptively recommend personalized test questions to different users. Currently, such online information recommendation technology is used in examinations, such as GRE and GMAT. The recommended test questions will become more difficult as the test taker's level increases. Furthermore, by fusing multimodal data (photos of users answering questions), the accuracy of predicting users' cognitive state can be further improved, thus recommending test questions more efficiently.

[0052] This invention, in the context of edge-cloud collaboration, provides a solution for assessing users' knowledge levels by integrating text information from their answers and facial expressions. Existing solutions often suffer from several issues. For example, the closest comparable case is CN106407237B, which, as a cloud-centric paradigm, employs a linear complexity question selection strategy, making it unsuitable for large-scale question banks. Furthermore, relying solely on textual information leads to information gaps and susceptibility to environmental interference, such as user guessing and incorrect answers. This invention, however, also incorporates user facial expression modalities and utilizes a model architecture to dynamically and adaptively generate test questions for test-takers in an edge-cloud collaborative scenario.

[0053] In a preferred embodiment of the present invention, such as Figure 3 As shown, a multimodal online test question recommendation method based on edge-cloud collaboration is provided, which improves upon traditional CAT methods (such as...). Figure 2 To address the issue that the linear time complexity of the question selection strategy (as shown in the diagram) is too high to be used in large-scale question banks, and to further address the limitations of the recommended question bank itself, a novel question selection strategy was designed in the cloud that takes into account the diversity of knowledge points. This strategy adaptively requests the question bank and removes redundant questions. This method not only solves the problem that traditional CAT systems cannot be truly applied to large-scale question banks, but also ensures that the cognitive level of users is effectively tested while keeping the test length as short as possible, thereby helping candidates to conduct adaptive testing more efficiently and effectively.

[0054] like Figure 1 As shown, in a preferred embodiment of the present invention, the above-mentioned multimodal online test question recommendation method based on edge-cloud collaboration includes the following steps S1 to S5. The specific implementation process of each step will be described in detail below.

[0055] S1. The cloud retrieves question attributes from a pre-built global question bank. The question attributes include the knowledge points tested by the question and the difficulty level of the question. It constructs a two-dimensional knowledge point mapping matrix concept_map to map the relationship between each question and the knowledge points tested by each question, and constructs a two-dimensional difficulty level matrix diff_map to map the relationship between each question and the difficulty level of each question. The cloud then sends the two-dimensional knowledge point mapping matrix and the two-dimensional difficulty level matrix to the device.

[0056] The difficulty level of the question is either easy or hard. For the current question in the global question bank, the number of times each user answered correctly and incorrectly in the historical records are counted. If the number of correct answers is greater than or equal to the number of incorrect answers, the difficulty level of the current question is set to easy; otherwise, the difficulty level of the current question is set to hard.

[0057] It should be noted that in step S1 of this invention, the two-dimensional knowledge point mapping matrix and the two-dimensional difficulty level matrix are constructed in the following way:

[0058] S11. Number all questions in the global question bank from 1 to n, and number the knowledge points tested by the questions from 1 to m. Initialize the two-dimensional knowledge point mapping matrix concept_map and the two-dimensional difficulty level matrix diff_map respectively. After initialization, the value of each element in the two-dimensional knowledge point mapping matrix and the two-dimensional difficulty level matrix is ​​0.

[0059] S12. For the i-th question in the global question bank, if the i-th question involves the j-th knowledge point, set the value of the element in the i-th row and j-th column of the two-dimensional knowledge point mapping matrix to 1; if the i-th question is easy, set the value of the element in the i-th row and 1-th column of the two-dimensional difficulty level matrix diff_map to 1; if the i-th question is difficult, set the value of the element in the i-th row and 2-th column of the two-dimensional difficulty level matrix diff_map to 1; where i represents the index of the question and j represents the index of the knowledge point.

[0060] S13. After all questions in the global question bank have been traversed, the constructed two-dimensional knowledge point mapping matrix concept_map and the two-dimensional difficulty level matrix diff_map are obtained.

[0061] In this embodiment, the Assist2009 dataset is first downloaded. Then, all users' relevant question-answering records are extracted from this dataset. The knowledge points corresponding to each question are counted, and a two-dimensional knowledge point mapping matrix, concept_map, is constructed to represent the mapping relationship between questions and knowledge points. Each row of this matrix represents each question, and each column represents each knowledge point. Then, for each question in the global question bank, the number of times each user answered correctly and incorrectly in the historical records is counted. For a question, if there are many correct answers, it is set as an easy question; if there are many incorrect answers, it is set as a difficult question. The number of correct and incorrect answers for each question is counted, and average normalization is performed (all questions in the global question bank are numbered from 1 to n) to obtain the difficulty level of each question. A two-dimensional difficulty level matrix, diff_map, is constructed to represent the mapping relationship between questions and question difficulty levels. Each row of this matrix represents each question, and each column represents the difficulty level of each question.

[0062] S2. Before the user on the device starts answering questions, the cloud randomly selects questions from the global question bank to build a recommended question bank and sends it to the device. For the current set of questions in the recommended question bank, the device takes a picture of the user answering the current set of questions to obtain the facial image of each user. The device inputs the collected facial images into a trained expression recognition classifier model to classify the expression information and obtain the classification result of the user's expression. After the user completes the current set of questions, the device obtains the user's answer record information, which includes the user's answer result and the answer time.

[0063] It should be noted that in step S2 of the present invention, the classification results of the user's facial expressions include a total of 6 types, namely anger, tension, confusion, focus, confidence, and joy.

[0064] It should be noted that in step S2 of the present invention, the facial expression recognition classifier uses a 5-layer fully connected layer.

[0065] In this embodiment, the device takes photos of the user currently answering questions, obtaining multiple images for each user. Then, a clear and high-resolution key image is selected from these images as the corresponding facial image for each user. After obtaining the user's facial image, a pre-trained existing facial expression recognition classifier model (such as MLP) is used to classify facial expression information and predict the user's level of seriousness in answering the current question, resulting in a classification result of the user's facial expression, such as anger, tension, confusion, focus, confidence, and joy, and assigning a label from 0 to 5. Simultaneously, the user's answering record information also needs to be recorded. After the user completes the current set of questions, the user's answering record information is obtained, such as the user's answer result and the time taken to complete the questions—key information reflecting the user's cognitive state.

[0066] S3. The device inputs the user ID of the recommended question bank, the user's answer record information, the user's expression classification results, and the constructed two-dimensional knowledge point mapping matrix and two-dimensional difficulty level matrix into the cognitive diagnostic model trained using the previous set of questions, and obtains the user knowledge radar map diagnostic results and sends them to the cloud.

[0067] The cognitive diagnostic model is deployed on the device. After a user completes the current set of questions, the current set of questions is deleted from the recommended question bank. The user ID of the user who answered the recommended question bank, the user's answer record information, the user's expression classification results, the constructed two-dimensional knowledge point mapping matrix and two-dimensional difficulty level matrix, and the user's knowledge radar chart diagnostic results are re-inputted into the cognitive diagnostic model for training.

[0068] In this embodiment, the user ID e that answers the question will be... k The pre-constructed two-dimensional knowledge point mapping matrix (concept_map) and two-dimensional difficulty level matrix (diff_map) are used as inputs to the traditional deep cognitive diagnostic model. The user's facial expression classification results are used as auxiliary information for the cognitive diagnostic model. The device-side cognitive diagnostic model (e.g., NeuralCognitive Diagnosis Model, NeuralCDM) performs text-level cognitive diagnosis based on the user's response records, obtaining a text-level user knowledge state representation θ. The user's facial expression classification result ∈ can be used as a graph-level user knowledge representation. Therefore, at the decision layer, the user's facial expression classification result ∈ and the user knowledge state representation θ are fused, and a diagnostic report is given, i.e., the user knowledge radar chart diagnostic result θ. * =∈·θ, where · represents tensor broadcast multiplication. This is quite flexible; factors can be adjusted based on the evaluator's understanding of the real-world environment. For example, if the user is currently confused, but the text-level diagnostic result predicts the user's answer is correct, then the result is likely influenced by the guessing factor, meaning the weight of ∈ can be adjusted according to the actual situation. After the user completes the current set of questions, the user's knowledge radar map diagnostic result is used as new input to train the cognitive diagnostic model.

[0069] It should be noted that in the training process of the cognitive diagnostic model in step S3 of this invention, the user's answer to the question is used as the true label. If the answer is correct, the label value is 1, and if the answer is incorrect, the label value is 0. The cross-entropy loss between the probability of the user answering the question correctly and the true label is calculated. The parameters of the cognitive diagnostic model are updated based on minimizing the cross-entropy loss until the preset number of iterations is reached, the cognitive diagnostic model converges, and the cognitive diagnostic model trained for the current set of questions is obtained.

[0070] In this embodiment, the user's answer to the question is used as the true label, where a true label of 0 indicates an incorrect answer and a true label of 1 indicates a correct answer. The probability θ of the user answering the question correctly is calculated. * The binary cross-entropy loss BCE(θ) between the true label and the actual label * The parameters are backpropagated based on minimizing the cross-entropy loss (label), and training begins.

[0071] S4. The device determines whether there are any remaining questions in the recommended question bank: If not, the cloud automatically selects a recommended question bank that matches the user's current knowledge status from the global question bank based on the pre-built cloud question selection strategy and the user's knowledge radar chart diagnostic results; if there are, the device selects a set of questions from the recommended question bank for the user based on the pre-built device question selection strategy.

[0072] In step S4 of this invention, the device-side question selection strategy is very flexible. The framework of this invention is independent of existing question selection strategies. The device can select an existing question selection strategy for the user to select questions online, and the user answers the questions after receiving them. Currently, some of the better question selection strategies include BECAT and SECAT. In this embodiment, the device-side question selection strategy adopts Search-Efficient Computerized Adaptive Testing (SECAT).

[0073] It should be noted that the specific process of constructing the cloud-based topic selection strategy in step S4 of this invention is as follows:

[0074] S41. The device uploads the user ID of the recommended question bank and the cognitive diagnostic model trained using the previous set of questions to the cloud. After receiving the cloud, the i-th question q from the global question bank Q is uploaded. i The candidate ID for answering the i-th question is input together with ∈Q into the cognitive diagnostic model received in the cloud to obtain the gradient change EMC of the cognitive diagnostic model, where 1≤i≤|Q|, |Q| represents the number of questions in the global question bank, and i represents the index of the question in the global question bank.

[0075] It should be noted that, in this invention, the principle of obtaining the gradient change EMC of the cognitive diagnostic model in step S41 is as follows:

[0076] This invention proposes a scoring function, namely the gradient change EMC of the cognitive diagnostic model, which quantifies the quality of a question by measuring its information content, i.e., how much information it contains. This gradient change does not depend on the detailed information of the cognitive diagnostic model. The estimation of the user's knowledge state is updated by observing the user's answers, utilizing the information contained in the question. Therefore, the information content of a question can be scored by the degree of diagnostic change; the degree of change in the cognitive diagnostic model signifies the amount of information extracted from the question. Specifically, if the weights of the cognitive diagnostic model change significantly, the question can be considered informational; conversely, if the weights of the cognitive diagnostic model change almost no, the answer to the question provides little information. However, the problem with this approach is that it is impossible to know the user's answer before selecting a question. Therefore, this invention proposes the gradient change EMC of the cognitive diagnostic model, where the probability is predicted by the cognitive diagnostic model M (e.g., Neural Cognitive Diagnosis Model, NeuralCDM). Formally, let ΔM(r ki )=|θ(R k ∪{r ki})-θ(R k )| indicates adding a record r ki = <e k ,q i ,a ki The resulting gradient change in the cognitive diagnostic model, where R... k This represents the problem-solving record of the k-th user. ki = <e k ,q i ,a ki > represents the answer status of the k-th user ID, the i-th question ID in the global question bank, and the k-th user's answer to the i-th question in the global question bank. ki The triples formed, {r ki} represents the triple r ki The set formed, e k ,q i ,a ki Let a represent the ID of the k-th user, the i-th question in the global question bank, and the answer status of the k-th user on the i-th question in the global question bank, respectively. ki (0 indicates an incorrect answer, 1 indicates a correct answer). ΔM(r) ki ) indicates adding a record r ki Gradient changes in the post-cognitive diagnostic model. θ(R) k ∪{r ki}) represents the problem-solving record R of the k-th user. k Add record r ki After being input into the cognitive diagnostic model M, the model parameters of the trained cognitive diagnostic model M are: θ(R) kR represents the problem-solving record of the k-th user. k The model parameters of cognitive diagnostic model M are obtained after inputting them into the cognitive diagnostic model M for training.

[0077] For each question q in each global question bank Q i This invention defines the gradient change EMC of the cognitive diagnostic model as follows:

[0078]

[0079] p = M(e k ,q i ∣θ(R k ))

[0080] Among them, EMC(q) i ) represents the i-th question q i The resulting gradient change in the cognitive diagnostic model M Represents variable a ki Based on the expected value calculated from the probability distribution p, ΔM represents the gradient change of the cognitive diagnostic model M, and p represents the gradient change of the cognitive diagnostic model M in the parameter θ(R). k ) condition, e k ,q i The output probability is obtained as the input to the cognitive diagnostic model M, where M represents the cognitive diagnostic model. To improve computational efficiency, this invention uses r ki The resulting gradient is used to approximate ΔM(r) ki Instead of retraining the cognitive diagnostic model, this approximation is used. This approximation is particularly efficient for CDMs that use gradient-based training, such as neural network models.

[0081] S42. The change in knowledge point coverage for each question in the global question bank in cloud computing, and the i-th question in the global question bank q i Changes in knowledge point coverage IWKC(q) i The calculation method for ) is as follows:

[0082]

[0083] Where J represents the set of knowledge point indices, w j Additional importance weights for the j-th knowledge point; IncCov(j,q) i () represents the increment function, which is calculated as follows:

[0084]

[0085] Wherein, cnt(j,q) i The expression indicates whether the question related to the j-th knowledge point contains the i-th question q in the global question bank. i If it contains, then cnt(j,q)i If the value is 1, then cnt(j,q) is not valid. i The value is 0.

[0086] It should be noted that in this invention, the principle of calculating the knowledge point coverage change IWKC in step S42 is as follows: First, it starts with a simple Naive Knowledge Coverage (NKC) function, which simply calculates the i-th question q in the global question Q. i The proportion of knowledge concepts covered in all knowledge concepts:

[0087]

[0088] Among them, NKC(q) i ) represents the i-th question q i The proportion of related knowledge points to the total number of knowledge points, j∈J represents the knowledge point index in the set J consisting of knowledge point indices, ∑ j∈J Cov(j,q i ) indicates the relationship with question q i The number of related knowledge points This indicates whether the two-dimensional knowledge point mapping matrix concept_map[i][j] is equal to 1. If it is equal to 1, then... The value is 1 otherwise it is 0.

[0089] NKC has two main drawbacks: (1) it treats all knowledge concepts equally and cannot distinguish their importance, which may lead to unreasonable domain coverage; (2) it is too strict, and the binary condition is always 1 regardless of how many issues are involved, which may lead to an imbalance in knowledge coverage.

[0090] To address these issues, this invention proposes an advanced scoring function, IWKC, by introducing importance weights and soft-form incremental attributes:

[0091]

[0092] Among them, w j This is the additional importance weight of the knowledge point, which is a constant. IncCov(j,q) i ) represents Cov(j,q i Incremental improvements. For example, when the number of questions associated with j is 0, 1, 2, or 3, IncCov(j,q) i The values ​​gradually reached 0, 0.5, 0.67, 0.75, etc., while Cov(j,q) i The value jumps discontinuously between 0 and 1. The next step is to calculate the importance weight w in IWKC. jThe invention addresses the issue of representativeness. To quantify the representativeness of the questions, this invention first represents them using feature vectors so that each question can be considered a point in the embedded metric space. The closer a question is to its neighbors, the more representative it is. The importance weight of each knowledge concept is obtained by calculating the average of the representativeness (i.e., density) of its related questions. To perform the calculation, this invention utilizes historical student response data. Specifically, this invention has records of historical test takers preserved and used for training, represented as (E... H ,R H This invention embeds similarity as a test effect. To train the test effect embedding, this invention borrows the idea from Item2Vec. Specifically, for each historical record r... ki = <e k ,q i ,a ki >, This invention sets an input x ki This indicates which question was answered and whether the answer was correct:

[0093]

[0094] Among them, 1 |Q| (i) represents a one-hot vector of length |Q|, where only position i is equal to 1 or 0. |Q| This represents the zero vector of length |Q|. This represents vector concatenation. Then, this invention trains a Skip-Gram Negative Sampling (SGNS) model. Specifically, given a historical subject e... k ∈E H The optimization objective is defined as:

[0095]

[0096] in, This represents the set of historical problem-solving records for the k-th user. r represents the record of the k-th user's answer to the i-th question in their historical problem-solving records. ki , r represents the record of the k-th user's historical problem-solving process when answering the j-th question. kj , p(r kj ∣r ki ) represents σ((Wx ki ) T v j ) represents the Sigmoid activation function, W represents the learnable parameter matrix, and x ki This refers to the input x mentioned earlier. ki (Indicates which question was answered and whether the answer was correct), vj Representing the question q j The embedding vector, N neg Indicates the size of the negative sample. Indicate whether the question has been answered. and problems Was the answer correct? Indicate the problem The embedding vector.

[0097] Since the problem has already been represented in the test effect embedding space, this invention quantifies the representativeness of the problem. First, this invention calculates two problems q. i and q j Similarity of test results between them:

[0098]

[0099] Where |·| represents the Euclidean norm, and γ is the positive smoothing parameter. They represent q respectively i and q j The test results are embedded.

[0100] Next, define each problem q j The test effect density is the average similarity of its neighbors:

[0101]

[0102] in, Includes test results embedded in the space q j K N The nearest neighbor. K N This represents the preset value. The test results are embedded in the question space q. j K N Each question q of the nearest neighbors i The higher the test effect density, the more representative the problem is. Finally, the importance of each knowledge point is defined by averaging the test effect density of related problems.

[0103]

[0104] S43. The cloud uses gradient change EMC as the primary key and knowledge point coverage change IWKC for each question as the secondary key. It sorts the global question bank Q based on the primary and secondary keys, and then selects the top Q from the sorted global question bank. k A set of questions, serving as a recommended question bank tailored to the user's current knowledge level, are distributed to the user on the device, with the time of distribution recorded; among them, Q... k It is a preset positive integer.

[0105] S5. On the device side, iterate through the distribution time T of each question in the recommended question bank and determine whether the difference between the current time T' during the iteration and the distribution time T is greater than or equal to the preset expiration time Time_S. If T'-T>=Time_S, then remove the corresponding question from the recommended question bank on the device side. If not, then keep the corresponding question in the recommended question bank on the device side.

[0106] It should also be noted that the multimodal online test question recommendation method based on edge-cloud collaboration in the above embodiments can essentially be executed by a computer program or module. Therefore, similarly, based on the same inventive concept, another preferred embodiment of the present invention also provides a multimodal online test question recommendation system based on edge-cloud collaboration, corresponding to the multimodal online test question recommendation method based on edge-cloud collaboration provided in the above embodiments, which includes:

[0107] The data acquisition module is used to obtain the knowledge points tested by the questions and the difficulty level of the questions from a pre-built global question bank through the cloud, construct a two-dimensional knowledge point mapping matrix that maps the relationship between each question and the knowledge points tested by each question, construct a two-dimensional difficulty level matrix that maps the relationship between each question and the difficulty level of each question, and send the two-dimensional knowledge point mapping matrix and the two-dimensional difficulty level matrix to the device.

[0108] The difficulty level of the question is either easy or hard. For the current question in the global question bank, the number of times each user answered correctly and the number of times they answered incorrectly in the history are counted. If the number of correct answers is greater than or equal to the number of incorrect answers, the difficulty level of the current question is set to easy; otherwise, the difficulty level of the current question is set to hard.

[0109] The cognitive diagnosis module is used to construct a recommended question bank by randomly selecting questions from a global question bank in the cloud before the user starts answering questions on the device, and then sending it to the device. For the current set of questions in the recommended question bank, the module takes a picture of the user answering the current set of questions to obtain the corresponding facial image of each user. The collected facial images are input into a trained facial expression recognition classifier model to classify facial expression information and obtain the classification result of the user's facial expression. After the user completes the current set of questions, the module obtains the user's answer record information, which includes the user's answer result and the answer time.

[0110] The diagnostic results acquisition module is used to input the user ID of the answer recommendation question bank, the user's answer record information, the user's expression classification results, as well as the constructed two-dimensional knowledge point mapping matrix and two-dimensional difficulty level matrix into the cognitive diagnostic model trained using the previous set of questions, to obtain the user knowledge radar chart diagnostic results and send them to the cloud.

[0111] The cognitive diagnostic model is deployed on the device. After a user completes the current set of questions, the current set of questions is deleted from the recommended question bank. The user ID of the user who answered the recommended question bank, the user's answer record information, the classification results of the user's facial expressions, the constructed two-dimensional knowledge point mapping matrix and two-dimensional difficulty level matrix, and the user's knowledge radar chart diagnostic results are re-inputted into the cognitive diagnostic model for training.

[0112] The adaptive question selection module is used to determine whether there are any remaining questions in the recommended question bank on the device: if not, the cloud automatically selects a recommended question bank that matches the user's current knowledge status from the global question bank based on the pre-built cloud question selection strategy and the user's knowledge radar chart diagnostic results; if there are remaining questions, the device selects a set of questions from the recommended question bank for the user based on the pre-built device question selection strategy.

[0113] The question bank filtering and processing module is used to traverse the distribution time of each question in the device's recommended question bank and determine whether the difference between the current time during the traversal and the distribution time is greater than or equal to the preset expiration time. If it is satisfied, the corresponding question is removed from the device's recommended question bank; otherwise, the corresponding question is retained in the device's recommended question bank.

[0114] Similarly, based on the same inventive concept, another preferred embodiment of the present invention also provides a computer electronic device corresponding to the multimodal online test question recommendation method based on end-to-cloud collaboration provided in the above embodiments, which includes a memory and a processor;

[0115] The memory is used to store computer programs;

[0116] The processor is configured to implement the multimodal online test question recommendation method based on edge-cloud collaboration in the above embodiments when executing the computer program.

[0117] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0118] Therefore, based on the same inventive concept, another preferred embodiment of the present invention also provides a computer-readable storage medium corresponding to the multimodal online test question recommendation method based on end-to-cloud collaboration provided in the above embodiments. The storage medium stores a computer program, which, when executed by a processor, can realize the multimodal online test question recommendation method based on end-to-cloud collaboration in the above embodiments.

[0119] It is understood that the aforementioned storage media may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Furthermore, the storage media may also be various media capable of storing program code, such as USB flash drives, external hard drives, magnetic disks, or optical discs.

[0120] It is understood that the aforementioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0121] It should also be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the system described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here. In the embodiments provided in this application, the division of steps or modules in the system and method is merely a logical functional division, and there may be other division methods in actual implementation. For example, multiple modules or steps may be combined or integrated together, and a module or step may also be split.

[0122] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the invention. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention. Therefore, all technical solutions obtained through equivalent substitution or transformation fall within the protection scope of the present invention.

Claims

1. A multi-modal online test question recommendation method based on end-cloud collaboration, characterized in that, The specific steps are as follows: S1. The cloud obtains the knowledge points and the difficulty levels of the questions from the pre-constructed global question bank, constructs a two-dimensional knowledge point mapping matrix of the mapping relationship between each question and each knowledge point examined by the question, constructs a two-dimensional difficulty level matrix of the mapping relationship between each question and each difficulty level of the question, and sends the two-dimensional knowledge point mapping matrix and the two-dimensional difficulty level matrix to the device end; The difficulty level of the question is a simple question or a difficult question. For the current question in the global question bank, the number of correct answers and the number of wrong answers of all users in the historical records are counted respectively. If the number of correct answers is greater than or equal to the number of wrong answers, the difficulty level of the current question is set to a simple question, otherwise, the difficulty level of the current question is set to a difficult question; S2. Before the user of the device end starts answering the question, the cloud randomly selects questions from the global question bank to construct a recommended question bank and sends it to the device end. For the current set of questions in the recommended question bank, the device end takes a photo of the user who is currently answering the current set of questions to obtain a face image corresponding to each user; The device end inputs the collected face image into the trained expression recognition classifier model for expression information classification to obtain a classification result of the user's expression; after the user completes the current set of questions, the device end obtains the user's answer record information, which is the user's answer result and the time spent on answering the question; S3. The device end inputs the user number of the user who answers the recommended question bank, the user's answer record information, the classification result of the user's expression, and the constructed two-dimensional knowledge point mapping matrix and two-dimensional difficulty level matrix into the cognitive diagnosis model trained by the previous set of questions to obtain a user knowledge radar chart diagnosis result and send it to the cloud end; The cognitive diagnosis model is deployed on the device end. After the user answers the current set of questions, the answered current set of questions is deleted from the recommended question bank. The user number of the user who answers the recommended question bank, the user's answer record information, the classification result of the user's expression, the constructed two-dimensional knowledge point mapping matrix and two-dimensional difficulty level matrix, and the user knowledge radar chart diagnosis result are re-input into the cognitive diagnosis model for training; S4. The device end judges whether there are remaining questions in the recommended question bank: if not, the cloud automatically selects a recommended question bank matching the current knowledge state for the user in the global question bank according to the pre-constructed cloud question selection strategy and the user knowledge radar chart diagnosis result; if there are, the device end selects a set of questions for the user from the recommended question bank according to the pre-constructed device end question selection strategy; S5. The device end traverses the delivery time of each question in the recommended question bank, judges whether the difference between the current time and the delivery time during traversal satisfies the preset set expiration time: if it satisfies, the corresponding question is removed from the recommended question bank of the device end, if it does not satisfy, the corresponding question is retained in the recommended question bank of the device end.

2. The multi-modal online test question recommendation method based on end-cloud collaboration according to claim 1, characterized in that, In step S1, the two-dimensional knowledge point mapping matrix and the two-dimensional difficulty level matrix are constructed as follows: S11. All the questions in the global question bank are numbered, the knowledge points examined by the questions are numbered, and the two-dimensional knowledge point mapping matrix and the two-dimensional difficulty level matrix are initialized respectively. After initialization, each element value in the two-dimensional knowledge point mapping matrix and the two-dimensional difficulty level matrix is 0; S12. For the i-th question in the global question bank, if the i-th question involves the j-th knowledge point, the element value of the i-th row and the j-th column in the two-dimensional knowledge point mapping matrix is set to 1. If the difficulty level of the i-th question is a simple question, the element value of the i-th row and the first column in the two-dimensional difficulty level matrix is set to 1. If the difficulty level of the i-th question is a difficult question, the element value of the i-th row and the second column in the two-dimensional difficulty level matrix is set to 1. Wherein, i represents the index of the question, and j represents the index of the knowledge point; S13. After all the questions in the global question bank are traversed, the two-dimensional knowledge point mapping matrix and the two-dimensional difficulty level matrix are obtained.

3. The multi-modal online test question recommendation method based on end-cloud collaboration according to claim 1, characterized in that, In step S2, the classification result of the user's expression contains 6 kinds in total, which are anger, tension, confusion, concentration, confidence and joy.

4. The multi-modal online test question recommendation method based on end-cloud collaboration according to claim 1, characterized in that, In step S2, the expression recognition classifier adopts a 5-layer full connection layer.

5. The multi-modal online test question recommendation method based on end-cloud collaboration according to claim 1, characterized in that, In the training process of the cognitive diagnosis model in step S3, the user's answering question condition is taken as the real label. If the answer is correct, the label value is 1, and if the answer is wrong, the label value is 0. The cross-entropy loss between the probability of the user's answering question correctly and the real label is calculated. The parameters of the cognitive diagnosis model are updated based on minimizing the cross-entropy loss until a preset iteration number is reached. The cognitive diagnosis model converges, and the trained cognitive diagnosis model corresponding to the current set of questions is obtained.

6. The multi-modal online test question recommendation method based on end-cloud collaboration according to claim 1, characterized in that, In step S4, the device-side question selection strategy adopts the search efficient computerized adaptive testing (Search-Efficient Computerized Adaptive Testing).

7. The multi-modal online test question recommendation method based on end-cloud collaboration according to claim 2, characterized in that, In step S4, the specific process of constructing the cloud-side question selection strategy is as follows: S41. The device end uploads the user number of the answer recommendation question bank and the cognitive diagnosis model trained by the previous set of questions to the cloud together, and after the cloud receives it, the i-th question q i ∈Q and the examinee number who answered the i-th question are input into the cognitive diagnosis model received by the cloud to obtain the gradient change EMC of the cognitive diagnosis model, where 1≤i≤|Q|, |Q| represents the number of questions in the global question bank, and i represents the index of the question in the global question bank; S42. The change of knowledge point coverage of each question in the cloud computing global question bank, the i-th question q in the global question bank i The calculation method of the change of knowledge point coverage IWKC(q i ) of the i-th question q in the global question bank is: wherein J represents a knowledge point index set, w j is an additional importance weight of the jth knowledge point; IncCov(j, q i ) represents an incremental function, which is calculated in the following manner: wherein cnt(j, q i ) indicates whether the question related to the jth knowledge point contains the ith question q i in the global question bank, i.e., if yes, then the value of cnt(j, q i ) is 1, otherwise, the value of cnt(j, q i ) is 0. S43. The cloud end takes the gradient change EMC as a first keyword, takes the knowledge point coverage change IWKC of each question as a second keyword, sorts the global question bank Q according to the first and second keywords, selects the first Q questions in the sorted global question bank as a recommended question bank matching the current knowledge state, issues the recommended question bank to the user of the device end, and records the time of issuing the questions; wherein Q is a preset positive integer. k k a preset positive integer.​ 8. A multi-modal online test question recommendation system based on end-cloud collaboration, characterized in that, It includes: A data acquisition module for acquiring the knowledge points examined by the questions and the difficulty levels of the questions from the pre-constructed global question bank through the cloud, constructing a two-dimensional knowledge point mapping matrix of the mapping relationship between each question and each knowledge point examined by the question, constructing a two-dimensional difficulty level matrix of the mapping relationship between each question and the difficulty level of each question, and sending the two-dimensional knowledge point mapping matrix and the two-dimensional difficulty level matrix to the device side; The difficulty level of the question is a simple question or a difficult question. For the current question in the global question bank, the number of correct answers and the number of wrong answers of all users in the historical records are counted respectively. If the number of correct answers is greater than or equal to the number of wrong answers, the difficulty level of the current question is set to a simple question, otherwise, the difficulty level of the current question is set to a difficult question; The cognitive diagnosis module is configured to randomly select questions from a global question bank to construct a recommended question bank and send the recommended question bank to the device end before a user at the device end starts answering questions, for a current set of questions in the recommended question bank, take a photo of the user who is answering the current set of questions to obtain a face image corresponding to each user, input the collected face images into a trained expression recognition classifier model to classify expression information, and obtain a classification result of user expressions. When the user completes the current set of questions, obtain the user's answer record information, which is the user's answer results and the time spent on answering questions. The diagnosis result acquisition module is configured to input the user number of the answer recommended question bank, the user's answer record information, the classification result of the user's expressions, and the constructed two-dimensional knowledge point mapping matrix and two-dimensional difficulty level matrix into the cognitive diagnosis model trained by the previous set of questions to obtain a user knowledge radar chart diagnosis result and send the result to the cloud end. The cognitive diagnosis model is deployed at the device end, and when the user answers the current set of questions, the answered current set of questions is deleted from the recommended question bank, the user number of the answer recommended question bank, the user's answer record information, the classification result of the user's expressions, and the constructed two-dimensional knowledge point mapping matrix and two-dimensional difficulty level matrix, and the user knowledge radar chart diagnosis result are re-input into the cognitive diagnosis model for training. The adaptive question selection module is configured to determine whether there are remaining questions in the recommended question bank at the device end: if not, the cloud end automatically selects a recommended question bank matching the current knowledge state for the user from the global question bank according to a pre-constructed cloud end question selection strategy and the user knowledge radar chart diagnosis result; if so, the device end selects a set of questions for the user from the recommended question bank according to a pre-constructed device end question selection strategy. The question bank screening processing module is configured to traverse the delivery time of each question in the recommended question bank at the device end, determine whether the difference between the current time during traversal and the delivery time satisfies a preset set expiration time, if so, remove the corresponding question from the recommended question bank at the device end, and if not, keep the corresponding question in the recommended question bank at the device end.

9. A computer-readable storage medium, characterized in that, The storage medium has a computer program stored thereon, and when the computer program is executed by the processor, the method for recommending online test questions based on end-cloud collaboration according to any one of claims 1-7 is implemented.

10. A computer electronic device, comprising: The storage medium has a computer program stored thereon, and when the computer program is executed by the processor, the method for recommending online test questions based on end-cloud collaboration according to any one of claims 1-7 is implemented. The storage medium has a computer program stored thereon, and when the computer program is executed by the processor, the method for recommending online test questions based on end-cloud collaboration according to any one of claims 1-7 is implemented. ​

Images