Multi-modal personalized topic recommendation method and device, computer equipment and medium

Abstract

The invention relates to the technical field of big data, and discloses a multi-modal personalized topic recommendation method and device, computer equipment and a medium. The method comprises the following steps: calling a pre-trained question selector model; determining a target capability value of the target user according to the question recommendation request and a capability value table; the ability value table comprises grades, ages and current ability values of all historical users and is determined according to a target cognitive diagnosis model, and the model is obtained based on historical learning data training; the historical learning data is multi-modal data; determining related parameters of each question in the question bank; inputting the target ability value and the related parameters of each question into a question selector model to obtain a prediction score of each question, and determining the question corresponding to a preset number of prediction scores as a target question; and recommending the target question to the target user. Based on the technical scheme of the invention, personalized topic recommendation can be realized, and topics which are too simple or beyond the ability range are prevented from being recommended.

Description

Technical Field

[0001] The present invention relates to the field of big data technology, and in particular to a multimodal personalized topic recommendation method, device, computer equipment and medium. Background Art

[0002] With the rapid development of information technology, cognitive diagnosis is extremely important for evaluating users' knowledge and promoting personalized learning. In today's digital education environment, data is growing explosively. At the same time, users have an increasingly strong demand for personalized learning experience, and expect to receive accurate learning resources and topic recommendations based on their own learning progress in order to achieve efficient learning.

[0003] Traditional cognitive diagnosis models usually adopt fixed recommendation strategies and are unable to dynamically adjust the recommended content according to the user's real-time learning situation, resulting in poor personalized recommendation results.

[0004] Therefore, there is an urgent need for a topic recommendation method that can dynamically adjust the recommended content according to the user's real-time learning situation and improve the personalized recommendation effect. Summary of the invention

[0005] Therefore, the technical problem to be solved by the present invention is to overcome the problem that the fixed recommendation strategy usually adopted in the related art cannot dynamically adjust the recommended content according to the user's real-time learning situation, resulting in poor effect of personalized recommendation.

[0006] In order to solve the above technical problems, the present invention provides a multimodal personalized topic recommendation method, comprising:

[0007] When a topic recommendation request of a target user is obtained, a pre-trained topic selector model is called according to the topic recommendation request; the topic recommendation request includes the current grade and current age of the target user;

[0008] The target capability value of the target user is determined according to the topic recommendation request and the capability value table; wherein, when the target user is a new user, the target capability value of the target user is determined according to the current grade, current age and the capability value table of the target user; when the target user is a historical user, the target capability value of the target user is the current capability value of the target user in the capability value table; the capability value table includes the grade, age and current capability value of all historical users; the capability value table is determined according to a target cognitive diagnosis model; the target cognitive diagnosis model is obtained by training based on historical learning data; the historical learning data includes multimodal data corresponding to each historical user, and the multimodal data includes: user behavior records in picture format or text format, topic information, answer results and feedback information;

[0009] Determine the relevant parameters of each question in the question bank; the relevant parameters include: difficulty, discrimination, suspicion, knowledge point parameters, and question type parameters;

[0010] The target ability value of the target user and the relevant parameters of each question in the question bank are input into the question selector model to obtain the predicted score of each question in the question bank, and the questions corresponding to the first preset number of predicted scores of all the questions are determined as target questions; and the target questions are recommended to the target user.

[0011] In an optional implementation, when a topic recommendation request of a target user is obtained, before calling a pre-trained topic selector model according to the topic recommendation request, the method further includes:

[0012] Randomly divide the questions in the question bank into support sets and test sets, and initialize the ability value table;

[0013] Update the capability value table according to the historical users' answer results on the support set in the historical learning data and the target cognitive diagnosis model;

[0014] Inputting the updated capability value table and the relevant parameters of each question in the test set into the question selector model to obtain the predicted score of each question in the test set;

[0015] Determine a preset evaluation index according to the difference between the predicted score and the actual score of each question in the test set;

[0016] The parameters of the question selector model are adjusted according to the preset evaluation index.

[0017] In an optional implementation, determining a preset evaluation index according to a difference between a predicted score and an actual score of each question in the test set includes:

[0018] Determine the sum of squares of the difference values ​​between the predicted score and the actual score of each question in the test set, determine the average value of the sum of squares, determine the square root of the average value, and obtain the preset evaluation index.

[0019] In an optional implementation, adjusting the parameters of the question selector model according to the preset evaluation index includes:

[0020] Constructing a loss function according to the preset evaluation index;

[0021] The gradient of the loss function is determined according to a gradient descent algorithm, and the parameters of the question selector model are adjusted according to the gradient of the loss function.

[0022] In an optional implementation, when a topic recommendation request of a target user is obtained, before calling a pre-trained topic selector model according to the topic recommendation request, the method further includes:

[0023] Collect historical learning data from various data sources; the data sources include: user answering system, question management system, image storage system and text storage system; format the historical learning data and upload it to the target storage bucket for storage; pre-process the historical learning data uploaded to the target storage bucket; the pre-processing includes: data cleaning, data conversion and multi-modal feature extraction;

[0024] A data set is constructed based on the preprocessed historical learning data; an initial cognitive diagnosis model is trained based on the data set to obtain a target cognitive diagnosis model; the target cognitive diagnosis model is constructed based on an item response theory model and a neurocognitive diagnosis framework;

[0025] The multimodal data corresponding to each historical user is input into the target cognitive diagnosis model to obtain the current ability value of each historical user; and the ability value table is constructed according to the grade, age and current ability value of all historical users.

[0026] In an optional implementation, the training of the initial cognitive diagnosis model based on the data set to obtain the target cognitive diagnosis model includes:

[0027] The data set is randomly divided into a training set, a validation set and a test set; the training set is used to train an initial cognitive diagnosis model, and a cognitive diagnosis model is obtained after the training is completed; the validation set is used to adjust the parameters of the cognitive diagnosis model; and the test set is used to evaluate the performance of the cognitive diagnosis model after the parameters are adjusted;

[0028] When the performance of the cognitive diagnosis model after adjusting the parameters meets the preset conditions, the cognitive diagnosis model after adjusting the parameters is determined as the target cognitive diagnosis model.

[0029] In an optional implementation manner, after recommending the target topic to the target user, the method further includes:

[0030] The multimodal data corresponding to the target user is updated according to the answer result of the target user on the target question.

[0031] In a second aspect, the present invention provides a multimodal personalized topic recommendation device, comprising:

[0032] A first processing module is used to call a pre-trained topic selector model according to a topic recommendation request when a topic recommendation request of a target user is obtained; the topic recommendation request includes the current grade and current age of the target user;

[0033] The second processing module is used to determine the target capability value of the target user according to the topic recommendation request and the capability value table; wherein, when the target user is a new user, the target capability value of the target user is determined according to the current grade, current age and the capability value table of the target user; when the target user is a historical user, the target capability value of the target user is the current capability value of the target user in the capability value table; the capability value table includes the grade, age and current capability value of all historical users; the capability value table is determined according to a target cognitive diagnosis model; the target cognitive diagnosis model is obtained by training based on historical learning data; the historical learning data includes multimodal data corresponding to each historical user, and the multimodal data includes: user behavior records in picture format or text format, topic information, answer results and feedback information;

[0034] The third processing module is used to determine the relevant parameters of each question in the question bank; the relevant parameters include: difficulty, discrimination, suspicion, knowledge point parameters, and question type parameters;

[0035] The fourth processing module is used to input the target ability value of the target user and the relevant parameters of each question in the question bank into the question selector model, obtain the predicted score of each question in the question bank, determine the questions corresponding to the first preset number of predicted scores among the predicted scores of all questions as target questions; and recommend the target questions to the target user.

[0036] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the multimodal personalized question recommendation method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0037] In a fourth aspect, the present invention provides a computer-readable storage medium, a single computer-readable storage medium storing computer instructions, the computer instructions being used to enable a computer to execute the multimodal personalized question recommendation method of the first aspect or any corresponding embodiment thereof.

[0038] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions, wherein the computer instructions are used to enable a computer to execute the multimodal personalized topic recommendation method of the first aspect or any corresponding embodiment thereof.

[0039] The technical solution provided by the present invention has the following technical effects:

[0040] The topic recommendation request in the technical solution of the embodiment of the present invention includes two key information: the current grade and current age of the target user. Users of different grades and ages often have different knowledge reserves, cognitive levels, and learning needs. Recommending topics based on the current grade and age of the target user combined with other information can make the recommended topics more suitable for the user's actual learning stage and improve the accuracy of the recommendation.

[0041] For new users, their target ability value can be determined based on their grade, age and ability value table, ensuring that even in the absence of historical learning data or past test data, they can be initially matched with questions of appropriate difficulty level.

[0042] For historical users, their target capability values ​​directly use the current capability values ​​recorded in the capability value table, which is obtained by analyzing a large amount of historical learning data based on the target cognitive diagnosis model. These historical learning data cover a wealth of multimodal data, such as user behavior records, question information, image information, and text information. By mining and analyzing such comprehensive multimodal historical learning data, the determined user's capability value can more accurately reflect the user's true knowledge mastery and ability level, thereby achieving more accurate question recommendations.

[0043] When determining the relevant parameters of each question in the question bank, multiple dimensions are involved, including difficulty, discrimination, suspicion, knowledge point parameters, and question type parameters. The difficulty parameter can intuitively reflect the challenge of the question to users of different ability levels and reflect the requirements of the question on the user's ability level. The discrimination is used to reflect the ability of the question to distinguish students of different ability levels. The suspicion is used to reflect the probability of students answering the question correctly under guessing conditions. The knowledge point parameter clarifies the knowledge scope of the question and is used to reflect the coverage of different knowledge points. The question type parameter reflects the type of question, for example, multiple choice, fill-in-the-blank, short answer, or other forms of presentation.

[0044] By integrating the relevant parameters of the questions, we can comprehensively characterize the questions, so that the subsequent question selector model can more scientifically select suitable questions based on the user's ability value when making recommendations.

[0045] Using a pre-trained question selector model, which has been trained with a large amount of data and has learned the complex relationship between the user's ability value and the relevant parameters of the question. After inputting the target ability value of the target user and the relevant parameters of each question in the question bank, it can quickly and efficiently calculate the predicted score of each question, thereby accurately screening out the preset number of target questions that are most suitable for the user.

[0046] This not only saves recommendation time, but also avoids the subjective and one-sided problems that may arise in manual screening, improves the efficiency and quality of recommendations, and realizes truly personalized question recommendations. By accurately recommending questions that meet the target users' ability level, learning stage, and knowledge points, users will not feel frustrated when doing questions because the questions are too difficult, nor will they be unable to get effective training because the questions are too simple. This helps to maintain learning interest, enhance learning confidence, and thus improve learning results. For example, a junior high school student with medium math ability receives questions of appropriate difficulty that cover the knowledge points he is learning. He can also fill in the gaps during practice, consolidate the knowledge he has learned, and gradually improve his grades.

[0047] From the perspective of overall educational resource utilization, the technical solution of the present invention can make the questions in the question bank more reasonably allocated and used. Users of different ability levels can get the question resources that match them, avoiding the situation where high-quality question resources are overused by some users while other users cannot use them effectively, making the overall utilization efficiency of teaching resources higher. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the related technologies, the drawings required for use in the specific embodiments or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0049] Figure 1 is a flowchart of a multimodal personalized topic recommendation method according to an embodiment of the present invention;

[0050] Figure 2 is a flowchart of another multimodal personalized topic recommendation method according to an embodiment of the present invention;

[0051] Figure 3 is a schematic diagram of a recommended process of an embodiment of the present invention;

[0052] Figure 4 1 is a schematic diagram of the overall process of the multimodal personalized topic recommendation system according to an embodiment of the present invention;

[0053] Figure 5 is a structural diagram of a multimodal personalized topic recommendation device according to an embodiment of the present invention;

[0054] Figure 6 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0056] Traditional cognitive diagnosis methods usually rely on local computing resources and are difficult to process large-scale data and update in real time. Although traditional cognitive diagnosis models such as DINA and G-DINA can evaluate the user's knowledge, they have limitations in processing multimodal data (such as pictures and text) and realizing dynamic personalized question recommendations.

[0057] Limited data processing capabilities: Traditional methods have difficulty processing large-scale multimodal data, resulting in inaccurate cognitive diagnosis results. For example, traditional cognitive diagnosis models usually only process text data and cannot effectively process multimodal data containing images, resulting in reduced accuracy of evaluation results.

[0058] Lack of real-time performance: Traditional methods cannot update users’ cognitive diagnosis results and personalized recommendations in real time. For example, traditional cognitive diagnosis models usually require batch processing of data and cannot respond to changes in users’ learning behavior in real time, resulting in insufficient timeliness of personalized recommendations.

[0059] Personalized recommendation is not effective: Traditional methods have limited effectiveness in personalized topic recommendations and cannot meet users’ personalized learning needs. For example, traditional cognitive diagnosis models usually use fixed recommendation strategies and cannot dynamically adjust the recommended content based on users’ real-time learning conditions, resulting in poor personalized recommendation results.

[0060] Traditional standards or common solutions: There is currently no unified standard for multimodal data processing and dynamic personalized topic recommendation. Traditional educational technology standards mainly focus on data storage and transmission, but do not involve the specific implementation of multimodal data processing and personalized recommendation.

[0061] Traditional standards: IEEE 1484.1 standard: This standard mainly defines the interoperability of educational technology systems, including data formats, data exchange protocols, etc., but does not involve the specific implementation of multimodal data processing and personalized recommendations. IMS Global Learning Consortium: This organization has developed a series of educational technology standards, such as IMS Learning Design, IMS Question & Test Interoperability, etc., but these standards mainly focus on the interoperability of educational content and do not involve multimodal data processing and personalized recommendations.

[0062] Traditional common solutions: Traditional cognitive diagnosis models: such as DINA, G-DINA, etc. These models mainly process text data and cannot effectively process multimodal data.

[0063] Adaptive learning systems: such as Knewton, DreamBox, etc. Although these systems can provide a certain degree of personalized recommendations, they have limitations in processing multimodal data and real-time updates.

[0064] Big data analysis platforms: such as Hadoop and Spark. These platforms are mainly used for large-scale data processing, but are not optimized for multimodal data processing and personalized recommendations in the education field.

[0065] The embodiments of the present invention provide a multimodal personalized topic recommendation method, apparatus, computer equipment and medium to solve the above problems.

[0066] Cognitive diagnostic model: is a psychometric method that not only estimates the user's knowledge level in a specific area, but also diagnoses the user's strengths and weaknesses in more detailed cognitive attributes.

[0067] Elastic Compute Service (ECS): A basic cloud computing service that allows users to rent virtual servers in the cloud to run applications. It has the characteristics of high availability, elastic scalability and security.

[0068] Relational Database Service (RDS): provides relational database services that support multiple database engines and help users easily manage structured data.

[0069] Object Storage Service (OSS): The object storage service provided is suitable for storing large-scale unstructured data, such as pictures, videos, and log files.

[0070] Elastic MapReduce (EMR): Provides big data processing services for running complex data analysis and processing tasks.

[0071] AI platform: The AI ​​platform provides pre-trained AI models and machine learning tools, supporting a variety of artificial intelligence services such as natural language processing and image recognition. Users can use these tools to quickly build and deploy AI applications, or train custom machine learning models to meet specific business needs.

[0072] According to an embodiment of the present invention, an embodiment of a multimodal personalized topic recommendation method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer device such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0073] Figure 1 It is a flowchart of a multimodal personalized topic recommendation method according to an embodiment of the present invention.

[0074] like Figure 1 As shown, an embodiment of the present invention provides a multimodal personalized topic recommendation method, and the multimodal personalized topic recommendation method includes:

[0075] S101: When a topic recommendation request from a target user is obtained, a pre-trained topic selector model is called according to the topic recommendation request.

[0076] In this embodiment, the topic recommendation request includes the current grade and current age of the target user.

[0077] In this embodiment, the multimodal personalized topic recommendation method can be applied to students or other users who need to learn. A multimodal personalized topic recommendation system / platform can be designed based on the technical solution of the multimodal personalized topic recommendation method of the present invention.

[0078] S102: Determine the target capability value of the target user according to the topic recommendation request and the capability value table.

[0079] In this embodiment, when the target user is a new user, the target ability value of the target user is determined based on the current grade, current age and ability value table of the target user. When the target user is a historical user, the target ability value of the target user is the current ability value of the target user in the ability value table. The ability value table includes the grade, age and current ability value of all historical users. The ability value table is determined based on the target cognitive diagnosis model. The target cognitive diagnosis model is trained based on historical learning data. The historical learning data includes multimodal data corresponding to each historical user, and the multimodal data includes: user behavior records in picture format or text format, question information, answer results and feedback information.

[0080] User input: Users need to select grade and age options in the system so that the system can provide personalized question selection services based on the user's age group.

[0081] Data acquisition: The system retrieves parameters related to the average ability of the current grade and current age from the AI ​​platform as the initial ability value of the cold start user.

[0082] Initial capability value of cold-start users (new users): For users who have just joined the system, the average capability-related parameters of the current grade and / or current age can be used as the initial capability value of the cold-start user. For example, when the target user is a new user, the target capability value of the target user (new user) is the average of the current capability values ​​of all users with the same current grade and current age as the target user in the capability value table.

[0083] Current capability value of non-cold start users (historical users): For users who are already in the system, the user capability value estimated using historical answer records and the target cognitive diagnosis model. For example, when the target user is a historical user, the target capability value of the historical user F is the current capability value of the historical user F in the capability value table.

[0084] S103: Determine relevant parameters of each question in the question bank.

[0085] In this embodiment, the relevant parameters include: difficulty, discrimination, suspicion, knowledge point parameters, and question type parameters. Difficulty: The difficulty parameter of the question reflects the requirement of the question on the student's ability. Discrimination: The discrimination parameter of the question reflects the ability of the question to distinguish students of different ability levels. Suspicion: The suspicion parameter of the question reflects the probability that the student will answer the question correctly under guessing conditions. Knowledge point parameters: The knowledge point parameters involved in the question reflect the coverage of different knowledge points by the question. Question type parameters: The question type parameters of the question reflect the type of question, such as multiple-choice questions, fill-in-the-blank questions, short-answer questions, etc. As an example, it can be estimated by data mining techniques such as matrix factorization (MF).

[0086] In this embodiment, determining the relevant parameters of each question in the question bank specifically includes: loading the parameters of the trained target cognitive diagnosis model, and determining the relevant parameters of each question in the question bank through the target cognitive diagnosis model.

[0087] S104: Input the target ability value of the target user and the relevant parameters of each question in the question bank into the question selector model to obtain the predicted score of each question in the question bank, determine the questions corresponding to the first preset number of predicted scores among the predicted scores of all questions as target questions, and recommend the target questions to the target user.

[0088] In this embodiment, the preset number can be designed and modified according to actual needs. After the target topic is recommended to the target user, the target user can select a topic to answer. The multimodal data corresponding to the target user can be updated according to the target user's answer result on the target topic, and the multimodal data corresponding to the target user's answer result on the target topic can be added to the historical learning data of the target user to achieve real-time update of the learning data. It is also possible to redetermine the ability value of the target user based on the updated multimodal data corresponding to the target user and the target cognitive diagnosis model, thereby updating the ability value table based on the redetermined ability value of the target user, and achieving real-time update of the user's ability value, so as to dynamically adjust the recommended content according to the user's real-time learning situation, improve the personalized recommendation effect, and improve the accuracy of the recommendation.

[0089] In this embodiment, the question selector model includes a convolutional neural network model and a machine learning prediction model.

[0090] Convolutional neural network model: Use the convolutional neural network model of the AI ​​platform to process complex nonlinear relationships and extract high-order features of the target ability values ​​of the target users and the relevant parameters of each question in the question bank.

[0091] Machine learning prediction model: Use the latest machine learning prediction models, such as XGBoost, Light GBM, etc., take the target ability value of the target user and the relevant parameters of each question in the question bank as input, predict the target user's score on each question, and obtain the predicted score of the target user for each question in the question bank.

[0092] After obtaining the predicted score of each question in the question bank for the target user, it is necessary to select the target extraction and recommend it to the target user. The question selection strategy is the core link of personalized question recommendation. Its goal is to select the question that can predict the maximum score under the current ability state of the student as the most suitable question to be recommended to the student. The implementation of the current question selection model involves the principle of maximum information and maximum uncertainty to ensure that each selected question can provide the most diagnostic information and minimize the uncertainty of the student's ability value. Therefore, in this embodiment, the questions corresponding to the first preset number of predicted scores among the predicted scores of all questions are selected as target questions and recommended to the target user.

[0093] In an optional implementation, when a topic recommendation request from a target user is obtained, before calling a pre-trained topic selector model according to the topic recommendation request, the multimodal personalized topic recommendation method further includes: training and optimizing the topic selector model to obtain a trained topic selector model.

[0094] Training and optimizing the topic selector model to obtain a trained topic selector model specifically includes:

[0095] Sa1: Randomly divide the questions in the question bank into support sets and test sets, and initialize the ability value table.

[0096] In this embodiment, the specific method of initializing the capability value table refers to the scheme for constructing the capability value table described below, which will not be described in detail here. The capability value of the new user in the capability value table obtained by initializing the capability value table is the average value of the current capability values ​​of all users in the capability value table with the same current grade and current age as the new user. The capability value of the historical user is the capability value estimated using the historical learning data and the target cognitive diagnosis model.

[0097] In this embodiment, a group of questions can be selected from the question bank as a support set for updating the student's ability value, and another group of questions can be selected from the question bank as a test set for verifying the accuracy of the updated ability value.

[0098] Sa2: Update the capability value table based on the historical users’ answer results on the support set in the historical learning data and the target cognitive diagnosis model.

[0099] In this embodiment, the target cognitive diagnosis model trained by the AI ​​platform is loaded, and the student's ability value is updated using the target cognitive diagnosis model according to the student's answer results on the support set.

[0100] Sa3: Input the updated capability value table and the relevant parameters of each question in the test set into the question selector model to obtain the predicted score of each question in the test set.

[0101] In this embodiment, the target cognitive diagnosis model trained by the AI ​​platform is used to diagnose the relevant parameters of each question in the test set.

[0102] Sa4: Determine the preset evaluation indicator based on the difference between the predicted score and the actual score of each question in the test set.

[0103] In this embodiment, Sa4 determines the preset evaluation index based on the difference value between the predicted score and the actual score of each question in the test set, specifically including: determining the sum of the squares of the difference values ​​between the predicted score and the actual score of each question in the test set, determining the average of the sum of squares, determining the square root of the average, and obtaining the preset evaluation index.

[0104] In this embodiment, the number of questions in the test set is obtained. The difference between the actual score of each question and the predicted score output by the question selector model is calculated. The sum of the squares of the difference values ​​is calculated. The average of the sum of squares is calculated. The square root of the average is taken to obtain the preset evaluation indicator RMSE.

[0105] Sa5: Adjust the parameters of the question selector model according to the preset evaluation indicators.

[0106] In this embodiment, the parameters of the topic selector model are adjusted according to preset evaluation indicators until the parameters of the topic selector model reach a fitting state to obtain a trained topic selector model. The topic selector model that reaches a fitting state can be actually applied to a multimodal personalized topic recommendation system constructed based on the multimodal personalized topic recommendation method. Adjusting the parameters of the topic selector model according to the preset evaluation indicators in Sa5 specifically includes: constructing a loss function according to the preset evaluation indicators. Determine the gradient of the loss function according to the gradient descent algorithm, and adjust the parameters of the topic selector model according to the gradient of the loss function. Conventional techniques in the art can be used to adjust the parameters of the topic selector model according to the preset evaluation indicators.

[0107] In the present invention, the multimodal personalized topic recommendation system includes: topic selection module. The topic selection module is a key component in the multimodal personalized topic recommendation system. Its main function is to dynamically select the most suitable topic for the student's current level according to the student's knowledge proficiency, that is, the ability value, so as to achieve personalized learning and accurate diagnosis. The present invention will elaborate on how to combine the powerful functions of the AI ​​platform to build an efficient and intelligent topic selection module.

[0108] The topic selection module includes: 1) user initialization, 2) topic parameter diagnosis, 3) user ability assessment and 4) topic selection.

[0109] 1) User initialization includes: User input: The user needs to select the grade / age option in the multimodal personalized question recommendation system so that the multimodal personalized question recommendation system can provide personalized question selection services based on the age group of the students. Data acquisition: The multimodal personalized question recommendation system extracts the average ability-related parameters of the current grade / age from the AI ​​platform as the initial ability value of the cold start user.

[0110] 2) Question parameter diagnosis includes: Model loading: The multimodal personalized question recommendation system loads the parameters of the target cognitive diagnosis model trained by the AI ​​platform. Question parameter diagnosis: Use the target cognitive diagnosis model to diagnose the relevant parameters of each question in the question bank.

[0111] 3) User capability assessment includes: cold start users and non-cold start users.

[0112] 4) Topic selection includes: model selection (convolutional neural network model and machine learning prediction model) and topic selection strategy.

[0113] In the present invention, the multimodal personalized topic recommendation system also includes: a truth value construction module.

[0114] The truth value construction module is a key component in the multimodal personalized question recommendation system. Its main function is to measure the effectiveness of the questions selected by the question selector model. Since the question selector is independent of the model, it is not feasible to directly use the model to determine whether a question is suitable for students. Therefore, a truth value construction module is proposed to obtain effective truth values ​​to measure the quality of the selected questions.

[0115] The core principle of the truth value construction module is to divide the questions in the question bank into a support set and a test set, use the students' answers on the support set to update the students' ability values, and then verify the accuracy of the updated ability values ​​on the test set. The specific steps are as follows:

[0116] Support set selection: Select a set of questions from the question bank as the support set to update the student's ability value.

[0117] Ability update: Based on the students’ answers on the support set, the target cognitive diagnosis model trained by the AI ​​platform is used to update the students’ ability values.

[0118] Test set validation: Use the updated capability table to make predictions on the test set and calculate the difference between the predicted score and the actual score, using RMSE as the measurement indicator.

[0119] RMSE evaluation: The smaller the RMSE value, the higher the score of the currently selected question, that is, the more suitable the selected question is for students.

[0120] The true value construction module includes: 1) Initializing the ability value. 2) Selecting the support set and the test set: Select a set of questions from the question bank as the support set to update the student's ability value. Select another set of questions from the question bank as the test set to verify the accuracy of the updated ability value. 3) Model loading: Load the target cognitive diagnosis model trained by the AI ​​platform. 4) Ability update: According to the student's answer results on the support set, use the target cognitive diagnosis model to update the student's ability value. Calculate RMSE. 5) RMSE calculation: Use the updated ability value to make predictions on the test set, and calculate the difference between the predicted score and the actual score, using RMSE as a measurement indicator. Steps for RMSE calculation: Get the number of questions in the test set. Calculate the difference between the true response score of each question and the model predicted probability, calculate the sum of the squares of the difference values, find the average of the sum of squares, take the square root of the average, and get RMSE.

[0121] In the present invention, the multimodal personalized topic recommendation system also includes: a topic selector training and optimization module.

[0122] The question selector training and optimization module is the core component of the multimodal personalized question recommendation system. Its main function is to update the model parameters of the question selector module by combining the predicted score of the output of the question selection module with the true value construction module for each student for all question banks, until the parameters of the question selector model reach a fitting state.

[0123] The topic selector training and optimization module includes: 1) data preparation, 2) topic selection module, 3) truth value construction module, 4) model parameter update, 5) model tuning and 6) model export and storage.

[0124] 1) Data preparation includes: Student ability parameters: For students newly added to the system, the initial ability value uses the parameters related to the average ability of the current grade / age. For students already in the system, the student ability values ​​are estimated using historical answer records and cognitive diagnosis models. Question parameters: Use the target cognitive diagnosis model trained by the AI ​​platform to diagnose the relevant parameters of the questions in the question bank.

[0125] 2) Question selection module for model loading: Load the question selection model trained by the AI ​​platform, and question selection: Select the question that best suits the student's current level based on the student's ability value in the ability value table and the relevant parameters of each question in the question bank, and predict the student's predicted score on the selected question.

[0126] 3) The truth value construction module selects the support set: a set of questions is selected from the question bank as the support set to update the student's ability value. The target cognitive diagnosis model can be used to update the student's ability value based on the student's answer results on the support set. Test set verification: Use the updated ability value to make predictions on the test set, and calculate the difference between the predicted score and the actual score, using RMSE as the measurement indicator.

[0127] 4) Model parameter update includes: Loss function definition: Define the loss function to measure the difference between the predicted score and the RMSE value. Gradient descent optimization: Use the gradient descent algorithm to update the model parameters of the question selector module according to the gradient of the loss function to reduce the difference. Iterative optimization: Repeat the above steps until the model parameters reach a fitting state, that is, the difference between the predicted score and the RMSE value is minimized.

[0128] 5) Model tuning includes: Hyperparameter tuning: Find the optimal hyperparameter combination, such as learning rate, regularization coefficient, etc. Model structure tuning: You can try different model structures, such as adding hidden layers, adjusting the number of neurons, etc., to improve the expressiveness of the model. Model integration: You can use model integration methods, such as bagging, boosting, etc., to improve the generalization ability and stability of the model.

[0129] 6) Model export and storage include: Model export: export the trained model parameters to ensure the reusability of the model. You can use the model export function provided by the AI ​​platform to easily export the model into different formats. Model storage: use the API interface of the RDS service to store the exported model parameters in the RDS service to ensure the persistence and scalability of the model. You can use the backup and recovery functions provided by the RDS service to ensure the security of the model data.

[0130] Figure 2 FIG. 1 is a flow chart of another multi-modal personalized topic recommendation method according to an embodiment of the present invention. Figure 2 As shown, another multimodal personalized topic recommendation method is provided in an embodiment of the present invention, and the multimodal personalized topic recommendation method includes:

[0131] S201: Train cognitive diagnostic models and construct competency value tables.

[0132] The preliminary preparation stage includes: 1) uploading historical learning data, and 2) data preprocessing.

[0133] The scheme for training cognitive diagnosis models includes: multimodal feature selection, feature conversion and combination, model selection and combination, model training and optimization, model evaluation, and model storage (storing the target cognitive diagnosis model).

[0134] 1) The historical learning data upload part includes: data collection, data formatting and data upload. Specifically: Data collection: collect historical learning data from various data sources. Data sources include: user answering system, question management system, image storage system and text storage system. Historical learning data includes multimodal data corresponding to each historical user, and multimodal data includes but is not limited to user behavior records, question information, answering results and feedback information in image format or text format. User behavior records: including user answering records, answering time, access records, etc., in text format. Question information: including text description, picture description and knowledge point description of the question. The answer result is the user's answer, and the feedback information is the feedback information of the user answering system on the user's answer. Data in image format includes: including picture answers submitted by users, picture descriptions in questions, pictures contained in the feedback information of the user answering system on the user's answer, etc. Data in text format includes: including text answers submitted by users, text descriptions in questions (question stem, options, prompt information), feedback information of the user answering system on the user's answer, etc. Data formatting and data upload: format the historical learning data and upload it to the target storage bucket for storage.

[0135] In this embodiment, the collected historical learning data can be formatted to ensure the consistency and integrity of the data. For example, data in different formats can be converted into a unified format, such as JSON, CSV, etc. The formatted historical learning data can be uploaded to the OSS service using the API interface of the OSS service to ensure the persistence and scalability of the data. For example, the PUT interface of the OSS service can be used to upload the data to a specified storage bucket.

[0136] 2) Data preprocessing is a key step to ensure data quality and model performance. In educational scenarios, data preprocessing is particularly important because high-quality data is the basis for building accurate cognitive diagnosis models. Based on this, after data upload, the historical learning data uploaded to the target storage bucket is preprocessed. Preprocessing includes: data cleaning, data conversion, and multimodal feature extraction.

[0137] Data cleaning: Clean the uploaded historical learning data to remove noise and redundant data to ensure data quality. For example, remove duplicate data, missing data, abnormal data, etc.

[0138] Removing duplicate data specifically includes: Duplicate records: duplicate operation records of users in the system, such as repeated submission of answers, repeated page visits, etc. Processing method: Use deduplication algorithms, such as timestamp deduplication, unique identifier deduplication, etc., to remove duplicate records. Timestamp deduplication: Determine whether the record is duplicated based on the timestamp, and remove duplicate records with the same timestamp. For example, for duplicate answer submission records with the same timestamp, retain the earliest record and remove subsequent duplicate records. Unique identifier deduplication: Determine whether the record is duplicated based on the unique identifier (such as user ID, question ID), and remove duplicate records with the same identifier. For example, for duplicate answer submission records with the same user ID and question ID, retain the earliest record and remove subsequent duplicate records.

[0139] Removing missing data specifically includes: Missing records: missing operation records of users in the system, such as not submitting answers, not visiting pages, etc. Processing method: Use interpolation algorithms, such as linear interpolation, KNN interpolation, etc., to fill in missing records. Linear interpolation: Use linear interpolation algorithms to fill in missing records based on adjacent records in the time series. For example, for missing answer time records in the time series, use the answer time of adjacent records for linear interpolation to fill in missing records. KNN interpolation: Use KNN interpolation algorithms to fill in missing records based on adjacent records in spatial data. For example, for missing answer time records in spatial data, use KNN interpolation to fill in missing records using the answer time of adjacent records.

[0140] Removing abnormal data specifically includes: Abnormal records: Records of abnormal operations by users in the system, such as extremely long answering times, abnormal access paths, etc. Handling method: Use abnormal detection algorithms, such as Z-score, IQR, etc., to identify and remove abnormal records. Z-score: Calculate the Z-score value of the record and identify and remove abnormal records that exceed the standard deviation. For example, for answering time records, calculate the Z-score value of each record and identify and remove abnormal records that exceed the standard deviation. IQR: Calculate the interquartile range (IQR) of the record and identify and remove abnormal records that exceed the interquartile range. For example, for answering time records, calculate the interquartile range (IQR) of each record and identify and remove abnormal records that exceed the interquartile range.

[0141] Removing noise data specifically includes: Noise data: Irrelevant information entered by users during the answering process, such as garbled characters, meaningless characters, etc. Handling method: Use text cleaning algorithms, such as regular expressions, stop word filtering, etc., to remove noise data. Regular expressions: Use regular expressions to match and remove garbled and meaningless characters in the text. For example, use regular expressions to match and remove garbled and meaningless characters in the text, such as [^a-zA-Z0-9\s]. Stop word filtering: Use a stop word list to filter and remove stop words and irrelevant information in the text. For example, use a stop word list to filter and remove stop words and irrelevant information in the text, such as "de", "le", "zai", etc.

[0142] Removing data with format errors specifically includes: Format errors: Format errors entered by users during the answering process, such as inconsistent formats, chaotic formats, etc. Handling method: Use format conversion algorithms, such as text standardization, format unification, etc., to correct format errors. Text standardization: Convert the text to a unified standard format, such as unifying case, unifying punctuation marks, etc. For example, convert all letters in the text to lowercase and unify punctuation marks to English punctuation marks. Format unification: Convert the text to a unified format, such as unifying the date format, unifying the time format, etc. For example, unify the date format in the text to YYYY-MM-DD and the time format to HH:MM:SS.

[0143] Data conversion: Convert the cleaned data, such as data format conversion, data encoding conversion, etc., to ensure data consistency. For example, convert text data into vector representation, convert image data into matrix representation, etc. Text data conversion specifically includes: Text vectorization: Convert text data into vector representation, such as TF-IDF, Word2Vec, etc. TF-IDF: Use the TF-IDF algorithm to convert text data into vector representation and extract keyword features in the text. Word2Vec: Use the Word2Vec algorithm to convert text data into vector representation and extract semantic features in the text. Image data conversion specifically includes: Image matrixization: Convert image data into matrix representation, such as grayscale, normalization, etc. Grayscale: Convert color images into grayscale images and extract the brightness features of the image. Normalization: Normalize the image data to ensure the consistency of the image data.

[0144] Multimodal feature extraction: Extract features from multimodal data, such as the user's answer time, answer accuracy, question difficulty, etc. The specific steps include:

[0145] Text feature extraction specifically includes:

[0146] Use natural language processing technology to extract features from text data, such as word frequency, part of speech, sentiment intensity, etc. For example, use the TF-IDF algorithm to extract text features, use the part-of-speech tagging algorithm to extract part-of-speech features, and use the sentiment analysis algorithm to extract sentiment intensity features. Word frequency features: Use the TF-IDF algorithm to extract word frequency features in the text to reflect the keyword information in the text. Part-of-speech features: Use the part-of-speech tagging algorithm to extract part-of-speech features in the text to reflect the grammatical information in the text. Sentiment features: Use the sentiment analysis algorithm to extract sentiment features in the text to reflect the sentiment information in the text.

[0147] Image feature extraction specifically includes:

[0148] Use image processing technology to extract features from image data, such as color features, texture features, shape features, etc. For example, use the color histogram algorithm to extract color features, use the Gabor filter algorithm to extract texture features, and use the edge detection algorithm to extract shape features. Color features: Use the color histogram algorithm to extract color features in the image, reflecting the color distribution information in the image. Texture features: Use the Gabor filter algorithm to extract texture features in the image, reflecting the texture information in the image. Shape features: Use the edge detection algorithm to extract shape features in the image, reflecting the shape information in the image.

[0149] Behavioral feature extraction specifically includes:

[0150] Extract features from user behavior data, such as answering time, answering order, answering interaction, etc. For example, calculate the statistical features of the user's answering time, answering order, and answering interaction, such as the average, variance, maximum, minimum, etc. Answering time features: Calculate the statistical features of the user's answering time, such as the average, variance, maximum, minimum, etc., to reflect the user's answering speed information. Answering order features: Calculate the statistical features of the user's answering order, such as sequence changes, sequence stability, etc., to reflect the user's answering strategy information. Answering interaction features: Calculate the statistical features of the user's answering interaction, such as the number of mouse clicks, the number of keyboard inputs, etc., to reflect the user's answering behavior information.

[0151] 3) Model training:

[0152] A data set is constructed based on the preprocessed historical learning data. An initial cognitive diagnosis model is trained based on the data set to obtain a target cognitive diagnosis model. The target cognitive diagnosis model is constructed based on the item response theory model and the neurocognitive diagnosis framework.

[0153] Before multimodal feature extraction, multimodal feature selection is required to determine the multimodal features that need to be extracted during feature extraction. Feature selection is a key step in building a high-quality cognitive diagnosis model. Features related to cognitive diagnosis will be selected from multimodal data, and necessary extraction and conversion will be performed.

[0154] Text feature selection includes: keyword selection, topic selection and sentiment selection, specifically:

[0155] Keyword selection: Select keywords related to cognitive diagnosis based on TF-IDF value, word frequency and other indicators. Topic selection: Select topics related to cognitive diagnosis based on indicators such as topic distribution and topic relevance. Emotion selection: Select emotional features related to cognitive diagnosis based on indicators such as emotion intensity and emotion polarity. Image feature selection includes: color feature selection, texture feature selection and shape feature selection. Specifically: Color feature selection: Select color features related to cognitive diagnosis based on indicators such as color histogram and color distribution. Texture feature selection: Select texture features related to cognitive diagnosis based on indicators such as Gabor filter response and texture pattern. Shape feature selection: Select shape features related to cognitive diagnosis based on indicators such as edge detection results and shape descriptors. Behavioral feature selection includes: answer time feature selection, answer order feature selection and answer interaction feature selection. Specifically: Answer time feature selection: Select answer time features related to cognitive diagnosis based on statistical characteristics of answer time. Answer order feature selection: Select answer order features related to cognitive diagnosis based on statistical characteristics of answer order. Question answering interaction feature selection: Based on the statistical characteristics of question answering interaction, question answering interaction features related to cognitive diagnosis are selected.

[0156] Feature transformation and combination include: standardization and normalization, feature combination, specifically:

[0157] Standardization and normalization: Convert features of different dimensions to the same dimension to avoid the impact of large differences in feature values ​​on model training.

[0158] Feature combination: Combining different types of features to form comprehensive features. For example, combining text features with behavioral features can better reflect the user's learning habits and knowledge mastery.

[0159] Model selection and combination: The item response theory model (IRT model) and the neurocognitive diagnosis framework (NeuralCD framework) will be combined to select the most suitable model for cognitive diagnosis, and explore how to integrate multimodal data into the model. IRT model: Basic principle: The IRT model assumes that there is a certain functional relationship between the user's ability (θ) and the difficulty (b) and discrimination (a) of the question. Common IRT models include 1PL, 2PL and 3PL models. Multimodal extension: Multimodal features can be used as inputs to the IRT model, such as integrating question text features, image features, etc. into the estimation of question difficulty and discrimination, so as to build a more complex IRT model. NeuralCD framework: Basic principle: The NeuralCD framework is a cognitive diagnosis model based on neural networks, which can handle complex nonlinear relationships and provide interpretable diagnostic results. Multimodal fusion: Multimodal features can be directly input into the NeuralCD framework, and the powerful feature extraction and fusion capabilities of neural networks can be used to build a more comprehensive cognitive diagnosis model. Model combination: NeuralCD-IRT: The IRT model is combined with the neural network, and the powerful fitting ability of the neural network is used to improve the accuracy and interpretability of the model. Specifically, the parameter estimation results of the IRT model can be used as the input of the neural network, or the output of the neural network can be used as the basis for the parameter estimation of the IRT model. Multimodal NeuralCD-GNN: Graph Neural Network (GNN) is used to process the association between knowledge points, and multimodal features are integrated into GNN to build a more complex cognitive diagnosis model.

[0160] The machine learning service of the AI ​​platform can be used to train and optimize cognitive diagnostic models. The initial cognitive diagnostic model is trained based on the data set to obtain the target cognitive diagnostic model, specifically including: randomly dividing the data set into a training set, a validation set, and a test set. The training set is used to train the initial cognitive diagnostic model, and the cognitive diagnostic model is obtained after the training is completed. The validation set is used to adjust the parameters of the cognitive diagnostic model. The test set is used to evaluate the performance of the cognitive diagnostic model after the parameters are adjusted. When the performance of the cognitive diagnostic model after the parameters are adjusted meets the preset conditions, the cognitive diagnostic model after the parameters are adjusted is determined as the target cognitive diagnostic model.

[0161] In this embodiment, the training set can be used to train the initial cognitive diagnosis model (machine learning model), and the cognitive diagnosis model can be obtained after the training is completed. In this embodiment, the distributed training function provided by the Tian AI platform can be used to accelerate the model training process. Grid search, random search and other methods can be used to find the optimal hyperparameter combination, such as learning rate, regularization coefficient, etc. for model hyperparameter tuning. Different model structures can be tried, such as adding hidden layers, adjusting the number of neurons, etc., to improve the expressiveness of the model for model structure tuning. Model integration methods such as Bagging, Boosting, etc. can be used for model integration to improve the generalization ability and stability of the model. The performance of the model can be evaluated using methods such as cross-validation, and the optimal model can be selected as the target cognitive diagnosis model.

[0162] Use the cross-validation method to evaluate the performance of the model, such as accuracy, recall, F1 value, etc. You can use the model evaluation function provided by the AI ​​platform to automatically generate an evaluation report. Model selection: Select the optimal model based on the results of cross-validation to ensure the generalization ability of the model. You can use the model management function provided by the AI ​​platform to conveniently manage and deploy models. Model storage: Store the trained model parameters in the RDS service to ensure the persistence and scalability of the model. Model export: Export the parameters of the trained target cognitive diagnosis model to ensure the reusability of the target cognitive diagnosis model. You can use the model export function provided by the AI ​​platform to easily export the target cognitive diagnosis model to different formats. Model storage: Use the API interface of the RDS service to store the exported parameters of the target cognitive diagnosis model in the RDS service to ensure the persistence and scalability of the target cognitive diagnosis model.

[0163] You can use the backup and recovery functions provided by the RDS service to ensure the security of the target cognitive diagnosis model data.

[0164] Specific plan for constructing the capability value table:

[0165] The multimodal data corresponding to each historical user is input into the target cognitive diagnosis model to obtain the current ability value of each historical user.

[0166] Construct a competency value table based on the grade, age, and current competency values ​​of all historical users.

[0167] The user identification, for example, user name, user ID, user grade, age and current ability value are stored in a corresponding relationship in the ability value table.

[0168] S202: When the target user's question recommendation request is obtained, the pre-trained question selector model is called according to the question recommendation request. For details, please refer to the relevant description of S101, which will not be repeated here. S203: Determine the target ability value of the target user based on the question recommendation request and the ability value table. For details, please refer to the relevant description of S102, which will not be repeated here. S204: Determine the relevant parameters of each question in the question bank. For details, please refer to the relevant description of S103, which will not be repeated here. S205: Input the target ability value of the target user and the relevant parameters of each question in the question bank into the question selector model to obtain the predicted question score of each question in the question bank, and determine the questions corresponding to the first preset number of predicted question scores among the predicted question scores of all questions as target questions. Recommend the target question to the target user. For details, please refer to the relevant description of S104, which will not be repeated here.

[0169] A multimodal personalized question recommendation system / platform can be designed based on the technical solution of the multimodal personalized question recommendation method of the present invention, and the multimodal personalized question recommendation system is connected to multiple data sources, such as a user answering system, a question management system, a picture storage system, and a text storage system.

[0170] The system architecture of the multimodal personalized topic recommendation system / platform may include: Data layer: storing historical learning data of users, user behavior records, topic information, image information, text information, knowledge point data, etc. Model layer: including trained topic selector model and other auxiliary models (such as target cognitive diagnosis model). Application layer: providing user interface, receiving student requests (for example, topic recommendation request), calling topic selector model for personalized topic recommendation, and presenting the recommendation results to students. The specific process of recommendation is as follows: Figure 3 As shown, including:

[0171] Student login system: When the target user is a new user, you need to register and fill in user information, such as grade and age. You can enter the user name as the user ID, or it can be randomly generated by the system. After successful registration, log in. When the target user is a historical user, load the historical learning data of the target user.

[0172] Students initiate topic recommendation requests: The multimodal personalized topic recommendation system can set a "start learning" button, or users can select or enter knowledge points and practice questions on the knowledge points. When students click the "start learning" button or select a knowledge point to practice, a topic recommendation request is generated. The system interface can display multiple knowledge points for users to directly select.

[0173] The system calls the question selector model: the question selector model is called according to the question recommendation request, and the predicted score of each question in the question bank is output according to the question selector model.

[0174] Generate a recommendation list: Based on the prediction results (the prediction score of each question in the question bank), select the top 10 questions with the highest prediction scores that match the knowledge points from the question bank and generate a recommendation list.

[0175] Present recommendation results: Show the recommendation list to students, and they can choose questions to practice.

[0176] Update learning data: After students complete the questions, the system records the answer results (questions and actual scores) and updates the students' learning data.

[0177] The multimodal personalized question recommendation system has set up a dynamic adjustment mechanism: Real-time update of learning data: The system will record students' answers in real time and update their learning data to make personalized recommendations more accurately. Dynamic update of model parameters: The system will dynamically update the question selector model parameters based on students' learning situation and feedback to improve the accuracy and effectiveness of recommendations.

[0178] Provide personalized learning suggestions: The system can provide personalized learning suggestions based on students' learning situation, such as recommending learning materials, adjusting learning plans, etc.

[0179] The network element devices involved in the technical solution of the present invention include: ECS: used for dynamically allocating computing resources. RDS: used for storing structured data. OSS: used for storing unstructured data. EMR: used for running big data processing tasks. AI platform: used for providing pre-trained AI models and machine learning tools.

[0180] The main process steps of the present invention are as follows Figure 4 As shown, it mainly includes:

[0181] Multimodal data input and preprocessing: The system receives and preprocesses multimodal data from multiple sources, including student behavior records, question information, pictures and text, etc. Main invention points: The key innovation of the technical solution of the present invention lies in the fusion and preprocessing of multimodal data, especially data cleaning, conversion and feature extraction methods, which provide a high-quality data foundation for subsequent model training.

[0182] Cognitive diagnosis model training: Use the preprocessed data to train the cognitive diagnosis model on the AI ​​platform to evaluate students' knowledge proficiency. Main invention point: The key innovation of the technical solution of the present invention is to combine the IR T model and the NeuralCD framework, and integrate multimodal data to build a more complex and accurate cognitive diagnosis model.

[0183] Question selection module: Dynamically select the questions that best suit the students' current level based on their knowledge proficiency, and predict the students' scores on the selected questions.

[0184] Approximate truth value construction module: By dividing the questions in the question bank into a support set and a test set, the student's performance on the support set is used to update the student's ability estimate, and then the accuracy of the updated ability estimate is verified on the test set, and the RMSE value is calculated to measure the quality of the selected questions. Main invention point: The key innovation of the technical solution of the present invention is to propose an approximate truth value construction method, which provides an effective truth value measurement standard for the question selector.

[0185] Topic selector training and optimization module: Combine the output prediction score of the topic selection module with the RMSE value of the approximate truth value construction module to update the topic selector model parameters until the model parameters reach a fitting state. Main invention point: The key innovation of the technical solution of the present invention is to propose a model parameter update method based on the RMSE value, continuously optimize the topic selector model, and improve the accuracy and effectiveness of the recommendation.

[0186] Dynamic personalized topic recommendation: Based on the trained topic selector model, dynamic personalized topic recommendation service is provided to students. Main invention point: The key innovation of the technical solution of the present invention is to update learning data in real time, dynamically adjust the recommendation strategy and provide personalized learning suggestions, so as to build an efficient and intelligent dynamic personalized topic recommendation system.

[0187] The present invention aims to solve the following technical problems:

[0188] Traditional question recommendations lack personalization: Traditional question recommendation systems usually use a fixed question bank or random question selection, and cannot provide personalized learning content based on individual differences among students. Insufficient accuracy of cognitive diagnosis models: Traditional cognitive diagnosis models mostly rely on single-modal data, and the model structure is relatively simple, making it difficult to accurately assess students' knowledge proficiency. Question selection strategies lack scientific basis: Traditional question selection strategies are mostly based on empirical rules, lack theoretical basis and data support, and it is difficult to ensure that the recommended questions can effectively improve students' learning outcomes.

[0189] The present invention can achieve the following technical effects:

[0190] Provide accurate personalized topic recommendations: The present invention can dynamically select topics that best suit students' current levels based on factors such as students' knowledge proficiency, learning goals, and learning styles, and achieve accurate personalized topic recommendations. Build a high-precision cognitive diagnosis model: The present invention combines the IRT model and the NeuroCD framework, and integrates multimodal data to build a more complex and accurate cognitive diagnosis model, which can more accurately evaluate students' knowledge proficiency. Improve the accuracy and efficiency of the topic selector: The present invention trains the topic selector based on the traditional machine learning model on the AI ​​platform, uses advanced algorithms and massive data to improve the accuracy and efficiency of the topic selector, and ensures that the recommended topics can effectively improve students' learning effects. Improve students' learning efficiency and learning interest: The present invention can help students find the most suitable learning path for themselves, avoid ineffective learning, and improve learning efficiency. At the same time, it can recommend more attractive topics based on students' interests and preferences to stimulate students' learning interest. Improve learning efficiency: Personalized recommendations can help students find the most suitable learning path for themselves, avoid ineffective learning, and improve learning efficiency. Stimulate learning interest: Personalized recommendations can recommend more attractive topics based on students' interests and preferences to stimulate students' learning interest. Promote personalized development: Personalized recommendations can help students develop personalized learning plans based on their own characteristics and needs, and promote personalized development. Multimodal data fusion and preprocessing: Improve data quality and enrich data dimensions through data cleaning, conversion and feature extraction, and provide a high-quality data foundation for subsequent model training. Cognitive diagnosis model combining IRT model and NeuralCD framework: Use the powerful fitting and feature extraction capabilities of neural networks to improve model accuracy, and integrate multimodal data to build a more comprehensive cognitive diagnosis model. Train question selectors based on traditional machine learning models on AI platforms: Use advanced algorithms and massive data to improve the accuracy and efficiency of question selectors, and achieve large-scale parallel computing to accelerate model training and optimization processes. Question selector model parameter update method based on RMSE value: Combine the predicted score with the RMSE value to continuously optimize the model parameters, improve the model accuracy, and achieve dynamic adjustment to optimize the recommendation effect based on student feedback. Dynamic personalized question recommendation system: Provide personalized learning experience based on students' knowledge proficiency, learning goals, learning style and other factors, improve learning efficiency and interest, and promote personalized development.

[0191] It should be noted that the contents not described in detail in the specification of the present invention belong to the common knowledge of those skilled in the art.

[0192] In this embodiment, a multimodal personalized topic recommendation device is also provided. A single device is used to implement the above-mentioned embodiment and optional implementation methods. The descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0193] Figure 5 4 is a schematic diagram of the structure of a multimodal personalized topic recommendation device according to an embodiment of the present invention.

[0194] The present invention provides a multi-modal personalized topic recommendation device, such as Figure 5 As shown, the multimodal personalized topic recommendation device includes:

[0195] The first processing module 11 is used to call the pre-trained topic selector model according to the topic recommendation request when obtaining the topic recommendation request of the target user. The topic recommendation request includes the current grade and current age of the target user.

[0196] The second processing module 12 is used to determine the target ability value of the target user based on the question recommendation request and the ability value table. Wherein, when the target user is a new user, the target ability value of the target user is determined based on the current grade, current age and ability value table of the target user. When the target user is a historical user, the target ability value of the target user is the current ability value of the target user in the ability value table. The ability value table includes the grade, age and current ability value of all historical users. The ability value table is determined based on the target cognitive diagnosis model. The target cognitive diagnosis model is trained based on historical learning data. The historical learning data includes multimodal data corresponding to each historical user, and the multimodal data includes: user behavior records in picture format or text format, question information, answer results and feedback information.

[0197] The third processing module 13 is used to determine the relevant parameters of each question in the question bank, including difficulty, discrimination, suspicion, knowledge point parameters, and question type parameters.

[0198] The fourth processing module 14 is used to input the target ability value of the target user and the relevant parameters of each question in the question bank into the question selector model, obtain the predicted score of each question in the question bank, determine the questions corresponding to the first preset number of predicted scores among the predicted scores of all questions as target questions, and recommend the target questions to the target user.

[0199] In an optional embodiment, the multimodal personalized question recommendation device also includes: a question selector model determination module, which is used to randomly divide the questions in the question bank into a support set and a test set, and initialize the capability value table. The capability value table is updated according to the historical users' answer results on the support set in the historical learning data and the target cognitive diagnosis model. The updated capability value table and the relevant parameters of each question in the test set are input into the question selector model to obtain the predicted score of each question in the test set. The preset evaluation index is determined according to the difference between the predicted score and the actual score of each question in the test set. The parameters of the question selector model are adjusted according to the preset evaluation index.

[0200] In an optional embodiment, the question selector model determination module includes: a preset evaluation index determination unit, used to determine the sum of squares of the difference values ​​between the predicted score and the actual score of each question in the test set, determine the average of the sum of squares, determine the square root of the average, and obtain a preset evaluation index.

[0201] In an optional embodiment, the topic selector model determination module includes: a parameter adjustment unit, which is used to construct a loss function according to a preset evaluation index, determine the gradient of the loss function according to a gradient descent algorithm, and adjust the parameters of the topic selector model according to the gradient of the loss function.

[0202] In an optional implementation, the multimodal personalized topic recommendation device further includes: a preparation module, a cognitive diagnosis model determination module and a capability value table determination module.

[0203] The preparation module is used to collect historical learning data from various data sources. Data sources include: user answering system, question management system, image storage system and text storage system. Format the historical learning data and upload it to the target storage bucket for storage. Preprocess the historical learning data uploaded to the target storage bucket. Preprocessing includes: data cleaning, data conversion and multimodal feature extraction.

[0204] The cognitive diagnosis model determination module is used to construct a data set based on the preprocessed historical learning data. The initial cognitive diagnosis model is trained based on the data set to obtain the target cognitive diagnosis model. The target cognitive diagnosis model is constructed based on the item response theory model and the neurocognitive diagnosis framework.

[0205] The capability value table determination module is used to input the multimodal data corresponding to each historical user into the target cognitive diagnosis model to obtain the current capability value of each historical user. The capability value table is constructed according to the grade, age and current capability value of all historical users.

[0206] In an optional embodiment, the cognitive diagnosis model determination module is specifically used to randomly divide the data set into a training set, a validation set and a test set. The training set is used to train the initial cognitive diagnosis model, and the cognitive diagnosis model is obtained after the training is completed. The validation set is used to adjust the parameters of the cognitive diagnosis model. The test set is used to evaluate the performance of the cognitive diagnosis model after the parameters are adjusted. When the performance of the cognitive diagnosis model after the parameters are adjusted meets the preset conditions, the cognitive diagnosis model after the parameters are adjusted is determined as the target cognitive diagnosis model.

[0207] In an optional implementation, the multimodal personalized question recommendation device further includes: a historical learning data updating module, which is used to update the multimodal data corresponding to the target user according to the answer results of the target user on the target question.

[0208] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments, and will not be repeated here. The multimodal personalized topic recommendation device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0209] The embodiment of the present invention also provides a computer device having the above Figure 5 The multimodal personalized topic recommendation device shown. Figure 6 , Figure 6 Schematic diagram of the hardware structure of the computer device according to the embodiment of the present invention. Figure 6 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In an optional embodiment, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor device). Figure 6 A processor 10 is taken as an example. The computer device further comprises a communication interface 30, which is used for the computer device to communicate with other devices or communication networks.

[0210] An embodiment of the present invention also provides a computer-readable storage medium, and the above-mentioned method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented by downloading through a network and originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware.

[0211] Part of the present invention may be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer.

[0212] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A multimodal personalized topic recommendation method, characterized in that: include: When a topic recommendation request of a target user is obtained, a pre-trained topic selector model is called according to the topic recommendation request; the topic recommendation request includes the current grade and current age of the target user; The target capability value of the target user is determined according to the topic recommendation request and the capability value table; wherein, when the target user is a new user, the target capability value of the target user is determined according to the current grade, current age and the capability value table of the target user; when the target user is a historical user, the target capability value of the target user is the current capability value of the target user in the capability value table; the capability value table includes the grade, age and current capability value of all historical users; the capability value table is determined according to a target cognitive diagnosis model; the target cognitive diagnosis model is obtained by training based on historical learning data; the historical learning data includes multimodal data corresponding to each historical user, and the multimodal data includes: user behavior records in picture format or text format, topic information, answer results and feedback information; Determine the relevant parameters of each question in the question bank; the relevant parameters include: difficulty, discrimination, suspicion, knowledge point parameters, and question type parameters; The target ability value of the target user and the relevant parameters of each question in the question bank are input into the question selector model to obtain the predicted score of each question in the question bank, and the questions corresponding to the first preset number of predicted scores of all the questions are determined as target questions; and the target questions are recommended to the target user.

2. The method according to claim 1, characterized in that When a topic recommendation request of a target user is obtained, before calling a pre-trained topic selector model according to the topic recommendation request, the method further includes: Randomly divide the questions in the question bank into support sets and test sets, and initialize the ability value table; Update the capability value table according to the historical users' answer results on the support set in the historical learning data and the target cognitive diagnosis model; Inputting the updated capability value table and the relevant parameters of each question in the test set into the question selector model to obtain the predicted score of each question in the test set; Determine a preset evaluation index according to the difference between the predicted score and the actual score of each question in the test set; The parameters of the question selector model are adjusted according to the preset evaluation index.

3. The method according to claim 2, characterized in that The step of determining a preset evaluation index according to the difference between the predicted score and the actual score of each question in the test set includes: Determine the sum of squares of the difference values ​​between the predicted score and the actual score of each question in the test set, determine the average value of the sum of squares, determine the square root of the average value, and obtain the preset evaluation index.

4. The method according to claim 2, characterized in that: The adjusting the parameters of the question selector model according to the preset evaluation index includes: Constructing a loss function according to the preset evaluation index; The gradient of the loss function is determined according to a gradient descent algorithm, and the parameters of the question selector model are adjusted according to the gradient of the loss function.

5. The method according to claim 1, characterized in that When a topic recommendation request of a target user is obtained, before calling a pre-trained topic selector model according to the topic recommendation request, the method further includes: Collect historical learning data from various data sources; the data sources include: user answering system, question management system, image storage system and text storage system; format the historical learning data and upload it to the target storage bucket for storage; pre-process the historical learning data uploaded to the target storage bucket; the pre-processing includes: data cleaning, data conversion and multi-modal feature extraction; A data set is constructed based on the preprocessed historical learning data; an initial cognitive diagnosis model is trained based on the data set to obtain a target cognitive diagnosis model; the target cognitive diagnosis model is constructed based on an item response theory model and a neurocognitive diagnosis framework; The multimodal data corresponding to each historical user is input into the target cognitive diagnosis model to obtain the current ability value of each historical user; and the ability value table is constructed according to the grade, age and current ability value of all historical users.

6. The method according to claim 5, characterized in that The step of training the initial cognitive diagnosis model based on the data set to obtain the target cognitive diagnosis model includes: The data set is randomly divided into a training set, a validation set and a test set; the training set is used to train an initial cognitive diagnosis model, and a cognitive diagnosis model is obtained after the training is completed; the validation set is used to adjust the parameters of the cognitive diagnosis model; and the test set is used to evaluate the performance of the cognitive diagnosis model after the parameters are adjusted; When the performance of the cognitive diagnosis model after adjusting the parameters meets the preset conditions, the cognitive diagnosis model after adjusting the parameters is determined as the target cognitive diagnosis model.

7. The method according to claim 1, characterized in that After recommending the target topic to the target user, the method further includes: The multimodal data corresponding to the target user is updated according to the answer result of the target user on the target question.

8. A multimodal personalized topic recommendation device, characterized in that: include: A first processing module is used to call a pre-trained topic selector model according to a topic recommendation request when a topic recommendation request of a target user is obtained; the topic recommendation request includes the current grade and current age of the target user; A second processing module, configured to determine a target capability value of the target user according to the topic recommendation request and the capability value table; Wherein, when the target user is a new user, the target capability value of the target user is determined according to the current grade, current age and capability value table of the target user; when the target user is a historical user, the target capability value of the target user is the current capability value of the target user in the capability value table; the capability value table includes the grade, age and current capability value of all historical users; the capability value table is determined according to the target cognitive diagnosis model; the target cognitive diagnosis model is obtained by training based on historical learning data; the historical learning data includes multimodal data corresponding to each historical user, and the multimodal data includes: user behavior records in picture format or text format, question information, answer results and feedback information; The third processing module is used to determine the relevant parameters of each question in the question bank; the relevant parameters include: difficulty, discrimination, suspicion, knowledge point parameters, and question type parameters; The fourth processing module is used to input the target ability value of the target user and the relevant parameters of each question in the question bank into the question selector model, obtain the predicted score of each question in the question bank, determine the questions corresponding to the first preset number of predicted scores among the predicted scores of all questions as target questions; and recommend the target questions to the target user.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the multimodal personalized question recommendation method according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the multimodal personalized topic recommendation method according to any one of claims 1 to 7.

Images