Intelligent management system for continuous education courseware resources based on big data

Abstract

The invention discloses an intelligent management system for continuous education courseware resources based on big data, and relates to the technical field of education resource management. The intelligent management system for the continuous education courseware resources based on the big data comprises a data acquisition module, a data preprocessing module, a data storage module, a data management module, a personalized recommendation module and a learning path planning module, through the efficient data acquisition and preprocessing module, courseware resource data can be captured and cleaned accurately in real time, a distributed storage system is flexible in configuration, and the safety and reliability of the data are ensured. And the data management module is introduced, so that courseware resources are clearly classified and conveniently searched, and the user experience is improved. And the personalized recommendation module generates accurate resource recommendation according to user behaviors, so that the pertinence and efficiency of learning are enhanced. In addition, the learning path planning module can dynamically adjust the learning path, it is ensured that the user learns orderly according to the course dependency relationship, and the learning effect is effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of educational resource management, and in particular to an intelligent management system for continuing education courseware resources based on big data. Background Art

[0002] Courseware resources refer to multimedia teaching resources used to assist teaching, training, demonstration and other activities, including text, pictures, audio, video and other forms. According to the usage scenario and teaching content, courseware resources can be divided into various types such as course courseware, experimental courseware, question bank, case bank, material bank and so on. The management of courseware resources is conducive to improving teaching efficiency. Through effective management of courseware resources, teachers can quickly find the required teaching resources, thereby improving lesson preparation and teaching efficiency. It also helps to realize the sharing of teaching resources and promote communication and cooperation between different teachers.

[0003] Publication No. CN116562781A discloses a continuing education courseware resource management method, device, platform and system. Different courseware resource providers can provide courseware to be stored in the warehouse, which solves the technical problem of single courseware resources; the received courseware to be stored in the warehouse is subject to triple review, including review of content format, content quality, pricing and study hours. Only after passing the triple review can the courseware to be stored be put on the shelf for students to study, ensuring the quality of the courseware and solving the technical problem of difficult control of course quality and substandard course quality in the prior art.

[0004] As shown in the above technology, the entry and review management of continuing education courseware resources is very important for controlling course quality, but this alone is still not enough. How to review and manage continuing education courseware resources and apply them to actual learning and education, combine management and use, and rationally manage and utilize courseware resources to better assist learning and education is an urgent problem to be solved. Summary of the invention

[0005] In view of the deficiencies of the prior art, the present invention provides an intelligent management system for continuing education courseware resources based on big data, which solves the problem of how to combine management and use, and rationally manage and utilize courseware resources to better assist learning and education.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: a big data-based continuing education courseware resource intelligent management system, including the following modules:

[0007] The data collection module uses web crawler technology to capture courseware resource data, including videos, documents, and pictures, based on a preset URL list and crawling frequency;

[0008] The data preprocessing module is used to clean the collected data, process missing values ​​and outliers, and use standardized formulas to normalize the data to provide a data basis for subsequent processing;

[0009] A data storage module is used to store the processed data into a distributed storage system and configure the storage system according to the number of storage nodes, data shard size and replication factor parameters;

[0010] The data management module is used to classify and label courseware resources and build indexes for users to find and obtain the required resources;

[0011] Personalized recommendation module, which is used to collect user behavior data, use recommendation algorithms to generate personalized resource recommendation lists, and provide accurate recommendations based on user interests and learning behaviors;

[0012] The learning path planning module is used to plan the learning path based on the user's learning progress and course dependencies using a path planning algorithm, provide relevant course and resource recommendations, and dynamically adjust the learning path.

[0013] Preferably, in the data storage module, the node load is monitored in real time by an algorithm, and the data shard size is dynamically adjusted according to the load of the storage node;

[0014] The replication factor parameter adopts a replication strategy based on data access frequency. For frequently accessed data, the replication factor is increased to improve data access speed; and an intelligent replication algorithm is introduced to dynamically adjust the replication strategy according to data access mode and node status;

[0015] Use big data analysis technology to predict data growth trends and plan storage node expansion strategies in advance. When additional storage nodes are needed, prioritize placing data shards near nodes with high data access volume to reduce data migration and access delays.

[0016] Preferably, the step of adjusting the size of the data slices is:

[0017] a1 monitors the load of each node in real time;

[0018] a2 According to the average load And threshold T to determine whether the shard size needs to be adjusted:

[0019] a2.1 Calculate the average load of node i: Where N is the total number of storage nodes, L i represents the current load of node i;

[0020] a2.2 Determine whether the shard size needs to be adjusted:

[0021] If L i >T and S i / N i If it is greater than the average shard size, reduce the shard size;

[0022] If L i < T and S i / N i < If it is less than the average shard size, increase the shard size;

[0023] a2.3 Adjust the shard size: S new，i = S i ±ΔS, where S i represents the total size of the original data shards on node i, and S new，i represents the total size of the new data shards on node i, and ΔS is the shard size adjustment amount, which is a fixed value or a value dynamically calculated based on the load;

[0024] a3 Adjust the shard size as needed and update the number of shards:

[0025] a4 Redistribute the data shards to each node.

[0026] Preferably, the steps of the replication strategy are:

[0027] b1 Monitor the access frequency of the data. Let the access frequency of data d be A f (d);

[0028] b2 Determine whether the data is frequently accessed based on the access frequency threshold. The method for determining whether the data is frequently accessed is: if A f (d)>T freq , then data d is frequently accessed data, where T freq represents the access frequency threshold;

[0029] b3 For frequently accessed data, increase its replication factor. The way to adjust the replication factor is: R new，f (d)=R f (d)+ΔR f where R f (d) represents the original replication factor number of data d, and R new，f (d) represents the new replication factor number of data d, and ΔR f represents the replication factor adjustment amount;

[0030] b4 Redistribute the data replicas to each node.

[0031] Preferably, the steps of the storage node expansion strategy are:

[0032] c1 Monitor the data access volume of each node;

[0033] c2 calculates the data migration priority based on the data access volume and distance factor. The formula for calculating the data migration priority is: Among them, D a (p) is the amount of data access on node p, P(p, q) is the probability or priority of data migration from node p to node q, and the distance factor is calculated based on the physical distance between nodes and network delay factors. k D a (k) represents the amount of data access to all nodes k D a (k) performing a summation;

[0034] c3 predicts the future data growth trend. The formula for predicting the data growth trend is: D future (t) = G(t), where G(t) represents the data growth trend prediction function, which is used to predict the data growth amount in the future time period t;

[0035] c4 determines the number of new nodes based on the prediction results. The calculation formula for the number of new nodes is: Among them, D future (t) represents the predicted data growth in the future time period t, ∑ n S n Indicates the total storage capacity of all current storage nodes. Indicates rounding up;

[0036] c5 migrates data shards to the newly added nodes according to the data migration priority.

[0037] Preferably, the work content of the personalized recommendation module specifically includes:

[0038] d1 Data collection and preprocessing: Collect historical user behavior data, including course clicks, viewing time, ratings, favorites, and sharing behavior data; preprocess the data to remove noise data, including abnormal clicks and invalid views; use the user-course matrix to represent user behavior data, where rows represent users, columns represent courses, and matrix elements represent the intensity of a user's behavior on a course;

[0039] d2 Feature extraction: extract features from user behavior data, such as user preference features, user activity features, and calculate user similarity;

[0040] D3 recommendation algorithm selection and implementation: Select recommendation algorithms based on collaborative filtering, including user-based collaborative filtering and item-based collaborative filtering:

[0041] d3.1 User-based collaborative filtering: Based on user similarity, find other users who are similar to the target user and generate a recommendation list based on the preferences of these users;

[0042] d3.2 Project-based collaborative filtering: Based on the similarity between courses, find other courses similar to the target course and generate a recommendation list based on the popularity of these courses.

[0043] Preferably, the user similarity is calculated as follows: given two vectors u and v, where u and v both have Y dimensions, that is, u = (ru1, ru2, ..., ru Y ) and v = (rv1, rv2, ..., rv Y ), the formula of cosine similarity sim(u, v) is: Among them, y and rv y are the y-th subsets in sets u and v respectively.

[0044] Preferably, in the personalized recommendation module, considering that user interests change over time, a time decay factor is introduced to assign different weights to historical behavior data, with recent behaviors being assigned a larger weight and long-term behaviors being assigned a smaller weight; an exponential decay function f(t)=e -λt , where t represents the time interval and λ is the attenuation coefficient.

[0045] Preferably, the working content of the learning path planning module specifically includes:

[0046] Use path planning algorithms to plan learning paths based on user learning progress and course dependencies;

[0047] Provide relevant course and resource recommendations to help users complete learning tasks efficiently, and dynamically adjust the learning path according to the user's learning goals and progress.

[0048] Preferably, the steps of learning path planning are specifically as follows:

[0049] e1 Course dependency construction: Analyze the dependency between courses, and assume that the course dependency graph is a directed graph G = (V, E), where V is a node set representing courses, and E is an edge set representing the dependency between courses; the edge weight w (a, b) represents the dependency strength from node a of the course to node b of the course or the priority of the learning order;

[0050] e2 User learning progress tracking: Record the user's learning progress. Let the user's learning progress be a vector Z = (Z1, Z2, ..., Z g ), where Z j represents the user's learning progress for the jth course; for courses that have not yet started, Z j =0; for completed courses, Z j =1; for the course being studied, 0 <Z j <1;

[0051] e3 Learning path planning: The goal is to find a path from the user's current learning state to the user's learning goal, that is, from the course node corresponding to the non-zero element in vector Z to the path to complete all required courses, specifically:

[0052] Let Z(a) represent the set of paths to node a, and C(a) represent the set of subsequent nodes of node a;

[0053] For each node a, calculate the cost of the optimal path to reach node a, represented by f(a): f(a) = min b∈C(a) {f(b)+w(b,a)};

[0054] Initial conditions: For the starting node s, f(s) = 0; for other nodes, f(a) = ∞ at the beginning, indicating unreachable;

[0055] Termination condition: When the f(a) values ​​of all nodes are calculated, or a path from the starting point to the end point is found, the algorithm terminates;

[0056] e4 personalized learning path optimization:

[0057] Introducing personalized learning paths: Planning personalized learning paths for users based on their learning styles and interest preferences; Introducing user preference vector O = (O1, O2, ..., O L ), where O c represents the user's preference score for the cth course;

[0058] When calculating f(a), consider user preferences and modify the recursive formula to: f(a) = min b∈C(a) {f(b)+w(b,a)-α·r a}, where λ is the preference adjustment coefficient;

[0059] Dynamically adjust the learning path: According to the user's learning feedback, dynamically adjust the edge weight w(b, a) or the user preference vector r to optimize the learning path.

[0060] The present invention provides a big data-based intelligent management system for continuing education courseware resources. Compared with the prior art, it has the following beneficial effects:

[0061] 1. This big data-based continuing education courseware resource intelligent management system can capture and clean courseware resource data in real time and accurately through efficient data collection and preprocessing modules, providing rich and high-quality learning materials for continuing education. Its distributed storage system has flexible configuration, ensuring the security and reliability of data. The introduction of the data management module makes the courseware resources clearly classified and easy to find, which improves the user experience. The personalized recommendation module generates accurate resource recommendations based on user behavior, enhancing the pertinence and efficiency of learning. In addition, the learning path planning module can dynamically adjust the learning path to ensure that users learn in order according to the course dependencies, effectively improving the learning effect. In summary, the system significantly optimizes the management and application of continuing education courseware resources and promotes the process of educational informatization.

[0062] 2. The intelligent management system for continuing education courseware resources based on big data realizes the optimal allocation of storage resources by real-time monitoring of node load and dynamically adjusting the size of data shards, effectively reducing the read and write pressure of high-load nodes and improving the overall storage performance. This function ensures the high availability of data and the stability of the system; the introduction of replication strategy based on data access frequency and intelligent replication algorithm makes the allocation of data copies more reasonable. For frequently accessed data, the data access speed is significantly improved by increasing the replication factor, thereby optimizing the user experience. At the same time, the intelligent replication algorithm can dynamically adjust the replication strategy according to the data access mode and node status, further balancing data reliability and system performance; the system also uses big data analysis technology to predict data growth trends and plan storage node expansion strategies in advance accordingly. This function not only avoids storage bottlenecks caused by data surges, but also reduces data migration and access delays by preferentially placing data shards near nodes with large data access volume, further improving system efficiency.

[0063] 3. This big data-based continuing education courseware resource intelligent management system collects and preprocesses user historical behavior data, extracts multidimensional features, and weights historical behavior data with a time decay factor. This module can more accurately reflect the user's current interest preferences. At the same time, a recommendation algorithm based on collaborative filtering is used to make recommendations based on user similarity and course similarity, which not only improves the diversity of recommendations, but also enhances the personalization of recommendations. These improvements jointly improve the user experience, make the recommendation results more in line with user expectations, and enhance the interactivity and attractiveness of continuing education.

[0064] 4. This big data-based continuing education courseware resource intelligent management system can provide users with the optimal path from the current state to the learning goal by building a course dependency graph and tracking the user's learning progress. At the same time, the module introduces personalized learning path optimization, adjusts the path planning according to the user's learning style and interest preferences, and improves the pertinence and efficiency of learning. In addition, the module can also dynamically adjust the path based on user learning feedback to ensure that the learning path always meets the actual needs of the user. These improvements have jointly improved the user's learning experience, enhanced the flexibility and effectiveness of continuing education, made the learning process more in line with personal needs, and improved the quality and satisfaction of learning outcomes. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 is a system module block diagram of the present invention;

[0066] Figure 2 It is a schematic diagram of the overall process of the present invention;

[0067] Figure 3 It is a schematic diagram of a flow chart of a second embodiment of the present invention;

[0068] Figure 4 It is a schematic diagram of a process of a third embodiment of the present invention;

[0069] Figure 5 It is a schematic diagram of a fourth embodiment of the present invention. DETAILED DESCRIPTION

[0070] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0071] The present invention provides the following four technical solutions:

[0072] Figure 1-Figure 2 The first embodiment is shown: an intelligent management system for continuing education courseware resources based on big data, including the following modules:

[0073] The data collection module uses web crawler technology to capture courseware resource data, including videos, documents, and pictures, based on a preset URL list and crawling frequency;

[0074] The data preprocessing module is used to clean the collected data, process missing values ​​and outliers, and use standardized formulas to normalize the data to provide a data basis for subsequent processing;

[0075] A data storage module is used to store the processed data into a distributed storage system and configure the storage system according to the number of storage nodes, data shard size and replication factor parameters;

[0076] The data management module is used to classify and label courseware resources and build indexes for users to find and obtain the required resources;

[0077] Personalized recommendation module, which is used to collect user behavior data, use recommendation algorithms to generate personalized resource recommendation lists, and provide accurate recommendations based on user interests and learning behaviors;

[0078] The learning path planning module is used to plan the learning path based on the user's learning progress and course dependencies using a path planning algorithm, provide relevant course and resource recommendations, and dynamically adjust the learning path.

[0079] Through efficient data acquisition and preprocessing modules, the system can capture and clean courseware resource data in real time and accurately, providing rich and high-quality learning materials for continuing education. Its distributed storage system has flexible configuration, ensuring the security and reliability of data. The introduction of the data management module makes the courseware resources clearly classified and easy to find, which improves the user experience. The personalized recommendation module generates accurate resource recommendations based on user behavior, enhancing the pertinence and efficiency of learning. In addition, the learning path planning module can dynamically adjust the learning path to ensure that users learn in an orderly manner according to the course dependencies, effectively improving the learning effect. In summary, the system significantly optimizes the management and application of continuing education courseware resources and promotes the process of educational informatization.

[0080] Figure 3 The second embodiment is shown, which is mainly different from the first embodiment in that: in the data storage module, the node load is monitored in real time by an algorithm, and the data shard size is dynamically adjusted according to the load of the storage node; for example, on a node with a higher load, the data shard is reduced to reduce the read and write pressure of a single node; the steps for adjusting the size of the data shard are:

[0081] a1 monitors the load of each node in real time;

[0082] a2 According to the average load And threshold T to determine whether the shard size needs to be adjusted:

[0083] a2.1 Calculate the average load of node i: Where N is the total number of storage nodes, L i Indicates the current load of node i (which can be a comprehensive evaluation of indicators such as CPU usage and I / O waiting time);

[0084] a2.2 Determine whether the shard size needs to be adjusted:

[0085] If L i > T and S i / N i > average shard size, then reduce the shard size;

[0086] If L i < T and S i / N i < average shard size, then increase the shard size;

[0087] a2.3 Adjust the shard size: S new，i = S i ± ΔS, where S i represents the total size of the original data shards on node i, S new，i represents the total size of the new data shards on node i, and ΔS is the shard size adjustment amount, which is a fixed value or a value dynamically calculated based on the load;

[0088] a3 Adjust the shard size as needed and update the number of shards:

[0089] a4 Redistribute the data shards to each node.

[0090] By real-time monitoring of node loads and dynamically adjusting the data shard size, the system realizes the optimized allocation of storage resources, effectively reduces the read / write pressure on high-load nodes, and improves the overall storage performance. This function ensures the high availability of data and the stability of the system.

[0091] The replication factor parameter adopts a replication strategy based on data access frequency. For frequently accessed data, increase the replication factor to improve data access speed; and introduce an intelligent replication algorithm to dynamically adjust the replication strategy according to the data access pattern and node status to balance data reliability and system performance; the steps of the replication strategy are:

[0092] b1 Monitor the access frequency of the data. Let the access frequency of data d be A f (d);

[0093] b2 Judge whether the data is frequently accessed according to the access frequency threshold. The method of judging whether the data is frequently accessed is: if A f (d)> T freq , then data d is frequently accessed data, where T freq represents the access frequency threshold;

[0094] b3 For frequently accessed data, increase its replication factor. The way to adjust the replication factor is: R new，f (d)= R f (d)+ ΔR f where, R f(d) represents the original replication factor of data d, R new，f (d) represents the new replication factor of data d, ΔR f Represents the replication factor adjustment amount;

[0095] b4 redistributes data copies to each node.

[0096] The introduction of replication strategies based on data access frequency and intelligent replication algorithms makes the distribution of data copies more reasonable. For frequently accessed data, by increasing the replication factor, the data access speed is significantly improved, thereby optimizing the user experience. At the same time, the intelligent replication algorithm can dynamically adjust the replication strategy according to the data access mode and node status, further balancing data reliability and system performance.

[0097] Use big data analysis technology to predict data growth trends and plan storage node expansion strategies in advance. When additional storage nodes are needed, data shards are placed near nodes with large data access volume to reduce data migration and access delays. The steps of the storage node expansion strategy are as follows:

[0098] c1 monitors the data access volume of each node;

[0099] c2 calculates the data migration priority based on the data access volume and distance factor. The formula for calculating the data migration priority is: Among them, D a (p) is the amount of data access on node p, P(p, q) is the probability or priority of data migration from node p to node q, and the distance factor is calculated based on the physical distance between nodes and network delay factors. k D a (k) represents the amount of data access to all nodes k D a (k) performing a summation;

[0100] c3 predicts the future data growth trend. The formula for predicting the data growth trend is: D future (t) = G(t), where G(t) represents the data growth trend prediction function, which is used to predict the data growth amount in the future time period t;

[0101] c4 determines the number of new nodes based on the prediction results. The calculation formula for the number of new nodes is: Among them, D future (t) represents the predicted data growth in the future time period t, ∑ n S n Indicates the total storage capacity of all current storage nodes. Indicates rounding up;

[0102] c5 migrates data shards to the newly added nodes according to the data migration priority.

[0103] The system also uses big data analysis technology to predict data growth trends and plan storage node expansion strategies in advance. This function not only avoids storage bottlenecks caused by data surges, but also reduces data migration and access delays by placing data shards near nodes with large data access volume, further improving system efficiency.

[0104] Figure 4 The third embodiment is shown, which is mainly different from the first embodiment in that the working content of the personalized recommendation module specifically includes:

[0105] d1 Data collection and preprocessing: Collect historical user behavior data, including course clicks, viewing time, ratings, favorites, and sharing behavior data; preprocess the data to remove noise data, including abnormal clicks and invalid views; use the user-course matrix to represent user behavior data, where rows represent users, columns represent courses, and matrix elements represent the intensity of a user's behavior on a course (such as click counts, viewing time, etc.);

[0106] d2 Feature extraction: extract features from user behavior data, such as user preference features (favorite course type, difficulty, etc.), user activity features (login frequency, viewing time, etc.), and calculate user similarity;

[0107] D3 recommendation algorithm selection and implementation: Select recommendation algorithms based on collaborative filtering, including user-based collaborative filtering and item-based collaborative filtering:

[0108] d3.1 User-based collaborative filtering: Based on user similarity, find other users who are similar to the target user and generate a recommendation list based on the preferences of these users;

[0109] d3.2 Project-based collaborative filtering: Based on the similarity between courses (which can be calculated using methods such as cosine similarity and Jaccard similarity), find other courses similar to the target course and generate a recommendation list based on the popularity of these courses.

[0110] The user similarity calculation method is: given two vectors u and v, where u and v both have Y dimensions, that is, u = (ru1, ru2, ..., ru Y ) and v = (rv1, rv2, ..., rv Y ), the formula of cosine similarity sim(u, v) is: Among them, y and rv y are the y-th subsets in sets u and v respectively.

[0111] In the personalized recommendation module, considering that user interests change over time, the time decay factor is introduced to assign different weights to historical behavior data, with recent behaviors being given a larger weight and long-term behaviors being given a smaller weight; the exponential decay function f(t) = e -λt , where t represents the time interval and λ is the attenuation coefficient.

[0112] By collecting and preprocessing user historical behavior data, extracting multidimensional features, and weighting historical behavior data with a time decay factor, this module can more accurately reflect the user's current interest preferences. At the same time, the recommendation algorithm based on collaborative filtering is used to combine user similarity and course similarity for recommendation, which not only improves the diversity of recommendations, but also enhances the personalization of recommendations. These improvements jointly improve the user experience, make the recommendation results more in line with user expectations, and enhance the interactivity and attractiveness of continuing education.

[0113] Figure 5 The fourth implementation mode is shown, and the main difference from the first implementation mode is that the working content of the learning path planning module specifically includes:

[0114] Plan the learning path using path planning algorithms (such as dynamic programming, greedy algorithms, etc.) based on the user's learning progress (such as completed courses, courses being studied, etc.) and course dependencies (such as prerequisite courses, follow-up courses, etc.);

[0115] Provide relevant course and resource recommendations to help users complete learning tasks efficiently, and dynamically adjust learning paths based on users’ learning goals and progress;

[0116] The specific steps of learning path planning are:

[0117] e1 Course dependency construction: Analyze the dependency between courses, and assume that the course dependency graph is a directed graph G = (V, E), where V is a node set representing courses, and E is an edge set representing the dependency between courses; the edge weight w (a, b) represents the dependency strength from node a of the course to node b of the course or the priority of the learning order;

[0118] e2 User learning progress tracking: Record the user's learning progress. Let the user's learning progress be a vector Z = (Z1, Z2, ..., Z g ), where Z j represents the user's learning progress for the jth course; for courses that have not yet started, Z j =0; for completed courses, Z j =1; for the course being studied, 0 <Z j <1;

[0119] e3 Learning path planning: The goal is to find a path from the user's current learning state to the user's learning goal, that is, from the course node corresponding to the non-zero element in vector Z to the path to complete all required courses, specifically:

[0120] Let Z(a) represent the set of paths to node a, and C(a) represent the set of subsequent nodes of node a, that is, all nodes b with edges (a, b)∈E;

[0121] For each node a, calculate the cost (or priority) of the optimal path (or path set) to reach node a, represented by f(a): f(a) = min b∈C(a) {f(b)+w(b,a)} (for the case of finding the minimum cost path);

[0122] Initial conditions: For the starting node s, f(s) = 0; for other nodes, f(a) = ∞ at the beginning, indicating unreachable;

[0123] Termination condition: When the f(a) values ​​of all nodes are calculated, or a path from the starting point to the end point is found, the algorithm terminates;

[0124] e4 personalized learning path optimization:

[0125] Introducing personalized learning paths: Planning personalized learning paths for users based on their learning styles and interest preferences; for example, for users who like challenges, we can recommend more difficult courses; for users who like to progress step by step, we can recommend more basic courses as prerequisites; introducing user preference vector O = (O1, O2, ..., O L ), where O c represents the user's preference score for the cth course;

[0126] When calculating f(a), consider user preferences and modify the recursive formula to: f(a) = min b∈C(a) {f(b)+w(b,a)-α·r a}, where λ is the preference adjustment coefficient;

[0127] Dynamically adjust the learning path: According to the user's learning feedback (such as course difficulty evaluation, learning satisfaction, etc.), dynamically adjust the edge weight w(b, a) or user preference vector r to optimize the learning path.

[0128] By building a course dependency graph and tracking user learning progress, the module can provide users with the optimal path from the current state to the learning goal. At the same time, the module introduces personalized learning path optimization, adjusts the path planning according to the user's learning style and interest preferences, and improves the pertinence and efficiency of learning. In addition, the module can dynamically adjust the path based on user learning feedback to ensure that the learning path always meets the actual needs of users. These improvements have jointly improved the user learning experience, enhanced the flexibility and effectiveness of continuing education, made the learning process more in line with personal needs, and improved the quality and satisfaction of learning outcomes.

[0129] Meanwhile, the contents not described in detail in this specification belong to the prior art known to those skilled in the art.

[0130] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0131] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An intelligent management system for continuing education courseware resources based on big data, characterized in that: Includes the following modules: The data collection module uses web crawler technology to capture courseware resource data, including videos, documents, and pictures, based on a preset URL list and crawling frequency; The data preprocessing module is used to clean the collected data, process missing values ​​and outliers, and use standardized formulas to normalize the data to provide a data basis for subsequent processing; A data storage module is used to store the processed data into a distributed storage system and configure the storage system according to the number of storage nodes, data shard size and replication factor parameters; The data management module is used to classify and label courseware resources and build indexes for users to find and obtain the required resources; Personalized recommendation module, which is used to collect user behavior data, use recommendation algorithms to generate personalized resource recommendation lists, and provide accurate recommendations based on user interests and learning behaviors; The learning path planning module is used to plan the learning path based on the user's learning progress and course dependencies using a path planning algorithm, provide relevant course and resource recommendations, and dynamically adjust the learning path.

2. According to the big data-based intelligent management system for continuing education courseware resources according to claim 1, it is characterized by: In the data storage module, the node load is monitored in real time through an algorithm, and the data shard size is dynamically adjusted according to the load of the storage node; The replication factor parameter adopts a replication strategy based on data access frequency. For frequently accessed data, the replication factor is increased to improve data access speed; And introduce intelligent replication algorithm to dynamically adjust replication strategy according to data access mode and node status; Use big data analysis technology to predict data growth trends and plan storage node expansion strategies in advance. When additional storage nodes are needed, prioritize placing data shards near nodes with high data access volume to reduce data migration and access delays.

3. According to the big data-based intelligent management system for continuing education courseware resources according to claim 2, it is characterized by: The steps of adjusting the data shard size are as follows: a1 monitors the load of each node in real time; a2 According to the average load And threshold T to determine whether the shard size needs to be adjusted: a2.1 Calculate the average load of node i: Where N is the total number of storage nodes, L i represents the current load of node i; a2.2 Determine whether the shard size needs to be adjusted: If L i >T and S i / N i > Average shard size, then reduce the shard size; If L i < is less than T and S i / N i < the average shard size, then increase the shard size; a2.3 Adjust the fragment size: S new，i =S i ±ΔS, where S i represents the total size of the original data shards on node i, S new，i represents the total size of the new data shards on node i, ΔS is the shard size adjustment amount, which is a fixed value or a value dynamically calculated based on the load; a3 Adjust the shard size as needed and update the number of shards: a4 redistributes data shards to each node.

4. According to the big data-based intelligent management system for continuing education courseware resources according to claim 2, it is characterized by: The steps of the replication strategy are: b1 monitors the access frequency of data, assuming that the access frequency of data d is A f (d); b2 determines whether the data is frequently accessed based on the access frequency threshold. The method for determining whether the data is frequently accessed is: f (d)>T freq , then data d is frequently accessed data, where T freq Indicates the access frequency threshold; b3 For frequently accessed data, increase its replication factor. The way to adjust the replication factor is: R new，f (d) = R f (d)+ΔR f , where R f (d) represents the original replication factor of data d, R new，f (d) represents the new replication factor of data d, ΔR f Represents the replication factor adjustment amount; b4 redistributes data copies to each node.

5. According to the big data-based intelligent management system for continuing education courseware resources according to claim 2, it is characterized by: The steps of the storage node expansion strategy are: c1 monitors the data access volume of each node; c2 calculates the data migration priority based on the data access volume and distance factor. The formula for calculating the data migration priority is: Among them, D a (p) is the amount of data access on node p, P(p, q) is the probability or priority of data migration from node p to node q, and the distance factor is calculated based on the physical distance between nodes and network delay factors. k D a (k) represents the amount of data access to all nodes k D a (k) performing a summation; c3 predicts the future data growth trend. The formula for predicting the data growth trend is: D future (t) = G(t), where G(t) represents the data growth trend prediction function, which is used to predict the data growth amount in the future time period t; c4 determines the number of new nodes based on the prediction results. The calculation formula for the number of new nodes is: Among them, D future (t) represents the predicted data growth in the future time period t, ∑ n S n Indicates the total storage capacity of all current storage nodes. Indicates rounding up; c5 migrates data shards to the newly added nodes according to the data migration priority.

6. The intelligent management system for continuing education courseware resources based on big data according to claim 1, characterized in that: The work content of the personalized recommendation module specifically includes: d1 Data collection and preprocessing: Collect historical user behavior data, including course clicks, viewing time, ratings, favorites, and sharing behavior data; preprocess the data to remove noise data, including abnormal clicks and invalid views; use the user-course matrix to represent user behavior data, where rows represent users, columns represent courses, and matrix elements represent the intensity of a user's behavior on a course; d2 Feature extraction: extract features from user behavior data, such as user preference features, user activity features, and calculate user similarity; D3 recommendation algorithm selection and implementation: Select recommendation algorithms based on collaborative filtering, including user-based collaborative filtering and item-based collaborative filtering: d3.1 User-based collaborative filtering: Based on user similarity, find other users who are similar to the target user and generate a recommendation list based on the preferences of these users; d3.2 Project-based collaborative filtering: Based on the similarity between courses, find other courses similar to the target course and generate a recommendation list based on the popularity of these courses.

7. The big data-based intelligent management system for continuing education courseware resources according to claim 6 is characterized by: The user similarity calculation method is: given two vectors u and v, where u and v both have Y dimensions, that is, u = (ru1, ru2, ..., ru Y ) and v = (rv1, rv2, ..., rv Y ), the formula of cosine similarity sim(u, v) is: Among them, y and rv y are the y-th subsets in sets u and v respectively.

8. The big data-based intelligent management system for continuing education courseware resources according to claim 6 is characterized by: In the personalized recommendation module, considering that user interests change over time, a time decay factor is introduced to assign different weights to historical behavior data, with recent behaviors being assigned a larger weight and long-term behaviors being assigned a smaller weight; Using exponential decay function f(t) = e -λt , where t represents the time interval and λ is the attenuation coefficient.

9. The intelligent management system for continuing education courseware resources based on big data according to claim 1, characterized in that: The working content of the learning path planning module specifically includes: Use path planning algorithms to plan learning paths based on user learning progress and course dependencies; Provide relevant course and resource recommendations to help users complete learning tasks efficiently, and dynamically adjust the learning path according to the user's learning goals and progress.

10. The intelligent management system for continuing education courseware resources based on big data according to claim 9, characterized in that: The steps of the learning path planning are specifically as follows: e1 Course dependency construction: Analyze the dependency between courses, and assume that the course dependency graph is a directed graph G = (V, E), where V is a node set representing courses, and E is an edge set representing the dependency between courses; the edge weight w (a, b) represents the dependency strength from node a of the course to node b of the course or the priority of the learning order; e2 User learning progress tracking: Record the user's learning progress. Let the user's learning progress be a vector Z = (Z1, Z2, ..., Z g ), where Z j represents the user's learning progress for the jth course; for courses that have not yet started, Z j =0; for completed courses, Z j =1; for the course being studied, 0 <Z j <1; e3 Learning path planning: The goal is to find a path from the user's current learning state to the user's learning goal, that is, from the course node corresponding to the non-zero element in vector Z to the path to complete all required courses, specifically: Let Z(a) represent the set of paths to node a, and C(a) represent the set of subsequent nodes of node a; For each node a, calculate the cost of the optimal path to reach node a, represented by f(a): f(a) = min b∈C(a) {f(b)+w(b,a)}; Initial conditions: For the starting node s, f(s) = 0; for other nodes, f(a) = ∞ at the beginning, indicating unreachable; Termination condition: When the f(a) values ​​of all nodes are calculated, or a path from the starting point to the end point is found, the algorithm terminates; e4 personalized learning path optimization: Introducing personalized learning paths: Planning personalized learning paths for users based on their learning styles and interest preferences; Introducing user preference vector O = (O1, O2, ..., O L ), where O c represents the user's preference score for the cth course; When calculating f(a), consider user preferences and modify the recursive formula to: f(a) = min b∈C(a) {f(b)+w(b,a)-α·r a }, where λ is the preference adjustment coefficient; Dynamically adjust the learning path: According to the user's learning feedback, dynamically adjust the edge weight w(b, a) or the user preference vector r to optimize the learning path.

Images