A data storage method and system for a smart education platform
Through technologies such as redundant data elimination, missing value interpolation, sharded storage and multi-dimensional grid mapping, the problems of data redundant, missing value and low storage efficiency of smart education platforms are solved, and efficient and secure data management and complexity are achieved. Query to adapt to the needs of massive educational data.
Patent Information
- Application Number
- CN202411225155.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-03
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2044-09-03
AI Technical Summary
Smart education platforms face problems such as data redundancy, missing values, low storage efficiency, poor security and high management complexity. Especially in big data and high concurrency scenarios, traditional storage methods are difficult to meet the needs of complex query and analysis.
Through redundant data elimination, missing value mean interpolation, sharded storage, multi-dimensional grid mapping, dynamic indexing and intelligent storage strategies, efficient deduplication, integrity, security and flexibility of data are achieved, and complex queries and analysis are supported.
It improves the efficiency and security of data storage, reduces the impact of data redundancy and missing values on analysis results, enhances the uniqueness and reliability of data, and supports the efficient management and query performance of massive educational data.
Smart Images

Figure CN118760686B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of data storage, and in particular to a data storage method and system for a smart education platform. Background Art
[0002] Initially, the smart education platform used the traditional file storage method to store data in the form of files on local hard disks or servers. This method is inconvenient to manage, has poor scalability, and is difficult to meet the needs of the big data era. Subsequently, the platform began to use relational databases to store structured teaching and learning data. This method can better manage structured data, but it is difficult to adapt to the storage of unstructured data (such as text, images, and videos), and the query efficiency decreases as the data size increases. As the amount of data increases further, the platform began to use distributed storage systems, such as distributed file systems and distributed databases, to achieve distributed storage and management of data. This method can effectively cope with large amounts of data and high concurrency, but the system complexity increases and the management and maintenance costs are high. There is no sufficient redundant data elimination and missing value processing mechanism, resulting in a large amount of duplicate information and incomplete data in the data set, which affects the accuracy and reliability of the analysis results. In addition, the static storage structure and indexing method used will lead to low data access efficiency, especially when the data volume is large or the query is complex. Summary of the invention
[0003] Based on this, it is necessary to provide a data storage method and system for a smart education platform to solve at least one of the above technical problems.
[0004] To achieve the above purpose, a data storage method for a smart education platform includes the following steps:
[0005] Step S1: Obtaining smart education platform usage data; performing redundant data elimination on the smart education platform usage data to obtain deduplicated platform usage data; performing missing value mean interpolation on the deduplicated platform usage data to generate standard platform usage data;
[0006] Step S2: Slice and store the standard platform usage data to generate course hierarchical storage data; encrypt the course hierarchical storage data independently to generate hierarchical encrypted storage data;
[0007] Step S3: Perform multidimensional grid mapping on the hierarchical encrypted storage data to obtain multidimensional grid storage data; perform dynamic index construction on the multidimensional grid storage data to generate a multidimensional grid index;
[0008] Step S4: constructing a storage strategy for the multidimensional grid storage data and the multidimensional grid index to obtain a multidimensional storage strategy;
[0009] Step S5: Adjust the storage method of the hierarchical encrypted storage data based on the multi-dimensional storage strategy, and introduce a data elimination mechanism to implement the data intelligent storage method of the smart education platform.
[0010] The present invention effectively reduces data redundancy by removing duplicate data, thereby saving storage space and improving processing speed. This cleaning process ensures the uniqueness and accuracy of data and avoids calculation errors caused by duplicate data. The method of filling missing values makes the data set more complete and consistent, avoiding analysis deviations caused by incomplete data. By interpolating the mean value, the reliability of data analysis results can be maintained while maintaining data consistency. The security of data is improved by storing data in fragments and encrypting each fragment independently. This method ensures that even if part of the data is leaked, the data of other parts are still protected, reducing the risk of overall data leakage. Independently encrypting each course data fragment can ensure higher security protection, so that sensitive information is encrypted more accurately. Mapping data to a multidimensional grid can organize and store data more efficiently, supporting complex data structures and query requirements. This method makes the storage layout of data more in line with actual application requirements and improves the flexibility of data management. Establishing a dynamic index supports fast data retrieval and access. The index can be adjusted according to the actual use of the data, optimizes the query performance, and improves the response speed of data access. By comprehensively considering the multidimensional characteristics and storage requirements of the data, an optimized storage strategy is formulated. This strategy can effectively manage data of different levels and categories, and improve the efficiency and flexibility of storage. An intelligent data elimination mechanism is introduced to dynamically adjust the storage strategy based on the access frequency and importance of data. This mechanism can automatically clean up expired or unimportant data, release storage resources, and improve the overall performance and resource utilization of the storage system. Comprehensive optimization has been carried out in data processing, encryption, storage and management to achieve higher security, efficiency and intelligent management, meeting the high requirements of the smart education platform for data storage.
[0011] The present invention also provides a data storage system for a smart education platform, which is used to execute the data storage method for the smart education platform as described above. The data storage system for the smart education platform includes:
[0012] The data processing module is used to obtain the usage data of the smart education platform; eliminate redundant data of the usage data of the smart education platform to obtain deduplicated platform usage data; perform mean interpolation of missing values on the deduplicated platform usage data to generate standard platform usage data;
[0013] The data encryption module is used to store the standard platform usage data in slices to generate course hierarchical storage data; to encrypt the course hierarchical storage data independently to generate hierarchical encrypted storage data;
[0014] The management optimization module is used to perform multi-dimensional grid mapping on the hierarchical encrypted storage data to obtain multi-dimensional grid storage data; dynamically construct indexes on the multi-dimensional grid storage data to generate multi-dimensional grid indexes;
[0015] A strategy building module is used to build a storage strategy for multi-dimensional grid storage data and multi-dimensional grid indexes to obtain a multi-dimensional storage strategy;
[0016] The storage adjustment module is used to adjust the storage method of hierarchical encrypted storage data based on a multi-dimensional storage strategy, and introduce a data elimination mechanism to implement the data intelligent storage method of the smart education platform.
[0017] The benefit of the present invention lies in that standardized platform usage data is generated through redundant data elimination and missing value interpolation, providing a high-quality data foundation for subsequent storage, storing data in a hierarchical manner according to importance, and encrypting and protecting key data, thereby improving data security. This hierarchical storage design can effectively balance storage costs and access speeds, and is more in line with actual application requirements. Data is spatially stored through multidimensional grid mapping, and rich indexes are constructed to support complex query and analysis requirements. This multidimensional grid storage structure has strong scalability and can better adapt to the management of massive educational data. Dynamic indexing and data elimination mechanisms are introduced, and storage strategies can be adaptively adjusted according to actual access patterns. This intelligent storage method can make full use of storage resources, improve the performance and efficiency of data access, and integrate multiple technologies such as data preprocessing, hierarchical storage, multidimensional indexing, and intelligent scheduling, which have significant advantages in ensuring data security and improving access performance. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Figure 1 A schematic diagram of the steps of a data storage method for a smart education platform;
[0019] Figure 2 for Figure 1 Detailed implementation steps of step S2 in the flowchart;
[0020] Figure 3 for Figure 1 Detailed implementation steps of step S3 in FIG.
[0021] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0022] The technical method of the present invention is described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by technicians in this field without creative work are within the scope of protection of the present invention.
[0023] In addition, the accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor methods and / or microcontroller methods.
[0024] It should be understood that, although the terms "first", "second", etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are used only to distinguish one unit from another unit. For example, without departing from the scope of the exemplary embodiments, the first unit may be referred to as the second unit, and similarly the second unit may be referred to as the first unit. The term "and / or" used herein includes any and all combinations of one or more of the listed associated items.
[0025] To achieve this, please refer to Figures 1 to 3 , a data storage method for a smart education platform, comprising the following steps:
[0026] Step S1: Obtaining smart education platform usage data; performing redundant data elimination on the smart education platform usage data to obtain deduplicated platform usage data; performing missing value mean interpolation on the deduplicated platform usage data to generate standard platform usage data;
[0027] Step S2: Slice and store the standard platform usage data to generate course hierarchical storage data; encrypt the course hierarchical storage data independently to generate hierarchical encrypted storage data;
[0028] Step S3: Perform multidimensional grid mapping on the hierarchical encrypted storage data to obtain multidimensional grid storage data; perform dynamic index construction on the multidimensional grid storage data to generate a multidimensional grid index;
[0029] Step S4: constructing a storage strategy for the multidimensional grid storage data and the multidimensional grid index to obtain a multidimensional storage strategy;
[0030] Step S5: Adjust the storage method of the hierarchical encrypted storage data based on the multi-dimensional storage strategy, and introduce a data elimination mechanism to implement the data intelligent storage method of the smart education platform.
[0031] The present invention effectively reduces data redundancy by removing duplicate data, thereby saving storage space and improving processing speed. This cleaning process ensures the uniqueness and accuracy of data and avoids calculation errors caused by duplicate data. The method of filling missing values makes the data set more complete and consistent, avoiding analysis deviations caused by incomplete data. By interpolating the mean value, the reliability of data analysis results can be maintained while maintaining data consistency. The security of data is improved by storing data in fragments and encrypting each fragment independently. This method ensures that even if part of the data is leaked, the data of other parts are still protected, reducing the risk of overall data leakage. Independently encrypting each course data fragment can ensure higher security protection, so that sensitive information is encrypted more accurately. Mapping data to a multidimensional grid can organize and store data more efficiently, supporting complex data structures and query requirements. This method makes the storage layout of data more in line with actual application requirements and improves the flexibility of data management. Establishing a dynamic index supports fast data retrieval and access. The index can be adjusted according to the actual use of the data, optimizes the query performance, and improves the response speed of data access. By comprehensively considering the multidimensional characteristics and storage requirements of the data, an optimized storage strategy is formulated. This strategy can effectively manage data of different levels and categories, and improve the efficiency and flexibility of storage. An intelligent data elimination mechanism is introduced to dynamically adjust the storage strategy based on the access frequency and importance of data. This mechanism can automatically clean up expired or unimportant data, release storage resources, and improve the overall performance and resource utilization of the storage system. Comprehensive optimization has been carried out in data processing, encryption, storage and management to achieve higher security, efficiency and intelligent management, meeting the high requirements of the smart education platform for data storage.
[0032] In the embodiment of the present invention, reference Figure 1 The above is a schematic diagram of the steps of a data storage method of a smart education platform of the present invention. In this example, the data storage method of the smart education platform includes the following steps:
[0033] Step S1: Obtaining the usage data of the smart education platform; performing redundant data elimination on the usage data of the smart education platform to obtain deduplicated platform usage data; performing mean interpolation of missing values on the deduplicated platform usage data to generate standard platform usage data;
[0034] In an embodiment of the present invention, usage data including user behavior, course access records, and academic performance are obtained from the smart education platform. Data cleaning tools, such as Python's Pandas library, are used to check the data for redundancy, and a hash algorithm (such as MD5 or SHA-256) is used to identify and delete duplicate records, thereby obtaining deduplicated usage data. For missing values in the data, mean interpolation technology is applied to fill in the missing items by calculating the mean of each data field. These interpolation values are calculated based on the overall statistical characteristics of the data. The result is a complete and non-redundant standard platform usage data set.
[0035] Step S2: Slice and store the standard platform usage data to generate course hierarchical storage data; encrypt the course hierarchical storage data independently to generate hierarchical encrypted storage data;
[0036] In the embodiment of the present invention, the standard platform usage data is stored in pieces according to the course, and the data of each course is divided into several small blocks, which are stored in the form of data blocks in a distributed storage system, such as Apache Hadoop HDFS. Each course data block is encrypted independently, and a symmetric encryption algorithm such as AES (Advanced Encryption Standard) is used for encryption processing to ensure the confidentiality of each data block. Each encrypted block is processed using a different encryption key to generate hierarchical encrypted storage data to ensure the security and privacy protection of the data during the storage process.
[0037] Step S3: Perform multidimensional grid mapping on the hierarchical encrypted storage data to obtain multidimensional grid storage data; perform dynamic index construction on the multidimensional grid storage data to generate a multidimensional grid index;
[0038] In an embodiment of the present invention, hierarchical encrypted storage data is mapped to a multidimensional grid. Using spatial data processing tools, such as Gephi or D3.js, the data is mapped to a multidimensional grid, with each dimension representing a different data feature (e.g., course type, user behavior frequency). Subsequently, the dynamic index construction technology is used to optimize the data retrieval process by creating and maintaining an index tree (e.g., a B+ tree or an R tree). Dynamic indexing supports automatic updates when data is added, deleted, or modified, improves query performance, and generates a corresponding multidimensional grid index.
[0039] Step S4: constructing a storage strategy for the multidimensional grid storage data and the multidimensional grid index to obtain a multidimensional storage strategy;
[0040] In the embodiment of the present invention, a storage strategy is designed based on multidimensional grid storage data and multidimensional grid index. Data storage optimization tools, such as the lifecycle management function of Amazon S3, are used to create storage rules, including hot and cold data tiering strategies and redundant backup strategies. The hot and cold data strategy migrates data with low access frequency to a low-cost storage layer, while frequently accessed data is kept in a fast-access storage layer. The redundant backup strategy ensures that each data block has multiple copies stored in different physical locations to prevent data loss.
[0041] Step S5: Adjust the storage method of the hierarchical encrypted storage data based on the multi-dimensional storage strategy, and introduce a data elimination mechanism to implement the data intelligent storage method of the smart education platform.
[0042] In the embodiment of the present invention, the storage mode of hierarchically encrypted storage data is adjusted based on a multi-dimensional storage strategy. Data storage management tools, such as Elasticsearch, are used to configure and adjust storage to adapt to actual data access patterns and storage requirements. A data elimination mechanism is introduced to automatically remove data that is no longer needed by setting data retention policies (such as based on time or access frequency). Data management scripts (such as Python or Shell scripts) are used to regularly check and delete expired data to ensure the effective use of storage resources and maintain system performance and reasonable allocation of storage space.
[0043] Preferably, step S1 comprises the following steps:
[0044] Step S11: Acquire a smart education platform learning terminal; perform real-time monitoring on the smart education platform learning terminal to obtain usage data of the smart education platform;
[0045] Step S12: Formatting the smart education platform usage data to generate consistent platform usage data;
[0046] Step S13: removing duplicate data from the consistent platform usage data to obtain deduplicated platform usage data;
[0047] Step S14: performing missing value pattern recognition on the deduplication platform usage data to obtain a data missing value pattern;
[0048] Step S15: Perform missing value mean interpolation on the deduplication platform usage data according to the data missing value pattern to generate standard platform usage data.
[0049] The benefit of the present invention is that by real-time monitoring of the learning terminal of the smart education platform, the platform usage data can be obtained in time. This real-time data acquisition mechanism ensures the timeliness and accuracy of the data, and provides the latest basic data for subsequent data processing and analysis. Formatting the usage data of the smart education platform to generate a consistent usage data format helps to standardize data from different sources or in different formats. This consistency is crucial for data fusion, comparison and analysis, and can significantly improve the efficiency of data processing and the reliability of analysis results. By removing duplicate data from the formatted data, redundant information can be effectively reduced and repeated calculation and storage of data can be avoided. This not only reduces the demand for data storage, but also improves the efficiency of data processing. By identifying the pattern of missing values in the data, the distribution of missing data and its causes can be understood. This process is a key step in improving data quality and data analysis for understanding the nature of missing data and providing a basis for subsequent missing value processing. Based on the identified missing value pattern, the mean interpolation of the usage data of the deduplication platform can effectively fill the missing data and reduce the impact of missing data on the analysis results. This interpolation method is simple and easy to implement, and plays an important role in maintaining the integrity of the data and the accuracy of the analysis. By implementing real-time data monitoring, formatting, consistency processing, duplicate data removal, missing value pattern recognition and mean interpolation, the data quality, processing efficiency and analysis accuracy have been significantly improved. These improvements help to more accurately evaluate and optimize the use of the smart education platform, thereby improving the platform's educational effectiveness and user experience.
[0050] In an embodiment of the present invention, a monitoring system is deployed on a learning terminal of a smart education platform, and a network protocol (such as HTTP / HTTPS) is used to collect learning activity data through an API interface, including user login, course browsing, and exercise submission information. A real-time data acquisition tool, such as Apache Kafka or AWS Kinesis, is used to ensure that data can be captured and transmitted to a data processing system in real time. The timestamp and event type during the data collection process are recorded to generate comprehensive smart education platform usage data. During the real-time monitoring process, the system continuously tracks terminal activities to obtain real-time data streams to ensure the timeliness and accuracy of the data. The raw data obtained from the learning terminal is formatted. A data format conversion tool, such as Python's Pandas library or Apache NiFi, is used to uniformly convert the data into a structured format (such as CSV or JSON). The data formatting steps include standardizing field names, unifying timestamp formats, and ensuring that all data fields conform to a predetermined data model. For unstructured data (such as log files), it is converted into structured data through a parser to ensure the consistency of each field. Duplicate data is removed for consistent platform usage data. Use data deduplication tools, such as the drop_duplicates() method in Pandas or the DISTINCT keyword in SQL, to detect and remove duplicate records. Specific operations include identifying duplicates based on unique identifiers in the data (such as user IDs and timestamps), and retaining unique records for each user at each time point. Generate deduplication platform usage data. Perform missing value pattern recognition on the deduplication platform usage data. Use missing value analysis tools, such as Python's missingno library, to analyze the distribution and pattern of missing values in the data. Generate a missing value matrix through visualization tools to identify the distribution characteristics and patterns of missing values (for example, some fields are missing more frequently in a specific time period). When performing pattern recognition, examine the dependencies of missing values and determine whether the missing pattern is related to the data collection process or specific events, so as to extract the data missing value pattern. Based on the identified data missing value pattern, perform missing value mean interpolation on the deduplication platform usage data. Use statistical computing tools, such as the sklearn library in Python, to calculate the mean of each field, and then replace the missing values with the mean of the corresponding field. The interpolation step includes calculating the mean and filling in missing values for numerical data fields, and calculating the mode value for categorical data fields. Generate standard platform usage data.
[0051] Preferably, step S2 comprises the following steps:
[0052] Step S21: classify the standard platform usage data into specific courses to obtain classified course usage data;
[0053] Step S22: Slice and store the standard platform usage data according to the classified course usage data to generate course hierarchical storage data;
[0054] Step S23: extracting course features from the course hierarchical storage data to obtain hierarchical course feature data;
[0055] Step S24: encrypting the course hierarchical storage data independently based on the hierarchical course characteristic data to generate hierarchical encrypted storage data.
[0056] The invention is beneficial in that the standard platform usage data is classified into specific courses, which helps to organize the data by course type. This classification not only makes data management more efficient, but also supports analysis and optimization for specific courses. For example, the education platform can adjust the course content, teaching methods or resource allocation according to the usage of different courses, thereby improving the quality and effect of education. The standard platform usage data is stored in pieces according to the classified course usage data to generate course hierarchical storage data. This method stores data in a hierarchical manner according to course categories and importance, improves the efficiency of data access, and allows the platform to dynamically adjust storage resources according to course requirements. This flexible storage method can effectively manage massive amounts of educational data while reducing the waste of storage space. Course feature extraction is performed on the course hierarchical storage data to obtain hierarchical course feature data. This process helps identify and extract the key features of each course, such as course difficulty, learning progress, and student participation, thereby providing data support for further analysis and decision-making. The extraction of course feature data can help the platform understand course content and student needs, and optimize course settings and teaching strategies. Based on the hierarchical course feature data, the course hierarchical storage data is encrypted independently to generate hierarchical encrypted storage data. This hierarchical encryption method ensures the security of course data. By independently encrypting different course data, data leakage and unauthorized access can be effectively prevented. At the same time, independent encryption also allows different levels of security policies to be implemented for different course data, further enhancing data protection measures. By classifying, sharding, extracting features, and independently encrypting course data, the accuracy of data management, storage efficiency, depth of analysis, and data security are improved. These improvements not only enable the smart education platform to better process and utilize large amounts of educational data, but also enhance the protection of data privacy and security, providing reliable data support for the long-term development of the platform.
[0057] As an example of the present invention, refer to Figure 2 As shown, in this example, step S2 includes:
[0058] Step S21: classify the standard platform usage data into specific courses to obtain classified course usage data;
[0059] In an embodiment of the present invention, the standard platform usage data is used to classify courses. First, a data analysis tool, such as Python's Pandas library or Apache Spark, is imported to perform data preprocessing. Field mapping is performed on the data, and the course data is marked according to a predetermined classification standard (such as course type, difficulty level or subject area). Classification algorithms, such as K-means clustering or decision trees, are used to analyze course characteristics and automatically classify courses into corresponding categories. Classified course usage data is generated by analyzing the attributes of each course (such as course subject, target audience, teaching content), ensuring that each course is correctly classified according to its characteristics for subsequent storage and processing.
[0060] Step S22: Slice and store the standard platform usage data according to the classified course usage data to generate course hierarchical storage data;
[0061] In an embodiment of the present invention, the standard platform usage data is sharded and stored according to the classified course usage data. A distributed storage system, such as Apache Hadoop HDFS or Amazon S3, is used to shard the course data according to the classification results. The data of each course category is stored as an independent storage block or directory, and these storage blocks are organized according to the course type or subject area. The sharded storage includes defining a storage strategy, such as partitioning data according to category, storing the data of each course category in a dedicated storage location, and generating course hierarchical storage data. This method improves storage efficiency and facilitates data retrieval and management.
[0062] Step S23: extracting course features from the course hierarchical storage data to obtain hierarchical course feature data;
[0063] In an embodiment of the present invention, course feature extraction is performed on the hierarchical storage data of courses. Feature engineering tools and techniques, such as the scikit-learn library of Python or the caret package of R language, are used to extract key features of course data. These features include keywords of course content, learning objectives, teaching methods, and user evaluations. The course description text is analyzed by natural language processing technology (such as TF-IDF or Word2Vec) to extract meaningful feature vectors. The result of feature extraction is hierarchical course feature data, which contains detailed feature information of each course and can support subsequent course analysis and processing.
[0064] Step S24: encrypting the course hierarchical storage data independently based on the hierarchical course characteristic data to generate hierarchical encrypted storage data.
[0065] In an embodiment of the present invention, based on the graded course feature data, the course-level storage data is encrypted independently. First, an encryption library, such as Python's cryptography library or Java's javax.crypto package, is used to generate a unique encryption key for each course data block. A symmetric encryption algorithm (such as AES) is applied to the data block of each course category to ensure that the data of each course is encrypted with an independent key. During the encryption process, the key management system (such as AWS KMS or Azure Key Vault) is responsible for the generation and storage of keys. Finally, hierarchical encrypted storage data is generated to ensure that the data of each course is effectively protected and encrypted during storage, and the security and privacy of the data are maintained.
[0066] Preferably, step S24 comprises the following steps:
[0067] Step S241: performing corresponding feature matching on the course grading storage data based on the graded course feature data to obtain matching course grading data;
[0068] Step S242: performing feature vectorization mapping on the graded course feature data to generate a course feature vector; performing segmentation on the course feature vector to obtain course feature vector segments;
[0069] Step S243: performing unique feature recognition on the course feature vector segment to obtain segment unique identification data; performing hash processing on the segment unique identification data to generate a feature segment hash value;
[0070] Step S244: XOR-mixing the hash values of the feature fragments to obtain a mixed identifier; performing secure hash processing on the mixed identifier to generate a course encryption key;
[0071] Step S245: encrypting the course graded storage data independently based on the course encryption key and matching the course graded data to generate initial graded encrypted storage data;
[0072] Step S246: construct a unique identification code for the graded course feature data to obtain the graded course unique identification data; use the graded course unique identification data to perform surface annotation on the initial graded encrypted storage data to generate graded encrypted storage data.
[0073] The benefit of the present invention lies in that the course data can be accurately matched with its features by performing feature matching and feature vectorization mapping on the course hierarchical storage data. This method not only helps to more accurately identify and extract the core features of the course data, but also converts the course data into a vector form that can be used for machine learning and data analysis. Feature vectorization and fragment cutting make the course data processing more detailed and efficient, which helps to carry out more complex analysis and modeling. By performing unique feature identification on the course feature vector fragment and generating a hash value, a unique and unforgeable data identifier can be created. This method effectively enhances the uniqueness and integrity check of the data, helps to prevent data tampering and ensure the authenticity of the data. The feature fragment hash value is XORed and securely hashed to generate a course encryption key. This process combines mixing and hashing technology to provide a more complex and secure encryption key generation mechanism. Through multi-level encryption processing, the key can be effectively prevented from being cracked, and the security of data encryption can be improved. Based on the generated course encryption key, the course hierarchical storage data is independently encrypted to ensure the security of each course data. The independent encryption method can effectively protect the sensitive information of each course and prevent unauthorized access or data leakage. Unique identification codes are constructed for the characteristic data of graded courses, and these identification codes are used to annotate the surface of encrypted data to generate graded encrypted storage data. This annotation method helps to embed unique identification information in encrypted data, which facilitates data management, retrieval and auditing, while maintaining data encryption protection. Through feature matching, vectorized mapping, unique identification, hash processing, secure mixing and independent encryption, the security of data, the accuracy of management and the complexity of encryption mechanism are improved. This multi-level processing method ensures that the data in the smart education platform can be processed and analyzed efficiently, and can be strictly protected, which is conducive to the long-term stable operation of the platform and the privacy protection of user data.
[0074] In an embodiment of the present invention, feature matching is performed on course graded stored data according to graded course feature data. First, a feature matching algorithm, such as K-Nearest Neighbors (KNN) or Support Vector Machine (SVM), is used to compare the graded course feature data with the stored data. During the feature matching process, the optimal match between each course data block and its features is determined by calculating the similarity of the feature vector. Feature matching calculations are performed using tools such as Python's scikit-learn library or TensorFlow, and matching course graded data are generated. A feature vectorization tool, sklearn.feature_extraction.text.TfidfVectorizer in Python, is used to convert course feature data (such as course descriptions and teaching objectives) into feature vectors. Feature vectorization includes converting text data into vector representations in numerical form. Then, vector fragment cutting is performed to divide the feature vector into several fixed-length fragments, for example, using Numpy's reshape function to cut the vector according to a specific size. Course feature vector fragments are generated to provide basic data for subsequent unique feature identification and encryption. Unique feature identification is performed on the course feature vector segments, and a unique feature extraction algorithm such as Principal Component Analysis (PCA) or a unique hash function is used to identify the unique identifier of each segment. Using Python's hashlib library, a hash algorithm (such as SHA-256) is applied to generate unique identification data for each feature segment. The segment unique identification data is hashed to convert the identified unique features into a fixed-length hash value. The feature segment hash value is generated. The feature segment hash values are XORed to generate a mixed identifier. First, multiple hash values are XORed bitwise using Python's ^ operator. For example, hash value A is XORed with hash value B to obtain an intermediate result. The mixed identifier is securely hashed, and the SHA-256 or SHA-3 algorithm is applied to re-hash the mixed identifier to generate a course encryption key. The course graded storage data is independently encrypted based on the generated course encryption key. First, each course graded data block is encrypted using a cryptography library such as Python's cryptography library and a symmetric encryption algorithm (such as AES). Each data block is encrypted with the corresponding course encryption key to ensure the confidentiality and security of the data block. The encryption process includes generating keys, setting encryption parameters (such as encryption mode is CBC or GCM), encrypting the data block, and generating initial hierarchical encrypted storage data. Each course data block is encrypted with an independent key to ensure data security.Construct unique identification codes for hierarchical course feature data, and use unique identifier generation tools, such as UUID libraries, to create unique identification codes for each course. Associate the identification codes with the initial hierarchical encrypted storage data, and use the identification codes to add surface annotations to each data block. Specific operations include embedding the identification codes into the metadata of the encrypted data, marking the source and characteristics of each data block. Generate hierarchical encrypted storage data.
[0075] Preferably, step S3 comprises the following steps:
[0076] Step S31: performing spatial data conversion on the hierarchical encrypted storage data to obtain hierarchical encrypted storage spatial data;
[0077] Step S32: performing multi-dimensional grid mapping on the hierarchical encrypted storage space data to obtain multi-dimensional grid storage data;
[0078] Step S33: dynamically reconstruct the index of the multi-dimensional grid storage data to generate a multi-dimensional grid index.
[0079] The invention is beneficial in that spatial data conversion is performed on hierarchical encrypted storage data, and the encrypted data can be mapped to a suitable spatial data structure. This conversion helps to optimize the way of data storage, making the distribution of data on physical storage devices more efficient and reducing the delay of access operations. In addition, spatial data conversion can improve the management and access efficiency of data, especially when processing large-scale data. Multidimensional grid mapping is performed on hierarchical encrypted storage spatial data, and the data is mapped to a multidimensional grid structure. This mapping method allows data to be organized into grids of multiple dimensions, supporting more complex query and analysis operations. Multidimensional grid mapping can improve the access efficiency of data, especially when multidimensional analysis is required, such as comparison of students' grades in various subjects and multi-angle evaluation of learning progress. Dynamic index reconstruction is performed on multidimensional grid storage data to generate a multidimensional grid index. This process optimizes the retrieval performance of data by constructing a dynamic index. The dynamic index can be adjusted in real time according to the actual use of the data, effectively improving the speed of data retrieval and query. Especially when frequent updates or inquiries of large amounts of data are required, dynamic index reconstruction can significantly reduce query time, improve the response speed of the system and user experience. Spatial data conversion helps to store encrypted data in a more efficient way and reduce storage costs. Multidimensional grid mapping supports more efficient data organization and access, especially for complex multidimensional data analysis. Dynamic index reconstruction makes data query and retrieval faster, adapts to data changes and actual usage, and improves the overall performance of the system. It enables the smart education platform to process, store and access large-scale data more efficiently, while improving the flexibility of the system and the user experience.
[0080] As an example of the present invention, refer to Figure 3 As shown, in this example, step S3 includes:
[0081] Step S31: performing spatial data conversion on the hierarchical encrypted storage data to obtain hierarchical encrypted storage spatial data;
[0082] In an embodiment of the present invention, spatial data conversion is performed on hierarchical encrypted storage data. First, a data conversion tool, such as GDAL (Geospatial Data Abstraction Library), is used to convert the encrypted storage data into a spatial data format. This process includes mapping the encrypted data to a geographic coordinate system (such as WGS84) and defining the coordinate system and projection mode of the spatial data. The specific implementation includes converting the encryption information of each data block into a spatial location identifier, applying a spatial data processing tool, and generating hierarchical encrypted storage spatial data.
[0083] Step S32: performing multi-dimensional grid mapping on the hierarchical encrypted storage space data to obtain multi-dimensional grid storage data;
[0084] In an embodiment of the present invention, multidimensional grid mapping is performed on hierarchically encrypted storage spatial data. A spatial data processing platform, such as ArcGIS or QGIS, is used to perform data mapping. Spatial data is converted into multidimensional grid data, which involves mapping each data block into a multidimensional grid according to predefined dimensions (e.g., spatial coordinates, timestamp, data type). The specific operation includes setting the grid dimension and resolution, and dividing the spatial data into grid cells according to these dimensions. The generated multidimensional grid storage data can organize the original encrypted data according to spatial and temporal information, making the data easy to query and analyze in a multidimensional space.
[0085] Step S33: dynamically reconstruct the index of the multi-dimensional grid storage data to generate a multi-dimensional grid index.
[0086] In an embodiment of the present invention, a dynamic data indexing tool, such as Elasticsearch or Apache Solr, is used to build and reconstruct a multidimensional grid index. The process includes creating an index structure (such as an R-tree or a KD-tree) to support data retrieval in a multidimensional space. The specific implementation steps include indexing each grid cell in the grid storage data, defining the dimension and range of the index, and updating the dynamic data changes in real time. Dynamic index reconstruction ensures that the index can adapt to data changes, improves data query performance and flexibility, and ultimately generates an optimized multidimensional grid index.
[0087] Preferably, performing multi-dimensional grid mapping on hierarchical encrypted storage space data includes:
[0088] Performing maximum principal component analysis on hierarchical encrypted storage space data to obtain the principal component with the maximum eigenvalue;
[0089] The contribution rate of the principal component with the largest eigenvalue is cumulatively calculated to generate the optimal dimension of the storage space;
[0090] Grid partitioning is performed on the hierarchical encrypted storage space data based on the optimal dimension of the storage space to obtain partitioned storage grid units;
[0091] Performing matrix transformation processing on hierarchical encrypted storage space data to generate an encrypted storage mapping matrix;
[0092] Data mapping is performed on the encrypted storage mapping matrix according to the partitioned storage grid units to obtain multi-dimensional grid storage data.
[0093] The invention is beneficial in that it can effectively reduce the dimension of the data while retaining important characteristic information by identifying and extracting the main components in the data through the maximum principal component analysis (PCA). This helps to reduce the complexity of the data, making subsequent storage and processing more efficient. The amount of data after the dimension reduction is smaller, the demand for calculation and storage is reduced, and the system processing speed and access efficiency are improved, which is particularly important when processing large-scale data. By calculating the contribution rate of the eigenvalue, the optimal dimension can be selected for storage to ensure that the key information of the data is effectively retained. This optimization can reduce the demand for storage space and improve the efficiency and accuracy of data processing. The optimal dimension selection helps to eliminate unnecessary redundant information, making the stored data more refined and effective, and reducing the storage and calculation burden caused by data redundancy. Grid partitioning of data based on the optimal dimension of the storage space helps to organize the data into structured grid units, making the storage and retrieval of data more efficient. This partitioning method supports more fine-grained data management and improves the efficiency of data access and maintenance. Grid partitioning provides structured storage for multidimensional data and supports more complex data query and analysis operations, such as multidimensional data slicing and data aggregation. By generating an encrypted storage mapping matrix through matrix transformation, data can be mapped to a multidimensional grid, improving the flexibility and efficiency of data storage. This mapping not only optimizes the storage structure of data, but also improves the access performance of data. The mapping process combines encryption technology to protect the privacy and security of data and ensure that sensitive information will not be leaked during the data mapping process. The multidimensional grid storage data structure enables data to be efficiently stored and accessed in multiple dimensions, which is particularly suitable for processing complex educational data such as student grades and course progress. The multidimensional grid structure makes data retrieval, analysis and visualization more efficient and intuitive, and can quickly obtain the required data views and analysis results, improving the speed and accuracy of decision-making. Through maximum principal component analysis, optimal dimension selection, grid partitioning, matrix transformation and data mapping, the data storage, management and processing efficiency of the smart education platform are significantly improved. These methods not only reduce data redundancy and optimize the storage structure, but also enhance data security and analysis capabilities, providing strong data support for the education platform.
[0094] In an embodiment of the present invention, a maximum principal component analysis (PCA) is performed on the hierarchical encrypted storage space data, using a data analysis tool, such as the PCA class in the scikit-learn library in Python. First, the hierarchical encrypted storage space data is input into the PCA algorithm to perform dimensionality reduction processing on the data. The implementation of PCA includes calculating the covariance matrix of the data, extracting eigenvalues and eigenvectors, and selecting principal components according to the size of the eigenvalues. The principal component with the largest eigenvalue is selected as the main characteristic dimension of the data, and the spatial information related to the principal component is retained. The principal component with the largest eigenvalue is finally used for subsequent dimension optimization and grid mapping processing. The contribution rate of the principal component with the largest eigenvalue is cumulatively calculated to determine the optimal dimension of the storage space. Using the results of PCA, the contribution rate of each principal component is calculated, that is, the ratio of the eigenvalue of each principal component to the sum of the eigenvalues of all principal components. By accumulating the contribution rate, the number of principal components required when the cumulative contribution rate reaches a set threshold (for example, 95%) is determined. The numpy library of Python is used for calculation to generate the optimal dimension of the storage space, which determines the main data dimension that needs to be retained in the multidimensional grid mapping. The data space is divided into equally spaced grid cells according to the optimal dimension. For example, the linspace function in Python's numpy library is used to perform spatial partitioning and project the data onto these grid cells. The result of grid partitioning is that each data block is assigned to a specific grid cell, generating a partitioned storage grid cell. The grid cells generated after data partitioning are mapped into a multidimensional matrix, and the dimension of the matrix is determined by the optimal dimension of the storage space. The data is rearranged into a matrix form using linear algebra tools, such as the reshape function in Python's numpy library. When performing matrix transformation processing, the data is linearly transformed by applying a transformation matrix (for example, through an eigenvector matrix) to generate an encrypted storage mapping matrix. This matrix maps each data cell to its corresponding grid position to prepare for multidimensional grid storage of the data. Data is mapped to the encrypted storage mapping matrix according to the partitioned storage grid cells to generate multidimensional grid storage data. First, the encrypted storage mapping matrix obtained after the matrix transformation processing is applied to the data of each grid cell, and the data is mapped to the corresponding grid position by matrix multiplication. The dot function in the numpy library is used to perform matrix multiplication to map the data to the multidimensional grid. Finally, the generated multi-dimensional grid storage data organizes data blocks through grid storage, so that the data can be effectively managed and queried in multi-dimensional space.
[0095] Preferably, dynamically constructing an index for multi-dimensional grid storage data includes:
[0096] Mapping filling curves of different dimensions to multi-dimensional grid storage data to obtain filled grid data;
[0097] Repeat the above steps until all dimension combinations are filled with grid data;
[0098] The multi-dimensional grid storage data is indexed and optimized based on the grid data filled in all dimensional combinations to obtain a filling optimized index;
[0099] Extract usage traces of the smart education platform usage data to obtain platform usage traces;
[0100] The platform uses trajectory and filling optimization indexes to integrate spatiotemporal filling indexes to obtain a multidimensional grid index.
[0101] The benefit of the present invention is that by mapping filling curves of different dimensions, multidimensional grid data can be effectively organized into a linear or approximately linear structure, and such data layout can better adapt to storage and retrieval operations. Filling curves (such as Z curves and Hilbert curves) help map multidimensional data into one-dimensional or low-dimensional space, reduce complex interactions between dimensions, and improve storage and retrieval efficiency. Dimension filling curve mapping organizes data into a continuous structure, which helps to improve the locality of data access, so that the performance when reading and processing data is better. By filling all dimensional combinations, it can be ensured that the index covers all combinations of multidimensional data and provides more comprehensive index support. This method can handle complex query and retrieval requests and improve the accuracy and efficiency of data retrieval. This comprehensive index construction allows more flexible and diverse queries on data, supporting various multidimensional data analysis scenarios, such as comprehensive learning performance analysis of students and course participation analysis. Index optimization based on the filled grid data of all dimensional combinations can effectively reduce the time complexity of query operations and improve data retrieval performance. Optimizing the index makes data access more efficient, especially when facing large-scale data, it can significantly shorten the query time. The optimized index structure is more compact, reduces storage overhead, and can better adapt to the dynamic changes of data. By extracting the platform usage trajectory, we can gain a deep understanding of the user's behavior patterns, learning habits, and interaction paths. This information is of great significance for optimizing the platform's functions and user experience and formulating personalized learning suggestions. The usage trajectory data can support personalized analysis and recommendations, and improve the education platform's ability to respond to user needs. By integrating the platform usage trajectory with the filling optimization index, an index structure for spatiotemporal data can be created. This comprehensive index can support complex spatiotemporal data analysis, such as the analysis of user behavior patterns at different times and places. The spatiotemporal filling index integration can better handle the timeliness and spatial characteristics of data, support real-time data analysis and dynamic data adjustment, and improve the overall processing capacity of the system. Through dimension filling curve mapping, comprehensive index coverage, index optimization, user trajectory extraction, and spatiotemporal data integration, the data storage and access efficiency of the smart education platform is significantly improved. This method not only optimizes the organizational structure of data and improves data retrieval performance, but also strengthens the ability to analyze user behavior, providing a more intelligent and efficient data management solution for the education platform.
[0102] In an embodiment of the present invention, different dimensions of filling curve mapping are performed on multi-dimensional grid storage data. First, a dimension (such as spatial position, timestamp, data type) is selected and mapped using a Hilbert curve or a Peano curve. The specific operation includes using the scipy.spatial.HilbertCurve module of the scipy library in Python to fill each dimension of the grid data according to the selected curve. This process projects the high-dimensional spatial data into a one-dimensional curve, thereby realizing the linearization of the data and maintaining the locality of the data. Repeat the above steps until the filled grid data of all dimensional combinations are obtained. This process includes filling curve mapping for combinations of different dimensions. For example, the spatial dimension and the time dimension are combined and the Hilbert curve is applied for filling mapping. The specific implementation includes using the numpy library of Python to generate a combination matrix of all dimensions, and applying a curve mapping algorithm to each combination to generate corresponding filled grid data. This step ensures that the data of each dimensional combination is subjected to curve mapping to generate comprehensive filled grid data, covering all data dimension combinations. Based on the filled grid data of all dimensional combinations, the multi-dimensional grid storage data is indexed and optimized. First, use index optimization algorithms, such as B-tree or KD-tree, to optimize the storage and query efficiency of the filled grid data. The specific implementation includes using the KDTree module in the scikit-learn library in Python to build an index tree. Insert the filled grid data into the index structure and optimize and adjust it according to the distribution of the data to reduce the time complexity of data retrieval. The resulting filled optimized index improves the retrieval speed and efficiency of multidimensional grid data and provides support for subsequent data integration and query. Extract usage trajectories from the usage data of the smart education platform. First, analyze the log data or user activity records of the smart education platform to extract the user's behavior trajectory. Use data processing tools, such as Python's pandas library, to perform time series analysis and trajectory extraction on the usage data to generate time series data of user activities. For example, convert the user's login, learning time, and access records into time series data and arrange them in chronological order. The obtained platform usage trajectory includes the activity path and time mark of each user, ready to be integrated with the filled optimized index. Perform spatiotemporal filling index integration on the platform usage trajectory and the filled optimized index. First, use spatiotemporal data integration tools, such as the spatialjoin function, to combine the data in the platform usage trajectory and the filled optimized index. Map the time and space information of the usage trajectory to the fill-in optimization index to form spatiotemporal index data. For example, through Python's geopandas library, spatially connect the platform usage trajectory data with the multidimensional grid index. The resulting multidimensional grid index combines spatiotemporal information to achieve efficient query and analysis of multidimensional data.
[0103] Preferably, step S4 comprises the following steps:
[0104] Step S41: performing data relationship driven analysis on the multidimensional grid storage data and the multidimensional grid index to generate a data relationship model;
[0105] Step S42: Analyze the storage requirements of the data relationship model to obtain the storage requirements of each data;
[0106] Step S43: Divide each data storage requirement into storage levels to obtain a data requirement storage level;
[0107] Step S44: constructing a storage strategy for the multidimensional grid storage data based on the data demand storage hierarchy to generate a multidimensional storage strategy.
[0108] The invention is beneficial in that data relationship driven analysis of multidimensional grid storage data and indexes can reveal the complex relationships and dependencies between data. By generating a data relationship model, the intrinsic structure and interaction of data can be better understood, which helps to optimize data storage and access strategies. Understanding data relationships can help the platform handle complex data operations, such as association analysis and interactive query. For the smart education platform, student performance analysis and course optimization operations can be performed more effectively. Storage demand analysis of the data relationship model helps to identify the specific storage requirements of different data types and data access modes. By understanding the storage requirements of each type of data in detail, resource allocation can be optimized to avoid resource waste. Accurate storage demand analysis can ensure the reasonable allocation of storage resources, improve the efficiency of data storage and access, and reduce unnecessary storage overhead. Storage hierarchies are divided for each data storage requirement, and different data requirements are divided into different storage hierarchies. This hierarchical storage strategy helps to optimize the storage structure and improve the access efficiency of data. Storage at different levels can be adjusted according to the importance and access frequency of the data, such as storing hot data in high-performance storage and storing cold data in low-cost storage media. Storage hierarchical division allows the platform to dynamically adjust the storage strategy according to actual needs, thereby adapting to changing data storage requirements. Building storage strategies based on data demand storage hierarchies can develop more efficient storage solutions. Multidimensional storage strategies can optimize data storage and management based on data access patterns, importance, and storage media characteristics to improve overall storage performance. By building multidimensional storage strategies, the platform can achieve higher flexibility and scalability to adapt to different data storage needs. This strategy can effectively cope with changes in data volume and fluctuations in user access patterns, ensuring that the system can maintain good performance under various loads. Through data relationship-driven analysis, storage demand analysis, storage hierarchy division, and storage strategy construction, the data storage and management capabilities of the smart education platform can be significantly improved. In-depth understanding of data relationships and requirements makes storage resource configuration more accurate and effectively reduces the waste of storage resources. Through hierarchical management and optimized storage strategies, the speed of data access and the overall performance of the system can be improved. Adapting to changing data needs, maintaining system efficiency and flexibility, and being able to cope with growing data volumes and complex user needs.
[0109] In an embodiment of the present invention, data relationship driven analysis is performed on multidimensional grid storage data and multidimensional grid indexes. A data relationship modeling tool, such as the networkx library in Python, is used to construct a data relationship model. First, data nodes and their relationships are extracted from multidimensional grid storage data and indexes. By constructing a data relationship graph, data nodes are regarded as nodes of the graph, and data relationships are regarded as edges. Graph algorithms in networkx, such as community detection and shortest path calculation, are used to analyze the relationship strength and data flow pattern between data nodes. The generated data relationship model reveals the connection and dependency between data stored in the data, providing a basis for subsequent storage demand analysis. Using the data relationship model, data access frequency, data size, and data update frequency are analyzed. Using a data analysis tool, such as the pandas library in Python, the storage requirements of each data node are calculated. This includes statistics on the storage capacity and access pattern of each type of data. By analyzing the attributes of each node and its edges in the data relationship model, the storage requirements and priorities of each data block are determined. The storage requirements of each data storage obtained include the size of the data, the access speed requirements, and the backup strategy, which provide a basis for the storage hierarchy division. According to the storage demand analysis results, the storage hierarchy of the data is determined. For example, using hierarchical storage management tools, such as the Hierarchical Storage Management (HSM) system, data can be divided into three levels: hot data, warm data, and cold data. The specific implementation includes using the pandas library to classify data requirements and allocate data to different storage media (such as solid-state drives, hard disk drives, or cloud storage) according to storage requirements. The generated data requirement storage hierarchy includes high-performance storage layers, medium-performance storage layers, and low-performance storage layers to optimize the storage and access efficiency of data. Using storage policy design tools, such as IBM SpectrumScale, corresponding storage policies are built according to the requirements of different storage levels. Specific steps include setting storage policies for each data level, such as tiered storage of hot and cold data, automatic data migration, and backup plans. By mapping the data requirement storage hierarchy to the storage policy, the storage location, access rights, and data redundancy strategy of the data are defined. The multidimensional storage strategy finally generated includes the distribution scheme of data in different storage levels, data migration rules, and backup strategies to ensure efficient data management and optimal use of storage resources.
[0110] Preferably, step S5 comprises the following steps:
[0111] Step S51: adjusting the storage mode of the hierarchical encrypted storage data based on the multi-dimensional storage strategy to obtain an adjusted storage database;
[0112] Step S52: performing data access monitoring on the adjustment storage database to generate monitoring access data;
[0113] Step S53: performing flow pattern recognition on the monitored access data to obtain each data flow pattern;
[0114] Step S54: Clean the bottom flow data of the adjusted storage database according to each data flow mode to execute the data intelligent storage method of the smart education platform.
[0115] The invention is beneficial in that the storage mode of hierarchical encrypted storage data is adjusted based on a multi-dimensional storage strategy, which can effectively optimize the utilization of storage resources. By adjusting the storage mode, data can be stored on the most suitable storage medium according to the access frequency and importance characteristics, thereby improving storage efficiency and system performance. Adjusting the storage database can reduce data access delay and improve reading speed. Especially when processing large-scale data, optimizing the storage structure can significantly improve the response speed and processing capacity of the system. Data access monitoring of the adjusted storage database can obtain access data in real time, which helps to track data usage, identify access patterns and discover potential problems. Real-time monitoring can timely discover system bottlenecks and abnormal access problems, and ensure the stability and reliability of the system. By monitoring access data, the platform can further optimize and adjust according to actual usage, such as adjusting storage strategies, upgrading storage devices, and improving overall data management efficiency. Traffic pattern recognition of monitored access data can reveal traffic characteristics and usage patterns of different data. This recognition can help the platform understand the peak period, trough period and its law of data access, thereby optimizing storage and resource allocation. Identifying traffic patterns helps predict and manage data traffic peaks, avoid system overload, and improve system stability and reliability. At the same time, the data storage strategy can be improved through the analysis of traffic patterns to ensure that high-traffic data can be accessed quickly and efficiently. The bottom traffic data cleaning of the adjusted storage database according to each data traffic pattern can remove unnecessary or outdated data, thereby optimizing the use of storage space. This cleaning can reduce storage overhead and improve the performance of the storage system. Cleaning traffic data can remove redundant, repeated or useless data and maintain the quality and accuracy of the data. This helps to improve the accuracy of subsequent data analysis and ensure the effectiveness of data intelligent storage methods. Through storage mode adjustment, data access monitoring, traffic pattern recognition and data cleaning, the data management and storage efficiency of the smart education platform can be significantly improved. By optimizing the storage mode and cleaning useless data, the utilization efficiency of storage resources can be improved. Real-time monitoring and traffic pattern recognition help optimize storage strategies, reduce system bottlenecks and improve performance. Intelligent storage decisions based on traffic patterns and access data make data management more forward-looking and efficient, and improve the overall data management capabilities of the platform.
[0116] In an embodiment of the present invention, the storage mode of hierarchical encrypted storage data is adjusted based on a multidimensional storage strategy. First, a storage management tool, such as IBM Spectrum Protect or Dell EMC Data Domain, is used to adjust according to the multidimensional storage strategy. The specific operation includes allocating data to different storage media according to the storage hierarchy (for example, storing high-frequency access data in a high-speed SSD, and low-frequency access data in a traditional HDD). The storage strategy is adjusted using the Ansible automated configuration tool to ensure that data is distributed in different storage hierarchies according to a predetermined strategy. The adjusted storage database finally generated optimizes the data storage location according to the storage strategy to improve access speed and storage efficiency. Data access monitoring is performed on the adjusted storage database. First, a data monitoring system, such as Splunk or Elastic Stack, is deployed to track database access activities in real time. The system is configured to capture the access frequency, access time, and access source of each data item. Detailed monitoring access data is generated through log recording and analysis. These data include the number of accesses, access patterns, and user behavior records, which are visualized using Grafana to help identify access trends and abnormal behaviors. Monitoring access data provides detailed insights into data access patterns and provides basic data for subsequent traffic pattern recognition. Traffic pattern recognition of monitoring access data. First, apply traffic analysis tools, such as Google Cloud BigQuery or AWS Kinesis, to analyze the collected monitoring access data. Use data mining techniques, such as cluster analysis or time series analysis, to identify data traffic patterns. The specific implementation includes using the scikit-learn library in Python to perform cluster analysis to identify different data access patterns (such as peak traffic and stable traffic). Classify the identified data traffic patterns to determine high-traffic and low-traffic time periods. The obtained traffic patterns help optimize storage strategies and cleaning processes. Adjust the storage database to clean the bottom traffic data according to each data traffic pattern. First, identify low-traffic data or data that has not been accessed for a long time, which is marked as "bottom traffic" in the traffic pattern analysis. Use data cleaning tools, such as Hive or Apache Spark in Apache Hadoop, to clean and archive these low-frequency data. Specific operations include removing no longer needed data from the storage database or migrating it to a storage medium with lower costs. Through SQL queries or ETL processes, delete or compress redundant data, free up storage space, and optimize data storage structure. The ultimate result of implementing data intelligent storage methods is to improve storage efficiency and reduce storage costs.
[0117] The present invention also provides a data storage system for a smart education platform, which is used to execute the data storage method for the smart education platform as described above. The data storage system for the smart education platform includes:
[0118] The data processing module is used to obtain the usage data of the smart education platform; eliminate redundant data of the usage data of the smart education platform to obtain deduplicated platform usage data; perform mean interpolation of missing values on the deduplicated platform usage data to generate standard platform usage data;
[0119] The data encryption module is used to store the standard platform usage data in slices to generate course hierarchical storage data; to encrypt the course hierarchical storage data independently to generate hierarchical encrypted storage data;
[0120] The management optimization module is used to perform multi-dimensional grid mapping on the hierarchical encrypted storage data to obtain multi-dimensional grid storage data; dynamically construct indexes on the multi-dimensional grid storage data to generate multi-dimensional grid indexes;
[0121] A strategy building module is used to build a storage strategy for multi-dimensional grid storage data and multi-dimensional grid indexes to obtain a multi-dimensional storage strategy;
[0122] The storage adjustment module is used to adjust the storage method of hierarchical encrypted storage data based on a multi-dimensional storage strategy, and introduce a data elimination mechanism to implement the data intelligent storage method of the smart education platform.
[0123] The benefit of the present invention lies in that standardized platform usage data is generated through redundant data elimination and missing value interpolation, providing a high-quality data foundation for subsequent storage, storing data in a hierarchical manner according to importance, and encrypting and protecting key data, thereby improving data security. This hierarchical storage design can effectively balance storage costs and access speeds, and is more in line with actual application requirements. Data is spatially stored through multidimensional grid mapping, and rich indexes are constructed to support complex query and analysis requirements. This multidimensional grid storage structure has strong scalability and can better adapt to the management of massive educational data. Dynamic indexing and data elimination mechanisms are introduced, and storage strategies can be adaptively adjusted according to actual access patterns. This intelligent storage method can make full use of storage resources, improve the performance and efficiency of data access, and integrate multiple technologies such as data preprocessing, hierarchical storage, multidimensional indexing, and intelligent scheduling, which have significant advantages in ensuring data security and improving access performance.
[0124] Therefore, the embodiments should be regarded as illustrative and non-restrictive from all points, and the scope of the present invention is limited by the appended claims rather than the above description, and it is therefore intended that all changes falling within the meaning and range of equivalent elements of the application documents are included in the present invention.
[0125] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.
Claims
1. A data storage method for a smart education platform. It is characterized in that The following steps are involved: Step S1: Obtaining usage data of the smart education platform; Eliminate redundant data from the smart education platform usage data to obtain deduplicated platform usage data; perform mean interpolation of missing values on the deduplicated platform usage data to generate standard platform usage data; Step S2: Slice and store the standard platform usage data to generate course hierarchical storage data; encrypt the course hierarchical storage data independently to generate hierarchical encrypted storage data; Step S3: Perform multi-dimensional grid mapping on the hierarchical encrypted storage data to obtain multi-dimensional grid storage data; Dynamically indexing the multidimensional grid storage data to generate a multidimensional grid index; wherein step S3 includes: Step S31: performing spatial data conversion on the hierarchical encrypted storage data to obtain hierarchical encrypted storage spatial data; Step S32: performing multi-dimensional grid mapping on the hierarchical encrypted storage space data to obtain multi-dimensional grid storage data; wherein performing multi-dimensional grid mapping on the hierarchical encrypted storage space data includes: Performing maximum principal component analysis on hierarchical encrypted storage space data to obtain the principal component with the maximum eigenvalue; The contribution rate of the principal component with the largest eigenvalue is cumulatively calculated to generate the optimal dimension of the storage space; Grid partitioning is performed on the hierarchical encrypted storage space data based on the optimal dimension of the storage space to obtain partitioned storage grid units; Performing matrix transformation processing on hierarchical encrypted storage space data to generate an encrypted storage mapping matrix; Performing data mapping on the encrypted storage mapping matrix according to the partitioned storage grid units to obtain multi-dimensional grid storage data; Step S33: dynamically reconstruct the multidimensional grid storage data to generate a multidimensional grid index; wherein, dynamically reconstructing the multidimensional grid storage data includes: Mapping filling curves of different dimensions to multi-dimensional grid storage data to obtain filled grid data; Repeat the above steps until all dimension combinations are filled with grid data; The multi-dimensional grid storage data is indexed and optimized based on the grid data filled in all dimensional combinations to obtain a filling optimized index; Extract usage traces of the smart education platform usage data to obtain platform usage traces; The platform usage trajectory and filling optimization index are integrated into the spatiotemporal filling index to obtain a multi-dimensional grid index; Step S4: constructing a storage strategy for the multidimensional grid storage data and the multidimensional grid index to obtain a multidimensional storage strategy; wherein step S4 includes: Step S41: performing data relationship driven analysis on the multidimensional grid storage data and the multidimensional grid index to generate a data relationship model; Step S42: Analyze the storage requirements of the data relationship model to obtain the storage requirements of each data; Step S43: dividing each data storage requirement into storage hierarchies to obtain a data requirement storage hierarchy; this includes using a hierarchical storage management tool to divide the data into three levels: hot data, warm data, and cold data; using a pandas library to classify the data requirements, and allocating the data to different storage media according to the storage requirements; the generated data requirement storage hierarchy includes a high-performance storage layer, a medium-performance storage layer, and a low-performance storage layer; Step S44: constructing a storage strategy for multidimensional grid storage data based on the data demand storage hierarchy to generate a multidimensional storage strategy; Step S5: Adjust the storage method of the hierarchical encrypted storage data based on the multi-dimensional storage strategy, and introduce a data elimination mechanism to implement the data intelligent storage method of the smart education platform.
2. The data storage method of the smart education platform according to claim 1, It is characterized in that Step S1 includes the following steps: Step S11: Acquire a smart education platform learning terminal; perform real-time monitoring on the smart education platform learning terminal to obtain usage data of the smart education platform; Step S12: Formatting the smart education platform usage data to generate consistent platform usage data; Step S13: removing duplicate data from the consistent platform usage data to obtain deduplicated platform usage data; Step S14: performing missing value pattern recognition on the deduplication platform usage data to obtain a data missing value pattern; Step S15: Perform missing value mean interpolation on the deduplication platform usage data according to the data missing value pattern to generate standard platform usage data.
3. The data storage method of the smart education platform according to claim 1, It is characterized in that Step S2 includes the following steps: Step S21: classify the standard platform usage data into specific courses to obtain classified course usage data; Step S22: Slice and store the standard platform usage data according to the classified course usage data to generate course hierarchical storage data; Step S23: extracting course features from the course hierarchical storage data to obtain hierarchical course feature data; Step S24: encrypting the course hierarchical storage data independently based on the hierarchical course characteristic data to generate hierarchical encrypted storage data.
4. The data storage method of the smart education platform according to claim 3, It is characterized in that Step S24 includes the following steps: Step S241: performing corresponding feature matching on the course grading storage data based on the graded course feature data to obtain matching course grading data; Step S242: performing feature vectorization mapping on the graded course feature data to generate a course feature vector; performing segmentation on the course feature vector to obtain course feature vector segments; Step S243: performing unique feature recognition on the course feature vector segment to obtain segment unique identification data; performing hash processing on the segment unique identification data to generate a feature segment hash value; Step S244: XOR-mixing the hash values of the feature fragments to obtain a mixed identifier; performing secure hash processing on the mixed identifier to generate a course encryption key; Step S245: encrypting the course graded storage data independently based on the course encryption key and the matching course graded data to generate initial graded encrypted storage data; Step S246: construct a unique identification code for the graded course feature data to obtain the graded course unique identification data; use the graded course unique identification data to perform surface annotation on the initial graded encrypted storage data to generate graded encrypted storage data.
5. The data storage method of the smart education platform according to claim 1, It is characterized in that Step S5 includes the following steps: Step S51: adjusting the storage mode of the hierarchical encrypted storage data based on the multi-dimensional storage strategy to obtain an adjusted storage database; Step S52: performing data access monitoring on the adjustment storage database to generate monitoring access data; Step S53: performing flow pattern recognition on the monitored access data to obtain each data flow pattern; Step S54: Clean the bottom flow data of the adjusted storage database according to each data flow mode to execute the data intelligent storage method of the smart education platform.
6. A data storage system for a smart education platform, It is characterized in that For executing the data storage method of the smart education platform according to claim 1, the data storage system of the smart education platform comprises: The data processing module is used to obtain the usage data of the smart education platform; eliminate redundant data of the usage data of the smart education platform to obtain deduplicated platform usage data; perform mean interpolation of missing values on the deduplicated platform usage data to generate standard platform usage data; The data encryption module is used to store the standard platform usage data in slices to generate course hierarchical storage data; to encrypt the course hierarchical storage data independently to generate hierarchical encrypted storage data; The management optimization module is used to perform multi-dimensional grid mapping on the hierarchical encrypted storage data to obtain multi-dimensional grid storage data; dynamically construct indexes on the multi-dimensional grid storage data to generate multi-dimensional grid indexes; A strategy building module is used to build a storage strategy for multi-dimensional grid storage data and multi-dimensional grid indexes to obtain a multi-dimensional storage strategy; The storage adjustment module is used to adjust the storage method of hierarchical encrypted storage data based on a multi-dimensional storage strategy, and introduce a data elimination mechanism to implement the data intelligent storage method of the smart education platform.
Citation Information
Patent Citations
Storage and management method for vector big data in water conservancy space
CN110059067A
Data storage method and system based on extensible multi-dimensional learning index structure
CN116303436A
Data storage method and system based on network teaching resources
CN117235187A
Internet-of-things data security sharing method and system
CN117792603A
Intelligent campus education data management system
CN118171954A
