Intelligent education portrait construction system fusing heterogeneous data cleaning and time series analysis

Abstract

The present application relates to the field of educational informatization and data analysis technology, in particular to a wisdom education portrait construction system fusing heterogeneous data cleaning and time series analysis; the system accesses education administration, online teaching, campus internet of things, one-card and teacher evaluation data, unifies multiple source marks to student mark SID, carries out dynamic cleaning in combination with field integrity, consistency and timeliness, extracts scores, comments and behavior track features, constructs education relationship database and processes data conflict, generates student whole cycle portrait and early warning, development label based on unified time axis, and updates quality threshold, weight and data source reliability according to intervention feedback; at the same time, intervention feedback record is used to update system configuration parameters; the present application improves the stability and explainability of education characteristic array.

Description

Technical Field

[0001] This invention relates to the field of educational informatization and data analysis technology, specifically to a smart education profile construction system that integrates heterogeneous data cleaning and time-series analysis. Background Technology

[0002] In existing intelligent education data processing, student-related information is usually scattered across multiple business systems such as the academic affairs system, online teaching platform, campus IoT, all-in-one card system, and teacher evaluation system. Different systems often have different identification methods, field naming, collection time sequence, and recording granularity for the same student. The resulting raw data includes both structured information such as grades and comments, as well as time-series information such as behavior logs and trajectory records. When using the above data for student profiling, early warning identification, and education analysis, it is usually necessary to first complete identity unification, quality assessment, missing data handling, feature extraction, and relationship association, and then generate student profile results based on these and push them to relevant business terminals.

[0003] However, existing methods often rely solely on a single primary key or simple rules for aggregation when processing multi-source heterogeneous data, making it difficult to simultaneously ensure field completeness, consistency, and timeliness. This results in some records still having poor usability, untraceable sources, and difficulty in determining conflicts even after cleaning. At the same time, there is a lack of a unified reliability transfer mechanism for filling in missing data, extracting features from text and behavioral sequences, and allocating weights to data from different sources. This makes subsequent relationship database construction and full-cycle profile generation susceptible to interference from the filled data, thereby affecting the accuracy and stability of early warning labels, development labels, and intervention feedback. Summary of the Invention

[0004] The purpose of this invention is to provide a smart education profile construction system that integrates heterogeneous data cleaning and time-series analysis, avoiding interference from data filling in on subsequent relationship database construction and full-cycle profile generation, and making it easier to balance the integrity, consistency and timeliness of multi-source heterogeneous data fields, thereby improving the accuracy and stability of early warning labels, development labels and intervention feedback.

[0005] The objective of this invention can be achieved through the following technical solutions:

[0006] The intelligent education profile construction system, which integrates heterogeneous data cleaning and time-series analysis, includes a data acquisition module, a data quality assessment and dynamic cleaning module, a multi-source feature extraction module, an education relationship database and data conflict handling module, a student full-cycle profile generation module, and a result distribution interface. The data acquisition module connects to multi-source business systems to collect data, performs identity mapping, and adds metadata to form a data record package.

[0007] The data quality assessment and dynamic cleaning module performs graded cleaning of data record packages based on field integrity, consistency, and timeliness, and outputs the cleaned data, cleaning rule information, and quality score.

[0008] The multi-source feature extraction module extracts numerical, textual, and sequence features based on grades, comments, and behavioral trajectories, and outputs an educational feature array and a feature description table.

[0009] The talent cultivation relationship database and data conflict handling module complete data association, optimization, and missing data supplementation based on a unified identifier and timeline, and maintain the reliability of the data source;

[0010] The student full-cycle profile generation module segments related data based on a unified timeline, generates profile tags according to the rules of the rule base, and outputs the results through the result distribution interface. At the same time, the intervention feedback records are used to update the system configuration parameters.

[0011] In some possible implementations, the data acquisition module performs identity normalization processing when extracting data from various data sources, mapping student ID, platform account, device number, and campus card number to a unified student identifier (SID), and attaching metadata to each data entry, including source system identifier, collection time, event occurrence time, field version number, and privacy classification label, to generate a data record package.

[0012] In some possible implementations, the data quality assessment and dynamic cleaning module comprehensively determines a quality score based on the assessment results of completeness, consistency, and timeliness, and executes a differentiated cleaning strategy accordingly:

[0013] When the quality score reaches the preset quality threshold, format unification and basic noise reduction are performed.

[0014] Based on the evaluation results of missing data, inconsistent content, or abnormal timeliness, corresponding actions are triggered to fill in missing data, normalize and map fields, or mark expired samples. At the same time, the weight of relevant data in feature extraction and data conflict handling is dynamically adjusted according to the marking results.

[0015] In some possible implementations, the data quality assessment and dynamic cleaning module includes a missing data imputation unit. When imputing missing data for behavioral records and online learning logs, constraints such as course sequence restrictions, historical stability of changes in the same semester, and restrictions on behavior within the same group are introduced as constraints.

[0016] After the filling is completed, the filling reliability and filling source type are generated and written into the cleaning rule information; for fields with reliability lower than the preset low reliability threshold, they are only used as auxiliary judgment basis in the establishment of relational database and early warning judgment.

[0017] In some possible implementations, the multi-source feature extraction module includes a numerical feature extraction unit, a text feature extraction unit, and a behavioral sequence feature extraction unit. When extracting features, it simultaneously receives field quality markers, missing markers, and imputation reliability to distinguish between four types of input states: true records, high-reliability imputation, low-reliability imputation, and those not participating in the construction of the main feature.

[0018] The input state is divided based on the comparison results between the filling reliability and the preset high reliability threshold and low reliability threshold.

[0019] In some possible implementations, the multi-source feature extraction module adopts a multi-data source weight allocation mechanism. By evaluating the similarity between the feature array of each data source and the corresponding education dimension benchmark array, and combining the upstream quality score, field filling reliability and data source reliability, the attention weight of each data source is comprehensively adjusted and determined, and finally a uniformly formatted education feature array and feature description table are output.

[0020] In some possible implementations, the education relationship database and the data conflict handling module establish a basic conceptual relationship library between entities and tags;

[0021] When performing data association, a unified student identifier, timeline, data record package, and educational feature array are comprehensively utilized to classify the established data associations into three source types: original record association, cleaned and filled association, and logical deduction association. The initial reliability of the original record association, cleaned and filled association, and logical deduction association is assigned to the three types in descending order, with the initial reliability of the original record association being higher than that of the cleaned and filled association, and the initial reliability of the cleaned and filled association being higher than that of the logical deduction association.

[0022] In some possible implementations, when dealing with conflicts between multiple records, the education relationship database and the data conflict handling module comprehensively evaluate the reliability of the data source, the freshness of time, the rule verification results, and the content matching degree to obtain the optimal conflict score.

[0023] When the difference in the preferred score between the preferred data and the second-best data reaches a preset conflict threshold, the preferred data is automatically adopted and the history of the replaced data is retained; when the difference does not reach the conflict threshold, the conflicting data is marked as pending manual verification and pushed out.

[0024] In some possible implementations, the student full-cycle profile generation module divides the student's historical data into multi-scale time segments according to a unified timeline, tracks the feature change trajectory of multiple educational dimensions, and identifies state changes to generate profile labels by evaluating the feature change magnitude, direction and label determination rules of adjacent time segments.

[0025] Simultaneously, the profiling results and suggested intervention measures are distributed to the corresponding business terminals, and the status improvement after intervention is collected to form a closed-loop feedback record.

[0026] The beneficial effects of this invention are:

[0027] 1. This invention maps student ID, platform account, device number, and campus card number into a unified student identifier, and adds source system identifier, collection time, event occurrence time, and privacy classification label to form a data record package. This enables unified access and unified key-value processing of multi-source heterogeneous data, and can effectively solve the problems of inconsistent identifiers, different collection cycles, and difficulty in aligning field semantics among different business systems.

[0028] 2. This invention achieves targeted processing of data of different quality by classifying and evaluating completeness, consistency and timeliness, and combining quality thresholds, anomaly type priorities and dynamic cleaning paths such as missing data filling, field normalization, unit conversion, content mapping and expiration marking. At the same time, it continuously writes back cleaning rule information, filling reliability and data source reliability, so that subsequent feature extraction and relation database construction can inherit data credibility and reduce the interference of abnormal data and filling data on the profiling results.

[0029] 3. This invention extracts numerical features, text features, and behavioral sequence features from grades, comments, and behavioral trajectories, respectively. It also combines field quality markers, missing data markers, imputation reliability, and multi-data source weight allocation mechanisms to differentiate between true records, high-reliability imputation, low-reliability imputation, and those not participating in the main feature construction, thereby improving the stability and interpretability of the educational feature array. Attached Figure Description

[0030] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0031] Figure 1 This is a structural block diagram of the intelligent education profile construction system that integrates heterogeneous data cleaning and time series analysis, according to an embodiment of the present invention. Detailed Implementation

[0032] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0033] Please see Figure 1The intelligent education profile construction system, which integrates heterogeneous data cleaning and time-series analysis, includes a data acquisition module, a data quality assessment and dynamic cleaning module, a multi-source feature extraction module, an education relationship database and data conflict handling module, a student full-cycle profile generation module, and a result distribution interface.

[0034] The data acquisition module connects to multiple source business systems to collect data, performs identity mapping, and adds metadata to form a data record package;

[0035] The data quality assessment and dynamic cleaning module performs graded cleaning of data record packages based on field integrity, consistency, and timeliness, and outputs the cleaned data, cleaning rule information, and quality score.

[0036] The multi-source feature extraction module extracts numerical, textual, and sequence features based on grades, comments, and behavioral trajectories, and outputs an educational feature array and a feature description table.

[0037] The talent cultivation relationship database and data conflict handling module complete data association, optimization, and missing data supplementation based on a unified identifier and timeline, and maintain the reliability of the data source;

[0038] The student full-cycle profile generation module segments the associated data based on a unified timeline, generates profile tags according to the rules of the rule base, and outputs the results through the result distribution interface. At the same time, the intervention feedback records are used to update the system configuration parameters.

[0039] When extracting data from various data sources, the data acquisition module performs identity normalization processing, mapping student ID, platform account, device number, and campus card number to a unified student identifier SID, and attaches metadata including source system identifier, collection time, event occurrence time, field version number, and privacy classification label to each data entry, generating a data record package;

[0040] The data quality assessment and dynamic cleaning module comprehensively determines a quality score based on the assessment results of completeness, consistency, and timeliness, and executes differentiated cleaning strategies accordingly:

[0041] When the quality score reaches the preset quality threshold, format unification and basic noise reduction are performed.

[0042] Based on the evaluation results of missing data, inconsistent content, or abnormal timeliness, corresponding actions are triggered to fill in missing data, normalize and map fields, or mark expired samples. At the same time, the weight of relevant data in feature extraction and data conflict handling is dynamically adjusted according to the marking results.

[0043] The system is deployed in the university's educational data processing platform to continuously receive student-related records generated by the academic affairs system, online teaching platform, campus IoT, campus card system, and teacher evaluation system, and complete data normalization, quality judgment, cleaning, profile generation, and result distribution within the same processing cycle.

[0044] Multiple business systems record the same student in different ways and with different collection periods. If they are directly aggregated and used, the subsequent profiling results are easily affected by inconsistent primary keys, incomparable recording times, and inconsistent field semantics. Therefore, the data collection module first forms a data record package with a unified structure, and then the subsequent modules process it according to the unified key value.

[0045] The data acquisition module accesses structured grade data, online learning logs, campus behavior trajectory data, and teacher comment texts through a preset extraction and adaptation process. During acquisition, identity normalization processing is first performed, mapping student ID, platform account, device number, and campus card number to a unified student identifier. The mapping relationship can be provided by the school's existing identity management relationship table or by existing account binding records;

[0046] After the mapping is completed, each record is appended with the source system identifier, collection time, event occurrence time, field version number and privacy classification label, and together with the original payload, it is encapsulated into a data record package and written to the temporary storage area. Subsequent modules use this data record package as the basic processing unit, thereby avoiding repeated primary key comparisons between modules.

[0047] After adopting the unified record structure described above, the data quality assessment and dynamic cleaning module reads the field information and time information from the data record package, calculates scores for completeness, consistency, and timeliness, and then sums and normalizes these scores according to preset weights to obtain the quality score. ;

[0048] The integrity score is directly determined by the ratio of the number of non-empty key fields in the record package to the expected total number; the consistency score is obtained by comparing whether the format of each field and the enumeration value fall into the pre-configured global standard dictionary, and dividing the number of fields that meet the standard by the total number of issued fields.

[0049] The timeliness score is divided into segments based on the time difference between the event occurrence time and the current system time. For example, a time difference of less than 3 days is scored as 1.0, 3 to 14 days is scored as 0.7, and more than 14 days is scored as 0.3.

[0050] The integrity score reflects whether key fields are complete; the consistency score reflects whether cross-system field formats, enumeration values, and units can be directly aligned; and the timeliness score reflects the availability between the time of the event and the current processing cycle.

[0051] The system integrity score (0.4), consistency score (0.3), and timeliness score (0.3) are weighted and summed. Since each score is between 0 and 1 and the weight sum is 1, the summation result is naturally limited to between 0 and 1, and thus serves as the normalized quality score. ;

[0052] After obtaining the quality evaluation results, the task orchestration process does not perform uniform processing. Instead, it determines whether the overall quality meets the standards and then selects the cleaning path based on the dominant anomaly type: when Greater than or equal to the quality threshold When the data is generally reliable, only format standardization and basic noise reduction are performed, with a preset quality threshold. The specific determination rules are as follows:

[0053] Extract the set of qualified data samples that have been manually reviewed and deemed to require no repair in the previous processing cycle. Calculate the statistical mean and standard deviation of the quality score for each sample in this set. Set the mean minus one standard deviation as the current quality threshold. ;

[0054] To avoid data loss during the system cold start phase, if the system is in the initial access phase or the total number of qualified data samples extracted is less than the preset minimum calculation base, thus failing to support the statistically bounded distribution, dynamic calculation will be suspended, and an initial static threshold will be temporarily used as the baseline. Until the sample pool accumulates to the target level;

[0055] And to prevent Due to interference from a large amount of sudden abnormal data, the data falls below the effective detection limit, and the system... Apply lower bound truncation protection to the dynamically updated results, that is, if the calculated results are... If the baseline defense threshold is lower than the preset threshold, then it will be forced to... This is limited to the defense threshold, thereby ensuring the basic convergence stability of the system cleaning model;

[0056] The preset baseline defense threshold is set based on the lowest acceptable quality score of the system's historical full set of valid data, for example, 0.6;

[0057] when Below the quality threshold When the system detects an error, it indicates that the data needs targeted repair. At this point, the system compares the proportions of various anomalies: when the missing proportion is the highest, it proceeds to the missing data filling path; when the content inconsistency proportion is the highest, it performs field normalization, unit conversion, and content mapping.

[0058] When the proportion of timeliness anomalies is highest, the corresponding record is marked as an expired sample, and this mark is synchronously written into the cleaning rule information. In the subsequent feature extraction and relationship processing stages, the reliability of the data source associated with this expired record is forcibly multiplied by a preset decay penalty constant. ,in This quantitatively reduces its usage weight;

[0059] When two or more types of anomalies have the highest proportions, the cleaning process is performed in the order of missing, content consistency, and timeliness to ensure that the cleaning process is deterministic.

[0060] The cleaning rule information and the cleaned data are stored in parallel. The cleaned data enters the downstream multi-source feature extraction module, while the cleaning rule information is written as supplementary metadata. The corresponding rule record area facilitates subsequent modules in determining the field source, cleaning path, and availability. If a source system does not provide any valid records in the current processing cycle, the data acquisition module only registers the missing status of that source system and does not generate virtual records.

[0061] The data quality assessment and dynamic cleaning module treats this source as a missing source when calculating integrity and triggers the missing path, instead of directly merging the record from the previous processing cycle of this source into the current cycle;

[0062] If a record fails to map a unified identifier, it will not enter the main process of profiling and will only be kept in the temporary storage area in a pending verification state. If a record lacks the event occurrence time and only retains the collection time, the timeliness calculation will use the collection time as a substitute basis, and the source of the time substitute will be marked in the cleaning rule information for downstream modules to distinguish and process.

[0063] After the above processing, the system forms a... Centered on a standardized data stream with unified time information and quality markers, the multi-source feature extraction module can directly read cleaned data based on grades, comments, and behavioral trajectories. The educational relationship database and data conflict resolution module can also follow a unified... The system establishes a cross-system connection with the timeline. The student full-cycle profile generation module segments the data based on the unified timeline and calls the rule base to generate early warning labels and development labels. The results are pushed to the teachers, counselors and related business terminals through the result distribution interface. Whether the early warning is valid and the effectiveness of the intervention are written back as feedback records, which are used to update the quality threshold, attention weight, conflict threshold and data source reliability.

[0064] In one embodiment, the data quality assessment and dynamic cleaning module includes a missing data imputation unit, which introduces course sequence restrictions, historical stability constraints of the same semester, and restrictions on behavior of the same group as constraints when imputing missing data for behavior records and online learning logs.

[0065] After the filling is completed, the filling reliability and filling source type are generated and written into the cleaning rule information; for fields with reliability lower than the preset low reliability threshold, they are only used as auxiliary judgment basis in the establishment of relational database and early warning judgment.

[0066] The multi-source feature extraction module includes a numerical feature extraction unit, a text feature extraction unit, and a behavioral sequence feature extraction unit. When extracting features, it simultaneously receives field quality markers, missing markers, and imputation reliability to distinguish between four types of input states: true records, high-reliability imputation, low-reliability imputation, and those not participating in the construction of the main feature.

[0067] The input state is divided based on the comparison results between the filling reliability and the preset high reliability threshold and low reliability threshold;

[0068] The multi-source feature extraction module adopts a multi-data source weight allocation mechanism. It evaluates the similarity between the feature array of each data source and the corresponding education dimension benchmark array, and combines the upstream quality score, field filling reliability and data source reliability to comprehensively adjust and determine the attention weight of each data source. Finally, it outputs a uniformly formatted education feature array and feature description table.

[0069] In the processing of student profiles in higher education institutions, behavioral records and online learning logs are most prone to continuous missing data. If only the values ​​retained from the previous processing cycle are used to fill in the missing data, or if the mean is used as a substitute, although the integrity of the fields can be restored, it will disrupt the sequential relationship of courses and the continuity of behavioral changes during the same period, which will further affect the subsequent feature extraction. Therefore, this implementation method introduces missing data with educational semantic constraints to fill in the missing data during the cleaning stage, and continues to transfer the reliability of the filled data to the feature extraction stage, rather than treating the filled data as equivalent to the original observations.

[0070] The missing data filling unit mainly processes the missing fields in behavior records and online learning logs. When filling, it reads the student's grade, major, course relationship, and historical records within the same period, and applies restrictions on the order of courses, the stability of historical changes in the same semester, and the behavior restrictions of the same group.

[0071] The same group refers to a group of students in the same grade and major; the course sequence restriction is used to constrain the occurrence time of relevant indicators of subsequent courses to be no earlier than the occurrence time of prerequisite courses;

[0072] The same semester historical change stationarity constraint is used to avoid abrupt changes in the supplementary results and the student's existing behavioral trajectory in the current semester; the same group behavior constraint is used to supplement the statistical regularity of the same group when individual history is insufficient;

[0073] In the specific logic of the imputation, when a certain behavioral data is missing, the mean value of that behavioral feature in the same group during this period is retrieved as the basic imputation candidate value.

[0074] If the time involved in the basic fill candidate value is earlier than the completion time of the student's prerequisite course, the fill time feature will be forcibly postponed to after the completion node of the prerequisite course. The specific time step of the postponement is determined based on the historical statistical average time interval between the prerequisite course and the current behavior of the same group.

[0075] If there is a lack of statistical references from the same group, a basic time stamp constant is set to 24 hours to avoid the absolute overlap between the time of completion and the time of prior repair, which would cause monotonicity conflicts in the downstream feature extraction sequence.

[0076] Apply the historical stability constraint of the same semester, that is, compare the candidate value with the moving average of the student's existing relevant behavioral indicators in the current semester. If the difference between the two exceeds the preset stable fluctuation range, the existing moving average is used as the final filling result; otherwise, the basic filling candidate value is used. The preset stable fluctuation range is set as the numerical range consisting of 15% above and below the moving average of the student's existing relevant behavioral indicators in the current semester.

[0077] After the imputation is completed, a corresponding imputation reliability is generated for each imputed field. The source type of the filler should be included, along with the field quality flag and missing flag, and written into the cleaning rule information; if If the value is below the reliability threshold for filling, the field can still be retained, but it will only be used as an auxiliary basis when building edges and making early warnings in relational databases, and will not be included in the main judgment process.

[0078] In the further data transformation stage, the multi-source feature extraction module does not directly perform indiscriminate encoding on the cleaned data, but instead synchronously reads field quality markers, missing markers, and... First, each field is divided into four categories: real records, high-reliability filling, low-reliability filling, and fields that do not participate in the construction of the main feature.

[0079] When a field is a raw, unfilled record, it is classified as a real record; when... Greater than or equal to When, it is classified as a high-reliability filler; when Greater than or equal to and less than When this occurs, it is classified as low-reliability filler;

[0080] when Less than When, it is marked as not participating in the construction of the main feature or only as a field to be verified, where, The high reliability threshold is used to determine high reliability filling. This is the low-reliability threshold used to determine low-reliability filling. and The reliability distribution of historical imputed samples can be predetermined. Specifically, the system extracts the reliability distribution of all imputed samples from the previous processing cycle. Construct a set, sort the values ​​in the set in descending order, and take the first 30% of the percentile values ​​as... Take the last 30% quantile value as And adjust the results on a rolling basis according to the accumulated results in subsequent processing cycles;

[0081] During the rolling update process, the system introduces an interval anti-degradation verification closed loop: if the variance of the reliability distribution of the samples filled in the current period is less than the preset distribution threshold, causing the absolute difference between the extracted top 30% quantile values ​​and the bottom 30% quantile values ​​to be less than the preset effective identification tolerance, resulting in an abnormal threshold convergence, the system suspends the update action of the current period and forces the use of the previous period's data. and It also outputs distribution degradation anomaly markers to the management panel to avoid the feature layer receiver making a judgment error of full field homogeneity; the preset effective identification tolerance is set to 0.1.

[0082] Through this state division, the numerical feature extraction unit, the text feature extraction unit, and the behavioral sequence feature extraction unit can respectively determine whether a field enters the main feature construction, and what weight to use after entering it;

[0083] Specifically, the performance data is extracted by the numerical feature extraction unit. The extraction rule is to map the original percentage scores or grades of each subject to standard grade points, and calculate the average grade points and the pass rate of core courses within the statistical period to form a structured numerical feature vector.

[0084] Teacher comments are extracted by the text feature extraction unit. The logical rule is to accurately match keywords with a pre-set educational sentiment dictionary, count the frequency of positive sentiment words and negative warning words in the comments, and divide the frequency of positive words by the total frequency of matched words to obtain the sentiment tendency feature between 0 and 1.

[0085] The historical data of the campus card trajectory is extracted by the behavior sequence feature extraction unit to extract behavioral habit features. The extraction process is based on the check-in location identifier and the difference between adjacent timestamps in the trajectory log, and the average daily stay time of the student in functional areas such as the library, study room and sports venue is classified and accumulated to generate a multi-dimensional behavior quantification array.

[0086] Meanwhile, the system stipulates that the input feature vector for the art literacy dimension is composed of a two-dimensional quantitative array consisting of the number of times students participate in on-campus arts and cultural activities and the percentage of credits for elective art courses; the input feature vector for the social practice dimension is composed of a two-dimensional quantitative array consisting of the cumulative hours of social volunteer service and the quantitative score of the level of awards in practice activities.

[0087] Fields marked as highly reliable fills can enter the corresponding feature cells to participate in the main calculations, but their contribution level is still subject to... Adjustments are made; for low-reliability imputation fields, they can be retained as weak inputs for explanatory output or auxiliary judgment; for fields that do not participate in the construction of main features, they are only registered in the area to be verified and are not included in the main feature array.

[0088] The above processing can prevent the imputed values ​​from losing their source differences after feature extraction, and prevent the portrait results from being interfered with by the imputed data.

[0089] In the education dimension construction phase, this implementation method adopts a multi-data source weight allocation mechanism to construct feature arrays around five education dimensions: moral character, academic level, physical and mental health, artistic literacy, and social practice.

[0090] The system calculates the normalized similarity between the feature arrays formed by each data source and the corresponding educational dimension benchmark array; the educational dimension benchmark array is a predefined full-dimensional reference feature vector, and the standard value of each dimension is composed of the historical average value of the corresponding quantitative features of outstanding students in the same group of the corresponding major, such as scholarship recipients over the years.

[0091] To ensure strict consistency between the matrix and the data structure and to avoid dimension mismatch errors caused by local features from different data sources, the system performs a feature space alignment operation before performing the dot product operation: based on the feature field identifiers contained in the feature array of the current data source, the entire educational dimension baseline array is spatially aligned; if the data source feature array has missing dimensions, the missing dimensions are padded with zeros before the operation to maintain the full rank consistency of the baseline feature space and ensure that the two arrays participating in the operation are in the same dimension vector space.

[0092] The normalized similarity calculation here uses cosine similarity logic. To avoid conflicts between different indicator dimensions and similarity deviations caused by negative values, all quantitative feature values ​​in the data source feature array have been uniformly mapped to the dimensionless non-negative interval of 0 to 1 using the min-max normalization method, so that the feature vector as a whole is located in the first quadrant of the high-dimensional space.

[0093] Take the dot product of the non-negative eigenvalues ​​of the feature array of the data source with the local reference subarray, and divide by the product of the moduli of the two arrays;

[0094] Before performing division operations, the system introduces a low-level matrix anomaly detection mechanism: if it is detected that the data source feature array degenerates into a zero vector due to the student's complete lack of current cycle behavior, resulting in the feature array having a modulus of 0, then the normalized similarity is directly determined to be 0, in order to avoid triggering a matrix operation division-by-zero error that could cause the system's computation link to be interrupted.

[0095] Under normal circumstances, the cosine similarity calculated in this way naturally falls between 0 and 1, which satisfies the value requirements of subsequent product scaling.

[0096] Similarity at the data source level and reliability of field-level imputation To address the hierarchical mismatch issue, the system pre-calculates all fields involved in the calculations for the current education dimension within the data source. The average value is used as the overall fill reliability of the data source;

[0097] Scaling is achieved by combining the upstream quality score of the data source, the overall imputation reliability, and the data source reliability with the normalized similarity.

[0098] For example, if a data source provides 3 fields, its The values ​​are 1.0, meaning the default values ​​for real records are 1.0, 0.8, and 0.6. The overall reliability of the fill is 0.8. If the similarity is 0.9, the upstream quality score is 0.9, and the data source reliability is 0.9, then the basic scaling result of the attention weight is 0.9×0.9×0.8×0.9.

[0099] Within the same education dimension, the scaled results from each data source are normalized again. This is done by dividing the scaled result of each data source by the sum of the scaled results of all data sources within that dimension, ensuring that the sum of attention weights equals 1. This results in a uniformly formatted array of education features. and Feature Description Table ;

[0100] The system records which data sources, field states, and event fragments constitute a certain dimension, facilitating the tracking of feature sources during subsequent relationship processing and manual review;

[0101] If a student's behavior log missing ratio exceeds the preset missing threshold in the current processing cycle, and the individual's historical samples are insufficient to support stable imputation, the missing data imputation unit will preferentially use available references under the same group constraints. If the constraints still cannot be met, the field will directly retain its missing status and be marked as not participating in the main feature construction, instead of forcibly generating low-confidence imputation values.

[0102] If the same field appears in both real records and supplementary records from different sources, the status identifier of the real records will be used first during feature extraction, and the supplementary records will only be used for supplementary explanations.

[0103] If a certain education dimension has only a single source available in the current period, then that source, after normalization, can independently form the attention weight for that dimension. The source is marked as insufficient for subsequent image interpretation;

[0104] After missing data filling, state partitioning, and dimensional weighting, the system obtains a unified educational feature array that can be directly entered into the relational database and profiling stages. Since the authenticity, reliability, and source weight of each field have been distinguished in the preliminary stage, subsequent modules can directly reference these results when establishing connections between students and courses, teachers, ability tags, and behavioral events, without having to return to the original logs to determine whether the fields are available.

[0105] In one implementation, the education relationship database and the data conflict handling module establish a basic conceptual relationship database between entities and tags;

[0106] When performing data association, a unified student identifier, timeline, data record package, and educational feature array are comprehensively utilized to classify the established data associations into three source types: original record association, cleaned and filled association, and logical deduction association. The initial reliability of the original record association, cleaned and filled association, and logical deduction association is assigned to the three types in descending order. The initial reliability of the original record association is higher than that of the cleaned and filled association, and the initial reliability of the cleaned and filled association is higher than that of the logical deduction association.

[0107] When dealing with conflicts in multi-source records, the education relationship database and data conflict handling module comprehensively evaluate the reliability of the data source, the freshness of time, the rule verification results, and the content matching degree to obtain the optimal conflict score.

[0108] When the difference in the conflict scores between the preferred and second-best data reaches a preset conflict threshold, the preferred data is automatically adopted and the history of the replaced data is retained; when the difference does not reach the conflict threshold, the conflicting data is marked as awaiting manual verification and pushed out.

[0109] The student full-cycle profile generation module divides student historical data into multi-scale time segments according to a unified timeline, tracks the feature change trajectory of multiple educational dimensions, and identifies status changes to generate profile labels by evaluating the feature change magnitude, direction and label determination rules of adjacent time segments.

[0110] Simultaneously, the profiling results and suggested intervention measures are distributed to the corresponding business terminals, and the status improvement after intervention is collected to form a closed-loop feedback record.

[0111] Once the cleaned data and educational feature array have been formed, it is difficult to express the continuous relationship between students, teachers, courses, behavioral events and ability labels if only basic data table association operations are performed, and it is also impossible to handle the situation where different systems have conflicting records for the same object.

[0112] Therefore, this implementation focuses the subsequent processing on the time-series profiling stage of the student lifecycle profile generation module, which is the database of educational relationships, the data conflict handling module, and the student lifecycle profile generation module, so that the results of the preceding processing can be transformed into relational data that can be used for early warning and intervention.

[0113] The database for nurturing relationships and the data conflict resolution module establish a basic concept relationship database, which includes five basic entities: students, teachers, courses, behavioral events, and ability tags, as well as tutoring relationships between students and teachers, elective relationships between students and courses, mastery relationships between students and ability tags, and attribution relationships between behavioral events and ability tags.

[0114] When establishing a relationship, it is not based solely on identical field names or a single primary key, but also on a unified student identifier. Unified timeline, data record package, and educational feature array Make a judgment;

[0115] In this way, explicit relationships in the original collection records, relationships that can be recovered after cleaning or filling, and relationships derived from existing associations based on preset rules can be uniformly written into the basic concept relationship library.

[0116] To improve the traceability of the logical derivation process, the process of deriving associations from existing associations based on preset rules adopts deterministic transitive logic:

[0117] If a student has already established an elective association with a certain course in the relational database, and simultaneously, there is an attribution association between that course and a specific competency tag, then the system continues this binary relational chain, automatically establishing a logical deduction mastery association between the student and that competency tag, and calculates an initial deduction score. This score is defined as the product of the reliability of the prior elective association and the reliability of the attribution association, multiplied by a preset attenuation factor. This preset attenuation factor is dynamically controlled by the time axis distance, and its determination logic is as follows:

[0118] Extract the absolute time difference between the establishment time of elective course association and the occurrence time of capability tag affiliation association event, and calculate the system attenuation constant and final attenuation factor:

[0119]

[0120]

[0121] in, This is the absolute time difference; The relational half-life constant defined for the system is set to 180 days. The system attenuation constant; The final decay factor is the absolute time difference. Relationship with half-life constant The unit is unified to day to ensure that the decay exponent is dimensionless; through this time-sequence constraint closed loop, the system automatically restricts the propagation of old failure relationships at the underlying topology.

[0122] Correspondingly, the system divides the established data associations into three source types: original record associations, cleansing and filling associations, and logical deduction associations, and assigns them an initial reliability of 0 to 1, with original record associations having a higher reliability than cleansing and filling associations, and cleansing and filling associations having a higher reliability than logical deduction associations.

[0123] To prevent data conflicts at the underlying numerical logic level, where high-scoring derivations outperform low-scoring original records, the system pre-divides the interval between 0 and 1 into three non-overlapping mapping intervals, such as the logical derivation interval. Cleaning and filling the area Original record range ;

[0124] The basic reliability or initial derivation score of various relationships calculated by the system is based on the theoretical domain boundary 0 and 1 as input extreme values. These values ​​are strictly mapped to the exclusive interval of their corresponding source type through the minimum-maximum linear scaling rule, and are used as the initial reliability of the final database write.

[0125] Through the above interval mapping process, subsequent conflict handling and profile determination can directly rely on numerical intervals to distinguish the strength of the relationship source, and mathematically guarantee the monotonically increasing attribute of the priority of the association source, without having to re-trace the edge construction process.

[0126] During the conflict resolution phase, if different systems have inconsistent records for the same educational object, such as inconsistent course-related status, behavioral event attribution, or tag values, the system will not immediately overwrite any of the records. Instead, it will first read the reliability, time freshness, rule verification results, and content matching degree of the corresponding data source.

[0127] Specifically, data source reliability is determined by directly referencing the 0-1 reputation value maintained by the preceding module; time freshness is assessed using a segmented logic, such as a score of 1.0 for conflicting data generated within one week, 0.8 for data generated within one month, and 0.5 for data generated after one month; content matching is based on the conflicting data and a table describing the students' historical characteristics. The deviation is quantified as follows: 1.0 is recorded as the deviation not exceeding the historical benchmark fluctuation range, 0.6 is recorded as the deviation reaching the first deviation level threshold, and 0.2 is recorded as the deviation reaching the second deviation level threshold.

[0128] The rule verification results are binarized based on whether they pass the routine business rationality verification, such as format completeness and deviation of non-critical value range. Passing is recorded as 1 and failing is recorded as 0, so as to avoid mixing serious verification abnormal data that affects the underlying operation of the system into the ordinary conflict comparison weighting process.

[0129] The above four indicators are all controlled within the range of 0 to 1. The system calculates the conflict optimization score by weighting the data source reliability (40%), time freshness (30%), content matching degree (20%), and rule verification result (10%), thereby integrating the attributes of different dimensions into a comparable comprehensive score.

[0130] If the score difference between the best and second-best data reaches or exceeds the conflict threshold If so, the preferred data will be used automatically, and the replaced data will be kept as a history record without physical deletion.

[0131] If the difference between the two scores is less than If the conflicting data is not automatically resolved, the system will mark it as pending manual verification and push it to the counselor's terminal through the result distribution interface.

[0132] The results after manual verification can be used as manually labeled samples for subsequent processing cycles. Continue iteratively to determine the specific iteration logic, which is as follows: initial system settings The value is 0.3. At the end of each processing cycle, the difference in scores is statistically analyzed as being less than 0.3. If the proportion of the sample set submitted for manual verification that still adopts the preferred data is greater than or equal to 80%, then... Reduce by 0.05 to minimize human intervention; if this percentage is less than 60%, then... Increase by 0.05; if the proportion is greater than or equal to 60% and less than 80%, then maintain. The reliability of the data source remains unchanged, and the data source reliability is adjusted based on the feedback results.

[0133] After the above relationship processing is adopted, the student full-cycle profile generation module performs segmented processing on the student's historical data along a unified time axis. The segmentation granularity includes semester segment, month segment and week segment. Feature arrays of five educational dimensions, namely moral character, academic level, physical and mental health, artistic literacy and social practice, can be extracted in each segment.

[0134] The system compares adjacent time segments, mainly judging the magnitude of change, the direction of change, and whether the corresponding label judgment thresholds are pre-stored in the rule base. When the conditions are met, at least one of the following labels can be generated: academic warning label, psychological concern label, and development potential label.

[0135] Taking the specific judgment logic tree for generating academic warning labels and development potential labels as an example: the system examines different physical meaning indicators contained in the feature array, such as the percentage score, online time in minutes, and the number of absences in frequency. By calling the dimensionless standardized values ​​of each indicator, the system calculates the difference between the dimensionless values ​​of each dimension of academic level in the current time period and the values ​​in the previous corresponding time period.

[0136] Based on the differences in the educational semantic direction of different indicators, such as rising grades representing a positive trend and increased absences representing a negative trend, the system performs reverse correction or positive retention on each difference value according to the semantic symbols preset in the rule base. The corrected dimensionless difference values ​​are weighted and summed to eliminate the interference of the original physical units and dimensional dimensions, and a unified dimensionless comprehensive change magnitude evaluation value is obtained.

[0137] Determine the overall direction of change. If the weighted difference sum is negative, it is marked as a downward direction; if it is positive, it is marked as an upward direction. When the direction of change is determined to be downward, and the absolute value of the change magnitude assessment value is greater than the academic decline warning threshold set in the rule base, an academic warning label is automatically triggered and generated. The academic decline warning threshold is preset to 0.2.

[0138] Conversely, if the change direction is a continuous segmentation showing an upward trend, and the total score of the current feature array, weighted by positive semantics, exceeds the excellent potential benchmark line set by the rule base, then a development potential label is generated. The excellent potential benchmark line is preset to 0.85.

[0139] If the overall assessment value of the physical and mental health dimension in the current time period decreases compared to the previous time period, and the absolute value of the decrease is greater than the psychological attention trigger threshold (preset to 0.3), or if the frequency of negative emotion feature words in this dimension exceeds 0.4, then a psychological attention tag will be automatically generated.

[0140] Once the profile results are generated, not only the tags themselves are retained, but also the triggering conditions and suggested intervention measures are also retained. The results are then distributed to the instructors, counselors, and the psychological counseling center through the results distribution interface, so that the corresponding roles can view and handle them according to their own permissions.

[0141] If a student only forms a partial association within the current processing cycle, resulting in insufficient segmented data for a certain educational dimension, the student full-cycle profile generation module will only perform label judgment on segments with continuous comparison conditions, and will not forcibly infer the status of missing segments.

[0142] If conflicting data is still pending manual verification, this part of the record will be included in the portrait judgment according to the preset basic reliability, so as to avoid interruption of the portrait process of the entire student due to waiting for manual processing.

[0143] If a logical derivation relationship is determined to be invalid by a new original record in a subsequent processing cycle, the system will overwrite the main relation state of the derivation relationship with the original record relationship, but the historical derivation path will still be retained.

[0144] If the result distribution interface fails to deliver to a terminal in the current period, the profile result and trigger condition record will continue to be stored in the distribution area and can be delivered repeatedly in subsequent periods without affecting the generation and writing back of the profile itself.

[0145] In the context of cultivating students in this university, the establishment of relationships, conflict optimization, and time-series profiling are continuously linked. The student relationship database and data conflict processing module provide a traceable data foundation and ensure that the data entering the profiling stage is comparable. The student full-cycle profiling generation module then identifies changes in student status based on time segments and rule thresholds.

[0146] After each business terminal reports whether the early warning is valid, whether intervention has been implemented, and the improvement status after intervention, the system generates an intervention feedback record and uses it to quantitatively update the data source reliability. When the feedback record shows that the early warning is valid and the intervention is effective, the core data source reliability update that triggers the early warning label is based on the following formula:

[0147]

[0148] in, Assuming the reliability of the current data source, To assess the reliability of the updated data source;

[0149] If an alert is marked as a false alarm, the system performs batch smoothing based on the following momentum decay formula:

[0150]

[0151] in, Assuming the reliability of the current data source, To ensure the reliability of the updated data source, Preset penalty time window The data source was marked as the total frequency of false alarms. This represents the maximum penalty for a single offense. This is the penalty slope coefficient; statistical analysis is performed within a preset penalty time window. Total frequency of false alarms Substituting into the formula, we can calculate the result. This updates the data source reliability, along with the quality threshold, attention weight, and conflict threshold, which are synchronously corrected according to a preset strategy, and serves as the corrected parameters for direct use in the next processing cycle.

Claims

1. A smart education profile construction system integrating heterogeneous data cleaning and time-series analysis, characterized in that: It includes a data acquisition module, a data quality assessment and dynamic cleaning module, a multi-source feature extraction module, a student education relationship database and data conflict handling module, a student full-cycle profile generation module, and a result distribution interface; The data acquisition module connects to multiple source business systems to collect data, performs identity mapping, and adds metadata to form a data record package; The data quality assessment and dynamic cleaning module performs graded cleaning of the data record package based on field integrity, consistency, and timeliness, and outputs the cleaned data, cleaning rule information, and quality score. The multi-source feature extraction module extracts numerical, textual, and sequence features based on grades, comments, and behavioral trajectories, and outputs an educational feature array and a feature description table. The talent cultivation relationship database and data conflict handling module complete data association, optimization, and missing data supplementation based on a unified identifier and timeline, and maintain the reliability of the data source; The student full-cycle profile generation module segments related data based on a unified timeline, generates profile tags according to the rules of the rule base, and outputs the results through the result distribution interface. At the same time, the intervention feedback records are used to update the system configuration parameters.

2. The intelligent education profile construction system integrating heterogeneous data cleaning and time-series analysis as described in claim 1, characterized in that, When extracting data from various data sources, the data acquisition module performs identity normalization processing, mapping student ID, platform account, device number, and campus card number to a unified student identifier (SID). It also adds metadata to each data entry, including the source system identifier, acquisition time, event occurrence time, field version number, and privacy classification label, to generate the data record package.

3. The intelligent education profile construction system integrating heterogeneous data cleaning and time-series analysis according to claim 1, characterized in that, The data quality assessment and dynamic cleaning module comprehensively determines the quality score based on the assessment results of completeness, consistency, and timeliness, and executes a differentiated cleaning strategy accordingly: When the quality score reaches the preset quality threshold, format unification and basic noise reduction are performed. Based on the evaluation results of missing data, inconsistent content, or abnormal timeliness, corresponding actions are triggered to fill in missing data, normalize and map fields, or mark expired samples. At the same time, the weight of relevant data in feature extraction and data conflict handling is dynamically adjusted according to the marking results.

4. The intelligent education profile construction system integrating heterogeneous data cleaning and time-series analysis according to claim 1, characterized in that, The data quality assessment and dynamic cleaning module includes a missing data imputation unit. When imputing missing data for behavior records and online learning logs, it introduces constraints such as course sequence restrictions, historical stability constraints of the same semester, and restrictions on behavior of the same group as constraints. After the filling is completed, the filling reliability and filling source type are generated and written into the cleaning rule information; For fields with reliability below the preset low reliability threshold, they are only used as auxiliary criteria in establishing relational databases and early warning determination.

5. The intelligent education profile construction system integrating heterogeneous data cleaning and time-series analysis according to claim 4, characterized in that, The multi-source feature extraction module includes a numerical feature extraction unit, a text feature extraction unit, and a behavioral sequence feature extraction unit. When extracting features, it simultaneously receives field quality markers, missing markers, and imputation reliability to distinguish between four types of input states: true records, high-reliability imputation, low-reliability imputation, and those not participating in the construction of the main feature. The input state is divided based on the comparison results of the filling reliability with the preset high reliability threshold and the low reliability threshold.

6. The intelligent education profile construction system integrating heterogeneous data cleaning and time-series analysis according to claim 5, characterized in that, The multi-source feature extraction module adopts a multi-data source weight allocation mechanism. It evaluates the similarity between the feature array of each data source and the corresponding education dimension benchmark array, and combines the upstream quality score, field filling reliability and data source reliability to comprehensively adjust and determine the attention weight of each data source. Finally, it outputs a uniformly formatted education feature array and feature description table.

7. The intelligent education profile construction system integrating heterogeneous data cleaning and time-series analysis according to claim 1, characterized in that, The educational relationship database and data conflict handling module establish a basic conceptual relationship library between entities and tags; When performing data association, a unified student identifier, timeline, data record package, and educational feature array are comprehensively utilized to classify the established data associations into three source types: original record association, cleaned and filled association, and logical deduction association. The original record association, the cleaned and filled association, and the logical deduction association are assigned an initial reliability that decreases sequentially. The initial reliability of the original record association is higher than that of the cleaned and filled association, and the initial reliability of the cleaned and filled association is higher than that of the logical deduction association.

8. The intelligent education profile construction system integrating heterogeneous data cleaning and time-series analysis according to claim 1, characterized in that, When dealing with conflicts in multi-source records, the educational relationship database and data conflict handling module comprehensively evaluate the reliability of the data source, the freshness of time, the rule verification results, and the content matching degree to obtain the optimal conflict score. When the difference in the preferred score between the preferred data and the second-best data reaches a preset conflict threshold, the preferred data is automatically adopted and the historical record of the replaced data is retained; when the difference does not reach the conflict threshold, the conflicting data is marked as pending manual verification and pushed out.

9. The intelligent education profile construction system integrating heterogeneous data cleaning and time-series analysis according to claim 1, characterized in that, The student full-cycle profile generation module performs multi-scale time segmentation on the student's historical data according to the unified time axis, tracks the feature change trajectory of multiple educational dimensions, and identifies state changes by evaluating the feature change amplitude, direction and label determination rules of adjacent time segments to generate profile labels. Simultaneously, the profiling results and suggested intervention measures are distributed to the corresponding business terminals, and the status improvement after intervention is collected to form a closed-loop feedback record.

Images