A student growth tracking data processing method based on activity flow

Abstract

The application discloses a student growth tracking data processing method based on activity flow, specifically comprising the following steps: training five activity flow description models, autonomously learning activity flow, testing learning activity flow, interactive learning activity flow, sports activity flow and popular science activity flow; collecting data generated by students in different regions; storing the collected data in an ODS (On-Demand System) and updating the data by using a log-based CDC (Change Data Capture); extracting activity flow information from unstructured data, text, images / videos and audio in the ODS by using a deep learning mode; polling and calling SQL Merge by using an encapsulated ETL Engine (Extract, Transform, Load) to send the updated data into a DW (Data Warehouse); performing feature-level data fusion and decision-level data fusion on the data in the DW according to the preset five activity flow description models; and the method can be used for student growth tracking analysis.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of educational informatization, and particularly relates to a student growth tracking data processing method based on activity flow. BACKGROUND

[0002] In the era of educational informatization, student growth data is generated at all times, and can directly reflect the growth activities of students. Meanwhile, the growth activity data of students may change or the types of growth activities may be different in different learning states or different growth stages, so the learning activity preferences of students in different learning states and different learning stages can be potentially mined. Therefore, student growth data has high application value in focusing on the growth process of students.

[0003] Student growth activity data usually comes from different system platforms, and the data structure types are also different. The current analysis of student growth activities is usually only directed to the learning data generated by students in a specific system platform, and although the learning state and learning preferences of students can be analyzed, the data is insufficient, the data generated by students in other system platforms is not analyzed, and the data cannot be effectively aggregated, shared and managed, so the analysis of student growth activities may cause certain deviation, and the data value is not fully realized.

[0004] Data aggregation and fusion is a hot topic in the current research of educational informatization. Firstly, student growth activity data comes from many sources and has various data structures, and heterogeneous data includes structured and unstructured data. Secondly, there are many data standards, and there is great inconsistency between application systems and database management systems relied on by data sources. Finally, there is platform and application diversity, lack of top-level design and planning, and no unified open sharing system is formed, which leads to repeated collection and construction of data resources. The main purpose of data aggregation and fusion is to break the physical island of educational information data and form a unified data center to provide raw materials for subsequent student growth activity data mining.

[0005] The invention patent application with the Chinese patent application number CN202011591495.1 discloses a multi-source heterogeneous data unified aggregation method and system, which mainly includes the following technical solutions: collecting multi-source data through a data collection module (text collection unit, picture collection unit, audio and video collection unit); grouping the data in the data collection module, eliminating heterogeneity, extracting corresponding data features according to the category, matching with the existing data feature group, obtaining data features reaching the set correlation, and obtaining a data pool; a data decision model fuses the data in the data pool to obtain a fusion result. Grouping the data in the data collection module to eliminate heterogeneity may cause the loss of part of the data information of the heterogeneous data.

[0006] The invention patent application with the Chinese patent application number CN202110601250.0 discloses a student growth data analysis method, the main technical solution of which is: collecting corresponding data information in all aspects of morality, intelligence and body according to the information of the student's personal basic archives; the management personnel extracts and arranges high-quality data analysis schemes from the student data information; and the management personnel feeds back the student growth analysis results according to the data analysis schemes. The student growth data information is manually extracted and arranged to obtain the analysis scheme and the analysis result, which is time-consuming and laborious. SUMMARY

[0007] Therefore, in order to solve at least one of the above defects or improvement needs of the prior art, the present application proposes a student growth tracking data processing method based on activity flow, characterized in that the method comprises the following steps:

[0008] Collecting data of students in multiple activity scenarios;

[0009] Performing processing on the data and storing them in a basic data platform, the processing including dividing the data into structured data and unstructured data;

[0010] Extracting the stored data on the basic data platform and performing feature value extraction and model training to output an activity flow data analysis conclusion;

[0011] The model is a five-big-activity-flow-description model, including a student self-study tracking model, a test learning tracking model, an interactive learning tracking model, a sports activity tracking model, and a science popularization activity tracking model.

[0012] Further, the method further comprises an updating step of performing updating on the stored data: through the encapsulated ETLEngine polling calling SQL Merge, the updated data is sent into the data warehouse of the basic data platform.

[0013] Further, characterized in that the unstructured data includes text, image / video, and audio; and the processing models of different types of unstructured data are different.

[0014] Further, the processing method of the text data is: the processing of the text information is: performing named entity recognition based on BiLSTM bidirectional long short-term memory and CRF conditional random field, and performing entity information extraction using the trained model.

[0015] Further, the processing of the picture / video data is: based on the text recognition technology of CRNN+CTC, the target key information is extracted.

[0016] Further, the processing of the audio data is converting the audio into text, and processing the unstructured text data.

[0017] Further, the loss function of the model in the processing of the text information is:

[0018]

[0019] Wherein, Sreal is the score of the real path, Sj represents the score of the jth path, the number of labels is k, the length of the text is n, and there will be N=K n paths.

[0020] Further, the loss function of the model in the processing of the picture / video data is:

[0021]

[0022] loss=-ln(Π (x，z)∈S P(l|x))=-∑ (x，z)∈S ln P(l|x); wherein B -1 (l) represents all path sets of the text l after being transformed by the sequence-to-sequence mapping function B, and Pi is one of the path sets, the probability of each path is the product of the scores of the corresponding characters in each time step, x and z are sample variables.

[0023] The application further discloses a computer device, including a memory and a processor, the memory stores a computer program, characterized in that the processor executes the computer program and realizes the steps of the above method.

[0024] The application further discloses a computer readable storage medium, which stores a computer program, characterized in that the computer program is executed by a processor to realize the steps of the above method.

[0025] Overall, compared with the prior art, the above technical solutions of the present application can achieve the following beneficial effects:

[0026] (1) The present application firstly constructs the establishment and management of the architecture execution database based on the self-learning activity flow, the testing learning activity flow, the interactive learning activity flow, the sports activity flow and the popular science activity flow of the five pre-designed activity flow description models, and the five types of activity flow are enough to include most of the growth activities in the growth process of students.

[0027] (2) The second aspect, on the premise of establishing the above-mentioned model training architecture, the application proposes to extract activity stream information from unstructured data, text, images / videos, and audio by corresponding deep learning methods, thereby avoiding the loss of key information in unstructured data and the inaccuracy of subsequent tracking and analysis of students' growth activities.

[0028] (3) The present application is based on nine system platforms, namely, the national primary and secondary school wisdom platform, the education management information center platform, the China basic education quality monitoring platform, the student growth file reporting system, the China science and technology platform, the questionnaire system, the comprehensive quality evaluation activity system, the regional comprehensive quality evaluation platform, and the data reporting system for collecting students' growth data.

[0029] (4) The present application proposes to divide the collected data into structured data and unstructured data, store them in the basic data platform (ODS), and update the data through the change data capture (CDC). The change data capture (CDC) based on logs is realized through triggers or logs. When the source data table changes, the operation is recorded through the trigger or log attached to the table, and the downstream replays the database change record through the database bottom protocol, thereby realizing synchronization, high real-time performance, and accurate capture of various changes in the upstream.

[0030] (5) The present application proposes to extract activity stream related information from unstructured data, text, images / videos, and audio by corresponding methods according to the pre-designed five activity stream models. For unstructured text data, the present application proposes to use the named entity recognition method based on deep learning to extract information. For unstructured image and video data, the present application proposes to use the character recognition method based on CRNN+CTC to extract information. For unstructured audio data, the present application proposes to convert the video or audio into text by using tools such as Sonix, and then extract information based on the entity naming recognition of deep learning.

[0031] (6) The present application proposes to perform feature-level data fusion and decision-level data fusion according to the pre-designed five student growth activity stream description models in the data warehouse (DW). Feature-level data fusion is mainly based on hierarchical semantics, and decision-level data fusion is mainly based on context semantics. Finally, it can be used for tracking and analyzing students' growth activities, autonomous learning activities, test learning activities, interactive learning activities, sports activities, and popular science activities. BRIEF DESCRIPTION OF DRAWINGS

[0032] Figure 1 is the activity stream-based student growth tracking data aggregation and fusion method flowchart realized according to the present application.

[0033] Figure 2 is a data feature level and decision level data fusion structure diagram realized according to the present application. DETAILED DESCRIPTION

[0034] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and should not be used to limit the present application. In addition, the technical features involved in the various embodiments of the present application described below can be combined with each other as long as they do not conflict with each other.

[0035] As shown in Figure 1 The present application first discloses a student growth tracking data processing method based on activity flow, and one specific embodiment thereof includes the following steps:

[0036] 1) Nine system platforms are used as data sources to collect data generated by students in schools, families, communities and popular science sites, etc.

[0037] 2) The collected data is divided into structured data and unstructured data, which is stored in the basic data platform (ODS) and updated through the change data capture (CDC);

[0038] 3) In the basic data platform (ODS), according to the preset five activity flow description models, the unstructured data, text, image / video and audio are extracted through deep learning method to obtain activity flow information;

[0039] 4) The updated data is sent to the data warehouse (DW) through the polling call SQL Merge of the encapsulated ETL Engine;

[0040] 5) The data stored in the data warehouse is fused at the feature level and the decision level according to the preset five activity flow description models;

[0041] 6) Student growth activity tracking analysis, student self-learning tracking, test learning tracking, interactive learning tracking, sports activity tracking and popular science activity tracking are performed.

[0042] The present application collects relevant student growth data from nine system platforms. Unstructured data, text, images / videos, and audio are processed through corresponding methods, such as deep learning-based named entity recognition, CRNN+CTC-based character recognition, and video or audio conversion to text through tools such as Sonix, and then based on deep learning-based entity naming recognition, information extraction is performed according to the pre-designed five activity flow description models, and stored in the data warehouse (DW). The data in the data warehouse (DW) is fused at the feature level and at the decision level according to the pre-designed five activity flow description models, forming five types of student growth activity descriptions.

[0043] The pre-designed five activity flow description models, autonomous learning activity flow, test learning activity flow, interactive learning activity flow, sports activity flow, and popular science activity flow; the data generated by students in schools, families, communities, and popular science sites is collected through nine system platforms; student growth data is collected from various platforms and stored in the basic data platform (ODS), and updated through change data capture (CDC); unstructured data, text, images / videos, and audio in the basic data platform (ODS) are processed through deep learning-based named entity recognition, CRNN-based character recognition, and video or audio conversion to text through tools such as Sonix, and then based on deep learning-based entity naming recognition, activity flow information extraction is performed; the updated data is sent to the data warehouse (DW) through the polling call SQL Merge of the encapsulated ETL Engine; the data in the data warehouse (DW) is fused at the feature level and at the decision level according to the five activity flow description models; finally, student growth activity tracking analysis, autonomous learning tracking, test learning tracking, interactive learning tracking, sports activity tracking, and popular science activity tracking can be performed.

[0044] As shown in Tables 1-5 below, the specific factors of the attribute values of the five learning models in the embodiments of the present application are as follows,

[0045] Table 1 Autonomous learning tracking activity flow description sentence

[0046] Autonomous learning activity:

[0047] Attribute Attribute Value Example Actor (Actor_id, Actor_name) Zhang San, whose id number is the ID number Actor_grade Grade 7 Start_Time 2022-7-20 16:35 TimeSpan 30 minutes Space Laboratory, Library, Dormitory, Home LearningStrategy Read (document), Listen (audio), Watch (video), Do (problem) Affairs (LeanringTopic, LearningContent) Mathematical knowledge about trigonometric functions LeanringType Consolidate old knowledge / learn new knowledge

[0048] Table 2 Test learning tracking activity flow description sentence

[0049]

[0050] Attribute Attribute Value Example Actor (Actor_id, Actor_name) Zhang San, whose id number is the ID number Actor_grade Grade 7 Start_Time 2022-7-20 16:35 TimeSpan 30 minutes Space Laboratory, Library, Dormitory, Home Test_Affairs (TestTopic, TestContent) Weekly test / unit test in mathematics / English / history / Chinese Test_Start_Time 2022-7-20 16:00 Test_Knowledge_Num 15 Test_difficulty_degree Easy / medium / difficult Test_Score 82 ls_Correction (True: Correction_Time | False) Yes (2022-7-20 17:00) / No

[0051] Table 3 interactive learning tracking activity stream description sentence

[0052]

[0053]

[0054] Table 4 sports learning tracking activity stream description sentence

[0055] Sports activities:

[0056] Attribute Attribute Value Example Actor (Actor_id, Actor_name) Zhang San, whose id number is the ID number Actor_grade Grade 7 Start_Time 2022-7-20 16:35 TimeSpan 30 minutes Sports_Topic Jogging / Cycling / Yoga / Jumping Rope Sports_MaxTime 1 hour Sports_Consume 203 calories Sports_HeartRate 70 Sports_State Tired / General / Good

[0057] Table 5 science popularization activity tracking stream description sentence

[0058] Science popularization activities:

[0059] Attribute Attribute Value Example Actor (Actor_id, Actor_name) Zhang San, whose id number is the ID number Actor_grade Grade 7 Start_Time 2022-7-20 16:35 TimeSpan 30 minutes Space Science and Technology Museum / Museum PopularScience_Topic Principle of magnetic levitation ls_Submit (True: Submit_Time | False) Yes (2022-7-20 17:00) / No Level_Interest Not interested / General / Very interested

[0060] Although the present application is based on the nine system platforms, the national primary and secondary school wisdom platform, the education management information center platform, the China basic education quality monitoring platform, the student growth file reporting system, the China science and technology platform, the questionnaire system, the comprehensive quality evaluation activity system, the regional comprehensive quality evaluation platform and the data reporting system, etc. Platform related data information collected, but has scalability, can be extended to other platforms.

[0061] In the embodiment of the present application, in order to avoid the invasion of the source system, and cause the influence on the performance of the source system, the present technology adopts the change data capture (CDC) based on the log, which is realized through the trigger (Trigger) or the log. When the source data table changes, the operation will be recorded through the trigger or the log attached to the table, and the downstream will replay the database change record through the protocol of the database bottom layer, so as to realize the synchronization, the real-time is high, and the various changes of the upstream can be accurately captured.

[0062] Unstructured data mainly includes text, image / video, audio, etc., which contains a large amount of information about student learning. In order to extract the learning information contained in the learner learning, the present application adopts the deep learning method for text, image / video, audio, based on the named entity recognition of deep learning, based on the CRNN character recognition, and through the Sonix tool to convert video or audio into text, and then based on the entity named recognition of deep learning, the activity stream information is extracted, and the related activity stream data information is extracted.

[0063] For unstructured text data, the application adopts a named entity recognition based on deep learning. Named entity recognition refers to recognizing entities with specific meanings in text, mainly including names, time, place and other text. It is mainly based on BiLSTM (Bidirectional LSTM) bidirectional long short-term memory and CRF (Conditional Random Fields) conditional random field to perform named entity recognition, and finally uses the trained model to extract related entity information. BiLSTM bidirectional long short-term memory saves long dependencies by means of the structure of the storage unit, and adjusts the influence of the previous state on the current storage unit state through the input gate, output gate and forget gate. CRF conditional random field focuses on sentence level in sequence labeling, and labels according to feature templates, and obtains the optimal solution by Viterbi algorithm decoding to complete the labeling task. Finally, the key information of unstructured text data is extracted.

[0064] For unstructured picture or video data, the application adopts a text recognition technology based on CRNN+CTC to extract the key information needed. CRNN (Convolutional Recurrent Neural Network) is an end-to-end recognition of text sequence of indefinite length, without cutting single characters first, but converting text recognition into a sequence learning problem of time-dependent sequence, that is, image-based sequence recognition, that is, CNN extracts features of the picture, and then uses RNN to predict the sequence, and finally obtains the final result through a CTC translation layer to extract the key information of unstructured picture data.

[0065] For unstructured audio data, the application first converts video or audio into text through tools such as Sonix, and then processes unstructured text data to extract the key information of unstructured video or audio data.

[0066] As shown in Figure 2 The student growth tracking data processing based on activity flow realized according to the application performs feature-level data fusion and decision-level data fusion on the data stored in the data warehouse (DW) according to the pre-designed five activity flow description models in the following manner:

[0067] Five activity flow description models are designed in advance, mainly including autonomous learning activity flow, test learning activity flow, interactive learning activity flow, sports activity flow and science popularization activity flow. The application designs description sentences about the five activity flows, and each description sentence has a single attribute or a composite attribute for specific content description. Therefore, feature-level data fusion and decision-level data fusion are adopted to fuse and combine the structured data, and generate the corresponding activity flow description sentences. The feature-level data fusion can give the required characteristic information for decision-making to the greatest extent while ensuring instantaneity, and the fusion result has high precision. The decision-level data fusion has high fault tolerance and timeliness, and the data fusion is carried out based on the decision requirements of the five activity flow description models designed in advance, and finally can be used for student growth tracking analysis.

[0068] The extraction of activity flow information in unstructured data, text, images / video and audio and data fusion in the data warehouse (DW) used by the application include the following links:

[0069] For unstructured text data, the application proposes a model trained based on BiLSTM (Bidirectional LSTM) bidirectional long short-term memory and CRF (Conditional Random Fields) conditional random field for named entity recognition, to extract key information from unstructured text data. The model mainly uses BiLSTM (bidirectional long short-term memory) to generate emission scores (label vectors), and uses CRF to decode the optimal label path based on the emission scores. CRF (conditional random field) models in a global way, selects the path with the best effect from all possible paths, and uses Viterbi (Viterbi algorithm) to decode the path with the highest score from all paths. The loss function of CRF during model training is as follows, where Sreal is the score of the real path, i.e. the sequence expected to be decoded by CRF is this path, Sj represents the score of the jth path, the number of labels is k, the length of the text is n, and there are N = K n paths.

[0070]

[0071] For unstructured image data, the application proposes to use the CRNN+CTC-based text recognition technology for key information extraction. CRNN includes VGG11 and RNN two parts, and VGG11 is used for feature extraction, and then a double-layer BiLSTM is used to extract sequence information. The CTC algorithm is mainly used to solve the situation that the labels and predicted values in the neural network cannot be aligned, and the label distribution obtained from the cycle layer is converted into the final recognition result through de-duplication and integration. The total probability of mapping to the label text l and the loss function of CTC are as follows, where B-1 (I) represents all the path sets of the text I after the sequence-to-sequence mapping function B transformation, and Π is one of the path sets, the probability of each path is the product of the scores of the corresponding characters in each time step, x and z are sample variables.

[0072]

[0073] loss = -ln(Π (x，z)∈S P(l|x)) = -∑ (x，z)∈S ln P(l|x)

[0074] For unstructured video or audio data, the application proposes to first convert the video or audio into text by tools such as Sonix, and then process the unstructured text data to extract the key information of the unstructured video or audio data.

[0075] The content described in the specification is only an example of the application, and those skilled in the art can make various modifications or supplements to the described specific embodiments or replace them with similar ways, as long as they do not deviate from the content of the specification or exceed the scope defined by the claims.

Claims

1. A method for processing student growth tracking data based on activity flow, characterized in that, The above method includes the following steps: Collect student data across multiple activity scenarios; The data is processed and stored in a basic data platform. The processing includes dividing the data into structured data and unstructured data. The unstructured data includes text, images / videos, and audio. Different types of unstructured data have different processing models. On the aforementioned basic data platform, the stored data is extracted, and feature value extraction and model training are performed to output conclusions based on activity flow data analysis. The model is a five-activity flow description model, including a student self-directed learning tracking model, a test learning tracking model, an interactive learning tracking model, a sports activity tracking model, and a science popularization activity tracking model. The text processing involves: performing named entity recognition based on BiLSTM bidirectional long short-term memory and CRF conditional random field, and extracting entity information using the trained model; The loss function of the model in the text processing is: ； Where Sreal is the score of the actual path, Sj represents the score of the j-th path, the number of tags is k, and the text length is n, then N=K. n Path; The image / video processing involves extracting key target information using CRNN+CTC text recognition technology. The loss function of the model in the image / video processing is: ; of which B -1 ( l The transformation by function B, representing a sequence-to-sequence mapping, results in text. l The set of all paths, where Π is one of the paths, and the probability of each path is the product of the scores of the corresponding characters at each time step, where x and z are sample variables. The data to be stored is updated by: using the encapsulated ETL Engine to poll and call SQL Merge, and sending the updated data to the data warehouse of the underlying data platform.

2. The student growth tracking data processing method based on activity flow according to claim 1, characterized in that, The audio processing involves converting the audio into text, which is then processed as unstructured text data.

3. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method of claim 1 or 2.

4. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method of claim 1 or 2.

Images