Mobile learning context prediction method based on combined neural network

Abstract

A mobile learning context prediction method based on a combined neural network comprises the following steps: preprocessing data, data enhancement, processing data, constructing a combined neural network, training the combined neural network, combined neural network testing and inverse normalization, and combined neural network performance evaluation. Since the combined neural network is adopted, the network is applied to the mobile learning context prediction method, meaningful mobile learning context time sequence information is extracted, local features affecting the mobile learning context are effectively selected, and time and related features having greater influence are selected from the time dimension, so that the existing mobile learning context data can be more favorably used for prediction, and the prediction performance is better. The present application has the advantages of easy network construction, good network performance, wide application range, etc., and can be used in application scenarios such as course learning resource recommendation and WiFi data flow area prediction.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computer, and particularly relates to a mobile learning situation prediction method. BACKGROUND

[0002] Mobile learning (M-Learning) is a new mode of realizing learners' learning anytime and anywhere, is an important part of smart learning and smart education, and has become a front research direction in the field of educational information science. Mobile researches carried out in the field of education at home and abroad can be roughly divided into: analysis and identification of learners' mobile behavior, mining of learners' mobile behavior patterns, learners' mobile situation perception, and prediction of learners' mobile learning locations. The learning effect and learning efficiency of learners will be affected by the situation. Under different learning situations, learners will choose appropriate learning content and learning form according to the individualized needs of learning in a specific location and at a specific time. The research on mobile learning based on situation perception makes mobile learning have situationality, and its biggest advantage is to identify and acquire the mobile learning situation of learners, track the learning needs of learners, so as to better provide personalized learning services for learners and improve the learning quality and learning effect of learners.

[0003] By studying the different mobile learning contexts of learners and other related data, analyzing and mining the internal relations of these data have broad research prospects. So far, the mobile learning context prediction problem in learning context research has not been paid attention to by scholars. According to the time sequence information of the mobile learning context and the related sensor information, the next mobile learning context of the learner and the learner's staying time and mobile learning context switching time in the next mobile learning context can be predicted. According to the prediction results, appropriate forms and appropriate time length of course resources can be provided for learners. For example, if the next mobile learning context is in a sports field, audio learning materials with similar staying time can be recommended to the learner. In addition, operators can also deploy WiFi data flow areas according to the prediction results. Therefore, the mobile learning context prediction problem has great and far-reaching significance. The research on this problem is still challenging. The traditional time-dependent prediction method depends on artificial design features, and the calculation process of this method is relatively complex and time-consuming, which cannot predict the next mobile learning context of the learner and the learner's staying time and mobile learning context switching time in the next mobile learning context. The research method based on deep learning can solve this problem. This method designs a suitable neural network structure, which is used as network input after data preprocessing for training. In order to train a network with high accuracy and make the network have strong generalization ability, a large amount of data is usually needed for training. In fact, it is difficult to obtain enough data, which is also the limitation of this method. Data augmentation can solve this limitation. As an effective method to expand the amount of data, it has been verified in many experimental and research results. This technology transforms the data, and the transformed data is used as a supplement to the original data, thereby improving the accuracy and generalization ability of the network. Therefore, the data augmentation technology is also integrated in the research of the mobile learning context prediction problem. SUMMARY

[0004] The technical problem solved by the present application is to overcome the shortcomings of the prior art and provide a mobile learning context prediction method based on a combined neural network, which has simple network construction, good network performance and wide application range.

[0005] The technical solution adopted to solve the above technical problems is composed of the following steps:

[0006] (1) Preprocess data

[0007] The data set is derived from the mobile learning context data set R obtained by 50 learners using a data collection APP:

[0008] R={id,E,st,f,l,a,h,dv,w,S}

[0009] In the formula, id is the name of the learner, E is the mobile learning context, st is the arrival time of the learner, f is the floor where the learner is located, l is the longitude where the learner is located, a is the latitude where the learner is located, h is the altitude where the learner is located, dv is the model of the collection device, the weather type, S is the sensor; E takes the value [1, q], q takes the value of a limited positive integer; f takes the value [-1, 18], l, a, h take the value of a limited positive real number, w selects sunny, cloudy, cloudy, light rain, moderate rain, heavy rain; S is a sensor, S∈{s1, s1,..., s 73}, wherein s1 is a light sensor, s2 is an acceleration sensor, s3 is a direction sensor, and s4 is a step sensor.

[0010] The mobile learning context data set R is preprocessed as follows:

[0011] 1) Learner arrival time expansion

[0012] The learner arrival time st is expanded by the learner stay duration tp, and st i and st j are expanded by the learner stay duration tp i :

[0013] tp i = st j -st i (1)

[0014] In formula (1), st j represents the learner arrival time st corresponding to the jth sample, st i represents the learner arrival time st corresponding to the ith sample, i and j take the value of a limited positive integer at least 2, and i < j.

[0015] The learner arrival time st is expanded by the learner stay duration ti, and st k and st j are expanded by the learner stay duration ti k :

[0016] ti k = st k -st j (2)

[0017] In formula (2), st k represents the learner arrival time st corresponding to the kth sample, k takes the value of a limited positive integer at least 2, and j < k.

[0018] The data set R' after the arrival time expansion in the mobile learning context data set R is:

[0019] R′={id,E,st,f,l,a,h,dv,w,S,tp,ti};

[0020] 2) Select features

[0021] Using Pearson correlation coefficients, among the correlation coefficients of learner name (id), learner arrival time (st), learner floor (f), learner longitude (l), learner latitude (a), learner altitude (h), data acquisition device model (dv), weather type (w), sensor S, and mobile learning context (E), learner dwell time (tp), and mobile learning context switching time (ti) in the expanded dataset R′, features with an absolute value greater than 0.1 are selected as the feature selection dataset D.

[0022] D={id,st,f,w,s1,s2,s3,s4,E,tp,ti}

[0023] 3) Processing numerical values

[0024] Perform missing value imputation, outlier value replacement, and encoding conversion on the feature selection dataset D.

[0025] The missing value imputation method involves filling missing values ​​with 0; the abnormal value replacement method uses a random function to determine the learner's dwell time tp according to formula (3). i :

[0026] tp i =random(tp) b ,tp i-1 ),tp i ≥120 (3)

[0027] In equation (3), random() is a random function, and tp i tp ≥120 i For outliers, the outlier tp i Replace with the random function tp b Let tp represent the learner's dwell time corresponding to the b-th sample. i Let tp represent the learner's dwell time corresponding to the i-th sample, and b takes the value [5, 15], where b < i.

[0028] The encoding conversion described above involves performing an integer encoding conversion on the weather type w; the integer encoding conversion is as follows: using the weather type w (sunny, cloudy, partly cloudy, light rain, moderate rain, heavy rain) to represent the weather type w corresponding to the i-th sample. i Map to different integers in [1, 6]; convert all values ​​in the feature selection dataset D to floating-point numbers using the float function.

[0029] 4) Split the dataset

[0030] The feature selection dataset D is divided into a training set D train and a test set D test .

[0031] (2) Data augmentation

[0032] The training set D train is subjected to Gaussian white noise data augmentation according to formula (4), and the augmented training set D is obtained as follows:

[0033]

[0034]

[0035]

[0036] In formula (4), concat() represents g times of splicing of data processed by random() along the column, g takes values [20, 30], |D train | is the length of the training set, and takes values of finite positive integers, D train,p represents the p-th sample in the training set D train , and snr represents the signal-to-noise ratio of white noise, taking values [20, 40].

[0037] (3) Process data

[0038] The mobile learning context E and the weather type w in the Gaussian white noise data augmentation dataset D and the test set D test are converted by using the one-hot encoding method respectively; the mobile learning context E i corresponding to the i-th sample is converted into a one-dimensional vector with q elements, and the weather type w i corresponding to the i-th sample is converted into a one-dimensional vector with 6 elements.

[0039] All data X in the Gaussian white noise data augmentation dataset D and the test set D test are subjected to normalization processing according to formula (5), and X' is:

[0040]

[0041] The Gaussian white noise data augmentation dataset D and the test set D test are converted into supervised data D' with front and back time sequence relationship according to formula (6):

[0042] D' = shift(Ds, n) (6)

[0043] shift(Ds,n) = [[d1,d2,..,d n ],[d2,d3,..,d n+1 ],...,[d |Ds|-n+1 ,d |Ds|-n+2 ,..,d |Ds| ]]

[0044] In formula (6), shift() is a supervised data transformation function; Ds represents a Gaussian white noise data augmentation dataset or a test set D test , d1 represents the first sample in the dataset Ds, and n takes values [30, 50], n < |Ds|.

[0045] (4) Constructing a combined neural network

[0046] The combined neural network is connected by a normalization layer, a long short-term memory-convolutional neural network, a long short-term memory-attention neural network, a noise-long short-term memory neural network, a long short-term memory neural network, a fully connected neural network 1, and a fully connected neural network 2.

[0047] The output end of the normalization layer is connected to the input ends of the parallel long short-term memory-convolutional neural network and the long short-term memory-attention neural network, the output ends of the long short-term memory-convolutional neural network and the long short-term memory-attention neural network are connected to the input end of the noise-long short-term memory neural network and the input end of the long short-term memory neural network, and the output end of the long short-term memory neural network is connected to the input ends of the parallel fully connected neural network 1 and the fully connected neural network 2.

[0048] (5) Training the combined neural network

[0049] The Gaussian white noise data augmentation dataset is input to the normalization layer of the combined neural network, and the Adam optimizer is used for network training; the loss function uses the focal loss function according to formula (7) and the huber loss function according to formula (8).

[0050] FL(p t )=-α t (1-p t ) γ log(p t ) (7)

[0051]

[0052]

[0053] In formula (7), y E is the true value of the mobile learning context E, is the predicted value of mobile learning context E, p t is the probability value of the true value of mobile learning context E being the same as the predicted value of mobile learning context E, p takes values in [0, 1]; a t represents the weight value of the true value of mobile learning context E being the same as the predicted value of mobile learning context E, both a and g are parameters of the focal loss function, a takes values in [10 -7 , 10 -5 ], and g takes values in [1, 3].

[0054]

[0055] In formula (8), y tp is the true value of tp, is the predicted value of tp, and d is a parameter of the huber loss function, taking values in [1, 3]; y tp and can be replaced by y ti and

[0056] During training, the sets with batch sizes of {8, 16, 32, 64} are used for training respectively; the learning rate n is automatically adjusted according to formula (9) until the focal loss function and the huber loss function converge:

[0057]

[0058] In formula (9), n0 represents the initial learning rate, taking values in [0.0001, 0.01] as decimals, epoch represents the training rounds, taking values in [30, 50].

[0059] (6) Combination of neural network testing and inverse normalization

[0060] The test set D test is used for network testing, and the mobile learning context E, the learner's stay duration tp and the mobile learning context switching duration ti are predicted respectively; after the testing is completed, the final predicted data is inverse normalized to Y as follows:

[0061]

[0062] (7) Performance evaluation of combined neural network

[0063] The accuracy, precision, recall, F1-score, weighted precision, weighted F1-score and root mean square error evaluation indicators are used to evaluate the effect of the network on the test set.

[0064] The accuracy Accuracy is determined according to formula (11):

[0065]

[0066] TE m indicates the mth mobile learning context E m the number of samples predicted correctly, W indicates the test set D test the total number of samples, which is a finite positive integer.

[0067] The precision Precision is determined according to formula (12):

[0068]

[0069]

[0070] Precision m indicates the precision of the mth mobile learning context TE m FE m indicates the mth mobile learning context E m the number of samples predicted incorrectly.

[0071] The recall Recall is determined according to formula (13):

[0072]

[0073]

[0074] Recall m indicates the recall of the mth mobile learning context E m u takes values [1, q].

[0075] The F1-score is determined according to formula (14):

[0076]

[0077]

[0078] F1-score m indicates the F1-score of the mth mobile learning context E m

[0079] The weighted precision WP is determined according to formula (15):

[0080]

[0081] W m indicates the mth mobile learning context E m in the test set D​test the number of samples in the set.

[0082] The weighted F1-score WF1 is determined according to formula (16):

[0083]

[0084] The root mean square error RMSE is determined according to formula (17):

[0085]

[0086] In formula (17), when evaluating the prediction result of the learner stay duration tp, t v represents the actual value of the learner stay duration tp v corresponding to the vth sample, t v ' represents the predicted value of the learner stay duration tp v corresponding to the vth sample; when evaluating the prediction result of the mobile learning context switching duration ti, t v represents the actual value of the mobile learning context switching duration ti v corresponding to the vth sample, t v ' represents the predicted value of the mobile learning context switching duration ti v corresponding to the vth sample; v takes values [1, W].

[0087] The long short-term memory-convolutional neural network of the application is composed of a long short-term memory layer 1, a long short-term memory layer 2 and a convolutional layer connected in sequence.

[0088] The long short-term memory layer 1 of the application is composed of 64 neurons, and the activation function is a relu activation function; the long short-term memory layer 2 is composed of 32 neurons, and the activation function is a relu activation function; the convolutional layer is composed of 32 one-dimensional convolutional kernels with a size of 1, a step of 3 and a relu activation function.

[0089] The long short-term memory-attention neural network of the application is composed of a long short-term memory 1, a long short-term memory layer 2 and an attention layer connected in sequence.

[0090] The long short-term memory 1 of the application is composed of 64 neurons, and the activation function is a relu activation function; the long short-term memory layer 2 is composed of 32 neurons, and the activation function is a relu activation function; the attention layer is composed of 16 neurons, and the activation function is a softmax activation function.

[0091] The noise-long short-term memory neural network of the application is composed of a Gaussian noise layer, a long short-term memory layer and a random inactivation layer connected in sequence.

[0092] The standard deviation of the Gaussian noise layer of the application is 0.2; the long short-term memory layer is composed of 64 neurons, and the activation function is a relu activation function; the random inactivation rate of the random inactivation layer is 0.4.

[0093] The full connection neural network 1 of the application is composed of a full connection layer 1, a random inactivation layer and a full connection layer 2 in sequence.

[0094] The full connection layer 1 of the application is composed of 32 neurons, and the activation function is a leaky_relu activation function; the random inactivation rate of the random inactivation layer is 0.2; the full connection layer 2 is composed of 16 neurons, and the activation function is a softmax activation function.

[0095] The structure of the full connection neural network 2 of the application is the same as that of the full connection neural network 1, and the activation function used is a sigmoid activation function.

[0096] Compared with the prior art, the application has the following advantages:

[0097] Due to the data preprocessing step adopted by the application, the single time attribute arrival time in the original data can be expanded through time format conversion, which facilitates subsequent prediction of the learning duration of the learner in the mobile learning context and the switching duration of the mobile learning context; the data is subjected to feature selection, numerical processing and data set division, so that the data can be standardized for network training.

[0098] Due to the data enhancement step adopted by the application, the problem of insufficient data quantity is solved, the data quantity is effectively expanded, the generalization ability of network training is improved, and the prediction performance of the network is improved.

[0099] Due to the data processing step adopted by the application, the data is more standardized for subsequent training and convergence of the combined neural network by converting the data into one-hot encoding, 0-1 normalization processing and converting into supervised data.

[0100] Due to the combined neural network adopted by the application, the network is applied to the prediction method of the mobile learning context, meaningful mobile learning context time sequence information is extracted, the local features affecting the mobile learning context are effectively selected, and the time and related features with greater influence are selected from the time dimension, which is more conducive to predicting using existing mobile learning context data, and the prediction performance is better. BRIEF DESCRIPTION OF DRAWINGS

[0101] Figure 1 is a flowchart of embodiment 1 of the application.

[0102] Figure 2 is a structure diagram of a combined neural network.

[0103] Figure 3 is Figure 2 Structure diagram of a long short-term memory-convolutional neural network.

[0104] Figure 4 is Figure 2 Structure diagram of a long short-term memory-attention neural network.

[0105] Figure 5 is Figure 2 Structure diagram of a noise-long short-term memory neural network.

[0106] Figure 6 is Figure 2 Structure diagram of a fully connected neural network 1. DETAILED DESCRIPTION

[0107] The application will be further described in detail below in combination with the drawings and examples, but the application is not limited to the following examples.

[0108] Example 1

[0109] In Figure 1 In this embodiment, the mobile learning context prediction method based on a combined neural network comprises the following steps:

[0110] (1) Preprocessing data

[0111] The data set is derived from a mobile learning context data set R obtained by 50 learners using a data collection APP:

[0112] R={id,E,st,f,l,a,h,dv,w,S}

[0113] In the formula, id is the name of the learner, E is the mobile learning context, st is the arrival time of the learner, f is the floor where the learner is located, l is the longitude where the learner is located, a is the latitude where the learner is located, h is the height where the learner is located, dv is the model of the collection device, the type of weather, S

[0114] is a sensor; E takes the value [1, q], q takes the value of a limited positive integer, and in this embodiment, q takes the value of 9; f takes the value [-1, 18], l, a, and h take the value of a limited positive real number, w is selected from sunny, cloudy, overcast, light rain, moderate rain, and heavy rain; S is a sensor, S∈{s1,s1,...,s 73}, wherein s1 is a light sensor, s2 is an acceleration sensor, s3 is a direction sensor, and s4 is a step sensor.

[0115] The mobile learning context data set R is preprocessed as follows:

[0116] 1) Learner arrival time expansion

[0117] The learner arrival time st is extended by the learner stay duration tp, and st i and st j The learner stay duration tp is extended i :

[0118] tp i = st j - st i (1)

[0119] In formula (1), st j represents the learner arrival time st corresponding to the jth sample, st i represents the learner arrival time st corresponding to the ith sample, i and j are finite positive integers with a value of at least 2, and in this embodiment, i and j have values of [2, 32863], wherein i < j.

[0120] The learner arrival time st is extended by the learner stay duration tp, and st k and st j The learner stay duration tp is extended k :

[0121] ti k = st k - st j (2)

[0122] In formula (2), st k represents the learner arrival time st corresponding to the kth sample, k is a finite positive integer with a value of at least 3, and in this embodiment, k has a value of [3, 32863], wherein j < k.

[0123] The data set R' after the arrival time extension in the mobile learning context data set R is:

[0124] R' = {id, E, st, f, l, a, h, dv, w, S, tp, ti}

[0125] 2) Select features

[0126] The Pearson correlation coefficient is used to select the features with an absolute value of the correlation coefficient greater than 0.1 from the correlation coefficients between the learner name id, the learner arrival time st, the floor where the learner is located f, the longitude where the learner is located l, the latitude where the learner is located a, the height where the learner is located h, the type of the collection device dv, the weather type w, and the sensor S in the extended data set R' and the mobile learning context E, the learner stay duration tp, and the mobile learning context switching duration ti as the feature selection data set D:

[0127] D={id,st,f,w,s1,s2,s3,s4,E,tp,ti}

[0128] 3) Processing numerical values

[0129] Perform missing value imputation, outlier value replacement, and encoding conversion on the feature selection dataset D.

[0130] The missing value imputation method involves filling missing values ​​with 0; the abnormal value replacement method uses a random function to determine the learner's dwell time tp according to formula (3). i :

[0131] tp i =random(tp) b ,tp i-1 ),tp i ≥120 (3)

[0132] In equation (3), random() is a random function, and tp i tp ≥120 i For outliers, the outlier tp i Replace with the random function tp b Let tp represent the learner's dwell time corresponding to the b-th sample. i Let tp represent the learner's dwell time corresponding to the i-th sample, and b takes the value [5, 15]. In this embodiment, b takes the value 10, and b < i.

[0133] The encoding conversion described above involves performing an integer encoding conversion on the weather type w; the integer encoding conversion is as follows: using the weather type w (sunny, cloudy, partly cloudy, light rain, moderate rain, heavy rain) to represent the weather type w corresponding to the i-th sample. i Map to different integers in [1, 6]; convert all values ​​in the feature selection dataset D to floating-point numbers using the float function.

[0134] 4) Split the dataset

[0135] The feature selection dataset D is divided into a training set D in a 1:1 ratio. train Test set D test .

[0136] (2) Data Augmentation

[0137] Apply formula (4) to the training set D train Gaussian white noise data augmentation is performed to obtain the augmented training set. for:

[0138]

[0139]

[0140]

[0141] In equation (4), concat() means concatenating the data processed by random() along the column g times, where g takes the value [20, 30]. In this embodiment, g takes the value 25. |D train | is the length of the training set, and its values ​​are finite positive integers, D train,p The training set D represents train The p-th sample in the dataset is represented by snr, which represents the signal-to-noise ratio of white noise and has a value of [20, 40]. In this embodiment, snr is 30.

[0142] (3) Data processing

[0143] Augmentation datasets for Gaussian white noise data respectively and test set D test The mobile learning context E and weather type w are converted using a one-hot encoding method; the mobile learning context E corresponding to the i-th sample is... i Convert to a one-dimensional vector with q elements, and assign the weather type w corresponding to the i-th sample. i Convert to a one-dimensional vector with 6 elements.

[0144] Augmentation of Gaussian white noise dataset and test set D test All data X in the equation are normalized according to equation (5), and X′ is:

[0145]

[0146] Augmenting Gaussian white noise data dataset and test set D test Convert the data into supervised data D' with sequential temporal relationships according to equation (6):

[0147] D′=shift(Ds,n) (6)

[0148] shift(Ds,n)=[[d1,d2,..,d n ],[d2,d3,..,d n+1 ],...,[d |Ds|-n+1 ,d |Ds|-n+2 ,..,d |Ds| ]]

[0149] In equation (6), shift() is a supervised data transformation function; Ds represents the Gaussian white noise data augmentation dataset. Or test set D testd1 represents the first sample in the dataset Ds, and n takes the value [30, 50]. In this embodiment, n takes the value 40, and n < |Ds|.

[0150] (4) Constructing a combined neural network

[0151] exist Figure 2 In this embodiment, the combined neural network consists of a normalization layer, a long short-term memory-convolutional neural network, a long short-term memory-attention neural network, a noise-long short-term memory neural network, a long short-term memory neural network, a fully connected neural network 1, and a fully connected neural network 2. The output of the normalization layer is connected to the input of the parallel long short-term memory-convolutional neural network and the long short-term memory-attention neural network. One output of the long short-term memory-convolutional neural network and the long short-term memory-attention neural network is connected to the input of the noise-long short-term memory neural network, and the other output is connected to the input of the long short-term memory neural network. The output of the long short-term memory neural network is connected to the input of the parallel fully connected neural network 1 and the fully connected neural network 2.

[0152] exist Figure 3 In this embodiment, the Long Short-Term Memory-Convolutional Neural Network is composed of Long Short-Term Memory Layer 1, Long Short-Term Memory Layer 2, and a convolutional layer connected in series. Long Short-Term Memory Layer 1 consists of 64 neurons with the ReLU activation function, Long Short-Term Memory Layer 2 consists of 32 neurons with the ReLU activation function, and the convolutional layer consists of 32 one-dimensional convolutional kernels of size 1 with a stride of 3 and the ReLU activation function.

[0153] exist Figure 4 In this embodiment, the Long Short-Term Memory-Attention Neural Network is composed of Long Short-Term Memory 1, Long Short-Term Memory Layer 2, and an Attention Layer connected in series. Long Short-Term Memory 1 consists of 64 neurons with the ReLU activation function, Long Short-Term Memory Layer 2 consists of 32 neurons with the ReLU activation function, and the Attention Layer consists of 16 neurons with the softmax activation function.

[0154] exist Figure 5 In this embodiment, the noise-long short-term memory neural network consists of a Gaussian noise layer, a long short-term memory layer, and a random deactivation layer connected in series. The standard deviation of the Gaussian noise layer is 0.2, the long short-term memory layer consists of 64 neurons, the activation function is the ReLU activation function, and the random deactivation rate of the random deactivation layer is 0.4.

[0155] exist Figure 6In this embodiment, the full connection neural network 1 is composed of a full connection layer 1, a random inactivation layer, and a full connection layer 2 in sequence. The full connection layer 1 is composed of 32 neurons, the activation function is a leaky_relu activation function, the random inactivation rate of the random inactivation layer is 0.2, and the full connection layer 2 is composed of 16 neurons, and the activation function is a softmax activation function.

[0156] The structure of the full connection neural network 2 is the same as that of the full connection neural network 1, and the activation function used is a sigmoid activation function.

[0157] (5) Training the combined neural network

[0158] The Gaussian white noise data augmentation dataset is input into the normalization layer of the combined neural network, and the Adam optimizer is used for network training; the loss function uses the focal loss function of formula (7) and the huber loss function of formula (8) respectively:

[0159] FL(p t )=-α t (1-p t ) γ log(p t ) (7)

[0160]

[0161]

[0162] In formula (7), y E is the true value of the mobile learning context E, is the predicted value of the mobile learning context E, p t is the probability value that the true value of the mobile learning context E is the same as the predicted value of the mobile learning context E, and p takes a value of [0, 1]; α t represents the weight value that the true value of the mobile learning context E is the same as the predicted value of the mobile learning context E, both α and γ are parameters of the focal loss function, α takes a value of [10 -7 , 10 -5 ], γ takes a value of [1, 3], α of this embodiment takes a value of 10 -6 , and γ takes a value of 2.

[0163]

[0164] In formula (8), y tp is the true value of tp, is the predicted value of tp, and δ is a parameter of the huber loss function, taking a value of [1, 3], and δ of this embodiment takes a value of 2; y tp and may be replaced by y ti and

[0165] The training set is trained with batch size {8, 16, 32, 64} respectively. The learning rate η of training is automatically adjusted according to formula (9) until the focal loss function and huber loss function converge.

[0166]

[0167] In formula (9), η0represents the initial learning rate, which is a decimal number with a value of [0.0001, 0.01], and in the embodiment, η0has a value of 0.001; epoch represents the training round, which has a value of [30, 50], and in the embodiment, epoch has a value of 40.

[0168] (6) Combination of neural network test and inverse normalization

[0169] The test set D test is used for network testing, and the mobile learning context E, the learner's stay duration tpand the mobile learning context switching duration tiare predicted respectively; after the test is completed, the final predicted data Y is obtained. The inverse normalization processing Y is:

[0170]

[0171] (7) Performance evaluation of combined neural network

[0172] The accuracy, precision, recall, F1-score, weighted precision, weighted F1-score and root mean square error evaluation indexes are used to evaluate the effect of the network on the test set.

[0173] The accuracy Accuracy is determined according to formula (11):

[0174]

[0175] In formula (11), TE m represents the mthmobile learning context E m predicted correctly, and W represents the total number of samples in the test set D test , which is a finite positive integer.

[0176] The precision Precision is determined according to formula (12):

[0177]

[0178]

[0179] Precision m in formula (12) represents the precision of the mth mobile learning context TE m . m in formula (12) represents the number of samples predicted incorrectly by the mth mobile learning context TE m .

[0180] Recall is determined according to formula (13) :

[0181]

[0182]

[0183] Recall m in formula (13) represents the recall of the mth mobile learning context TE m , and u takes values in [1, q].

[0184] F1-score is determined according to formula (14) :

[0185]

[0186]

[0187] F1-score m in formula (14) represents the F1-score of the mth mobile learning context TE m .

[0188] Weighted Precision WP is determined according to formula (15) :

[0189]

[0190] W m in formula (15) represents the number of samples of the mth mobile learning context TE m in the test set D test .

[0191] Weighted F1-score WF1 is determined according to formula (16) :

[0192]

[0193] Root Mean Square Error RMSE is determined according to formula (17) :

[0194]

[0195] t v in formula (17) represents the actual value of the learner stay duration tp v corresponding to the vth sample, and t vtpv represents the predicted value of the learner's staying time tp corresponding to the vth sample v the predicted result of the mobile learning context switching time ti v t iv represents the actual value of the mobile learning context switching time ti corresponding to the vth sample v the predicted value of the mobile learning context switching time ti corresponding to the vth sample v t iv represents the actual value of the mobile learning context switching time ti corresponding to the vth sample v the predicted value of the mobile learning context switching time ti corresponding to the vth sample; v takes values [1, W].

[0196] The mobile learning context prediction method based on the combined neural network is completed.

[0197] Embodiment 2

[0198] The mobile learning context prediction method based on the combined neural network of this embodiment consists of the following steps:

[0199] (1) Preprocessing data

[0200] The mobile learning context data set R is preprocessed as follows:

[0201] 1) Learner arrival time expansion

[0202] This step is the same as that in Embodiment 1.

[0203] 2) Select features

[0204] This step is the same as that in Embodiment 1.

[0205] 3) Process numerical values

[0206] In formula (3), b takes values [5, 15], and in this embodiment, b takes value 5, b < i. The other steps of this step are the same as those in Embodiment 1.

[0207] 4) Divide the data set

[0208] This step is the same as that in Embodiment 1.

[0209] (2) Data enhancement

[0210] In formula (4), concat() represents that the data processed by random() is spliced along the column for g times, g takes values [20, 30], and in this embodiment, g takes value 20, snr represents the signal-to-noise ratio of white noise, and takes values [20, 40], and in this embodiment, snr takes value 20, and the other steps of this step are the same as those in Embodiment 1.

[0211] (3) Process data

[0212] In formula (6), n takes values [30, 50], and in this embodiment, n takes value 30, n < |Ds|. The other steps of this step are the same as those in Embodiment 1.

[0213] (4) Constructing the combined neural network

[0214] This step is the same as that in Embodiment 1.

[0215] (5) Training the combined neural network

[0216] In formula (7), both a and y are parameters of the focal loss function, a takes a value of [10 -7 , 10 -5 ], and y takes a value of [1, 3], a takes a value of 10 -7 and y takes a value of 1 in this embodiment. In formula (8), d is a parameter of the huber loss function and takes a value of [1, 3], d takes a value of 1 in this embodiment. In formula (9), n0 represents an initial learning rate and takes a value of [0.0001, 0.01] as a decimal number, n0 takes a value of 0.0001 in this embodiment, epoch represents a training round and takes a value of [30, 50], and epoch takes a value of 30 in this embodiment. The other steps of this step are the same as those in Embodiment 1.

[0217] The other steps are the same as those in Embodiment 1, and the mobile learning context prediction method based on the combined neural network is completed.

[0218] Embodiment 3

[0219] The mobile learning context prediction method based on the combined neural network in this embodiment comprises the following steps:

[0220] (1) Preprocessing data

[0221] The mobile learning context data set R is preprocessed as follows:

[0222] 1) Learner arrival time expansion

[0223] This step is the same as that in Embodiment 1.

[0224] 2) Selecting features

[0225] This step is the same as that in Embodiment 1.

[0226] 3) Processing numerical values

[0227] In formula (3), b takes a value of [5, 15], b takes a value of 15 in this embodiment, and b < i. The other steps of this step are the same as those in Embodiment 1.

[0228] 4) Dividing the data set

[0229] This step is the same as that in Embodiment 1.

[0230] (2) Data augmentation

[0231] In formula (4), concat() represents that the data processed by random() is spliced along the column for g times, g is [20, 30], g is 30 in the embodiment, snr represents the signal-to-noise ratio of white noise, and snr is [20, 40], snr is 40 in the embodiment, and other steps of the step are the same as those in embodiment 1.

[0232] (3) Processing data

[0233] In formula (6), n is [30, 50], n is 50 in the embodiment, and n < |Ds|. Other steps of the step are the same as those in embodiment 1.

[0234] (4) Constructing a combined neural network

[0235] The step is the same as that in embodiment 1.

[0236] (5) Training the combined neural network

[0237] In formula (7), both a and y are parameters of the focal loss function, a is [10 -7 , 10 -5 ], y is [1, 3], a is 10 -5 in the embodiment, and y is 3. In formula (8), d is a parameter of the huber loss function, d is [1, 3], and d is 3 in the embodiment. In formula (9), n0 represents an initial learning rate, which is a decimal number and is [0.0001, 0.01], n0 is 0.01 in the embodiment, epoch represents a training round, and epoch is [30, 50], epoch is 50 in the embodiment. Other steps of the step are the same as those in embodiment 1.

[0238] Other steps are the same as those in embodiment 1, and the mobile learning context prediction method based on the combined neural network is completed.

[0239] In order to verify the beneficial effects of the application, the inventors carried out laboratory simulation experiments by using the mobile learning context prediction method based on the combined neural network of the inventors in embodiment 1, and various experimental conditions are as follows:

[0240] 1. Experimental environment

[0241] Experimental environment: the server uses Intel Core i7-7820X CPU@3.60GHzx16, the operating system type is 64-bit, python 3.7.9 and TensorFlow 2.2.0 are used, and the third-party libraries are keras, sklearn, pandas, numpy, matplotlib and math.

[0242] 2. Experimental content and results

[0243] The experimental results of the mobile learning context prediction of different training batches are evaluated in Table 1, the accuracy of the mobile learning context prediction of different training batches is shown in Table 2, and the four evaluation indexes of the mobile learning context prediction of the training batch of 16 are shown in Table 3.

[0244] Table 3 shows the evaluation results of the performance of the mobile learning context prediction of each mobile learning context using the four evaluation indexes of accuracy, precision, recall and F1-score.

[0245] Table 1 shows the experimental results of the mobile learning context prediction of different training batches

[0246]

[0247] As shown in Table 1, the accuracy, precision, recall, F1-score, weighted precision and weighted F1-score of the combined network when the training batch is 16 are higher than those when the training batch is 8, 32 or 64, and the higher the evaluation value, the better the mobile learning context prediction method. The root mean square error of the learner's stay duration of the combined network when the training batch is 16 is lower than that when the training batch is 8, 32 or 64, and the lower the evaluation value, the better the mobile learning context prediction method. The root mean square error of the mobile learning context switching duration of the combined network when the training batch is 16 is higher than that when the training batch is 8, which is 0.07, and lower than that when the training batch is 32 or 64, and the lower the evaluation value, the better the mobile learning context prediction method. The above analysis shows that the mobile learning context prediction method of the combined network when the training batch is 16 is the best.

[0248] Table 2 shows the four evaluation indexes of the mobile learning context prediction of the training batch of 16

[0249]

[0250] As shown in Table 2, the accuracy, precision, recall and F1-score of the combined network for different mobile learning contexts when the training batch is 16 are different, and the individual evaluation value is relatively low.

[0251] Experimental conclusion: The method has good performance in each evaluation index, can effectively predict the next mobile learning context, the learner's stay duration in the mobile learning context and the switching duration of the mobile learning context, and can be applied to the recommendation of learning resources of different forms and durations of learners, and can also be applied to the application scenarios of WiFi data flow area prediction.

Claims

1. A mobile learning context prediction method based on ensemble neural networks, characterized in that: It consists of the following steps: (1) Preprocessed data The dataset is derived from a mobile learning context dataset R obtained by 50 learners using a data collection app. R = {id, E, st, f, l, a, h, dv, w, S} In the formula, id represents the learner's name, E represents the mobile learning context, st represents the learner's arrival time, f represents the floor the learner is on, l represents the learner's longitude, a represents the learner's latitude, h represents the learner's altitude, dv represents the data acquisition device model and weather type, and S represents the sensor; E takes values ​​of [1, q], where q is a finite positive integer; f takes values ​​of [-1, 18], where l, a, and h are finite positive real numbers, and w selects sunny, cloudy, partly cloudy, light rain, moderate rain, and heavy rain; S represents the sensor, and S∈{s1, s1, ..., s1} 73 }, where s1 is a light sensor, s2 is an acceleration sensor, s3 is a direction sensor, and s4 is a step sensor; The mobile learning scenario dataset R is preprocessed as follows: 1) Learner arrival time expansion The learner's arrival time st is extended to the learner's dwell time tp, and st is adjusted according to formula (1). i and st j Extend learner dwell time (tp) i : tp i =st j -st i (1) In equation (1), st j Let st represent the arrival time of the learner corresponding to the j-th sample. i Let st represent the arrival time of the learner corresponding to the i-th sample, where i and j are finite positive integers with a value of at least 2, and i < j; The mobile learning context switching time ti is extended according to the learner's arrival time st, and st is adjusted according to formula (2). k and st j Extend learner dwell time k : IT k =st k -st j (2) In equation (2), st k Let st represent the arrival time of the learner corresponding to the k-th sample, where k is a finite positive integer with a value of at least 2, and j < k; The time-expanded dataset R′ in the mobile learning context dataset R is: R′={id,E,st,f,l,a,h,dv,w,S,tp,ti}; 2) Select features Using Pearson correlation coefficients, among the correlation coefficients of learner name (id), learner arrival time (st), learner floor (f), learner longitude (l), learner latitude (a), learner altitude (h), data acquisition device model (dv), weather type (w), and sensor S in the expanded dataset R′ with mobile learning context E, learner dwell time (tp), and mobile learning context switching time (ti), features with an absolute value of correlation coefficient greater than 0.1 are selected as the feature selection dataset D. D={id,st,f,w,s1,s2,s3,s4,E,tp,ti} 3) Processing numerical values Perform missing value imputation, outlier value replacement, and encoding conversion on the feature selection dataset D; The missing value imputation method involves filling missing values ​​with 0; the abnormal value replacement method uses a random function to determine the learner's dwell time tp according to formula (3). i : tp i =random(tp b ,tp i-1 ),tp i ≥120 (3) In equation (3), random() is a random function, and tp i tp ≥120 i For outliers, the outlier tp i Replace with the random function tp b Let tp represent the learner's dwell time corresponding to the b-th sample. i Let tp represent the learner's dwell time corresponding to the i-th sample, and b takes the value [5, 15], b < i; The encoding conversion described above involves performing an integer encoding conversion on the weather type w; the integer encoding conversion is as follows: using the weather type w (sunny, cloudy, partly cloudy, light rain, moderate rain, heavy rain) to represent the weather type w corresponding to the i-th sample. i Map to different integers in [1, 6]; convert all values ​​in the feature selection dataset D to floating-point numbers using the float function; 4) Split the dataset The feature selection dataset D is divided into a training set D in a 1:1 ratio. train Test set D test ; (2) Data Augmentation Apply formula (4) to the training set D train Gaussian white noise data augmentation is performed to obtain the augmented training set. for: In equation (4), concat() means concatenating the data processed by random() along the column g times, where g takes the value [20, 30], |D train | is the length of the training set, and its values ​​are finite positive integers, D train,p The training set D represents train The p-th sample in the dataset, where snr represents the white noise signal-to-noise ratio and takes values ​​of [20, 40]; (3) Data processing Augmentation datasets for Gaussian white noise data respectively and test set D test The mobile learning context E and weather type w are converted using a one-hot encoding method; the mobile learning context E corresponding to the i-th sample is... i Convert to a one-dimensional vector with q elements, and assign the weather type w corresponding to the i-th sample. i Convert to a one-dimensional vector with 6 elements; Augmentation of Gaussian white noise dataset and test set D test All data X in the equation are normalized according to equation (5), and X′ is: Augmenting Gaussian white noise data dataset and test set D test Convert the data into supervised data D' with sequential temporal relationships according to equation (6): D′=shift(Ds,n) (6) shift(Ds,n)=[[d1,d2,..,d n ],[d2,d3,..,d n+1 ],...,[d |Ds|-n+1 ,d |Ds|-n+2 ,..,d |Ds| ]] In equation (6), shift() is a supervised data transformation function; Ds represents the Gaussian white noise data augmentation dataset. Or test set D test d1 represents the first sample in dataset Ds, and n takes the value [30, 50], n < |Ds|; (4) Constructing a combined neural network The ensemble neural network consists of a normalization layer, a long short-term memory-convolutional neural network, a long short-term memory-attention neural network, a noise-long short-term memory neural network, a long short-term memory neural network, a fully connected neural network 1, and a fully connected neural network 2. The output of the normalization layer is connected to the input of the parallel Long Short-Term Memory-Convolutional Neural Network and Long Short-Term Memory-Attention Neural Network. The output of the Long Short-Term Memory-Convolutional Neural Network and the Long Short-Term Memory-Attention Neural Network is connected to the input of the Noise-Long Short-Term Memory Neural Network in one path and to the input of the Long Short-Term Memory Neural Network in the other path. The output of the Long Short-Term Memory Neural Network is connected to the input of the parallel fully connected Neural Network 1 and fully connected Neural Network 2. (5) Training the combined neural network Augmenting Gaussian white noise data dataset The input is fed into the normalization layer of the combined neural network, and the Adam optimizer is used for network training; the loss functions are the focal loss function according to Equation (7) and the Huber loss function according to Equation (8), respectively. FL(p t )=-a t (1-p t ) γ log(p t ) (7) In equation (7), y E This is the true value of the mobile learning scenario E. This is the predicted value of mobile learning scenario E, p t α is the probability that the actual value of mobile learning scenario E is the same as the predicted value of mobile learning scenario E, where p takes the value [0,1]; t The weights representing the values ​​of the actual value and the predicted value of the mobile learning scenario E are the same. α and γ are both parameters of the focal loss function, with α taking values ​​in the range [10]. -7 10 -5 ], where γ takes the value [1,3]; In equation (8), y tp It is the actual value of tp. y is the predicted value of tp, δ is the parameter of the Huber loss function, and its value is [1,3]; tp and They can be replaced with y respectively. ti and During training, sets with batch sizes of {8, 16, 32, 64} are used for training respectively; the training learning rate η is automatically adjusted according to equation (9) until the focal loss function and the Huber loss function converge; In equation (9), η0 represents the initial learning rate, which is a decimal with a value of [0.0001, 0.01], and epoch represents the training epoch, which has a value of [30, 50]. (6) Testing and inverse normalization of ensemble neural networks Using test set D test Conduct network testing to predict mobile learning scenario E, learner dwell time tp, and mobile learning scenario switching time ti; after the test, the final prediction data will be used. The inverse normalization process for Y is: (7) Performance evaluation of ensemble neural networks The network's performance on the test set is evaluated using the metrics of accuracy, precision, recall, F1-score, weighted precision, weighted F1-score, and root mean square error. Accuracy is determined according to formula (11): In formula (11), TE m E represents the m-th mobile learning scenario. m The number of samples that were correctly predicted, W represents the number of samples in the test set D. test The total number of samples and the values ​​are finite positive integers. Determine the precision according to formula (12): In equation (12), Precision m Represents the m-th mobile learning scenario (TE) m The accuracy of FE m E represents the m-th mobile learning scenario. m The number of samples that were incorrectly predicted; Determine the recall rate using formula (13): In equation (13), Recall m E represents the m-th mobile learning scenario. m The recall rate, u takes the value [1, q]; Determine the F1-score according to formula (14): In equation (14), E represents the m-th mobile learning scenario. m F1 score; Determine the weighted precision WP according to formula (15): In equation (15), W m E represents the m-th mobile learning scenario. m In test set D test The number of samples in; Determine the weighting according to formula (16) The root mean square error (RMSE) is determined according to formula (17): In equation (17), when evaluating the predicted learner dwell time tp, t v The duration tp represents the learner's dwell time corresponding to the v-th sample. v The actual value of t v 'tp' represents the learner's dwell time corresponding to the v-th sample. v The predicted value of t; when evaluating the prediction results of the mobile learning context switching duration ti, t v The duration ti represents the mobile learning context switching time corresponding to the v-th sample. v The actual value of t v ' represents the mobile learning context switching duration ti corresponding to the v-th sample. v The predicted value; v takes the value [1, W].

2. The mobile learning context prediction method based on ensemble neural networks according to claim 1, characterized in that: In step (4) of constructing a combined neural network, the long short-term memory-convolutional neural network is composed of long short-term memory layer 1, long short-term memory layer 2, and convolutional layer connected in series.

3. The mobile learning context prediction method based on ensemble neural networks according to claim 2, characterized in that: The Long Short-Term Memory (LSTM) layer 1 consists of 64 neurons, with the ReLU activation function; the LSM layer 2 consists of 32 neurons, with the ReLU activation function; the convolutional layer consists of 32 one-dimensional convolutional kernels of size 1, with a stride of 3, and the ReLU activation function.

4. The mobile learning context prediction method based on ensemble neural networks according to claim 1, characterized in that: In step (4) of constructing a combined neural network, the long short-term memory-attention neural network is composed of long short-term memory 1, long short-term memory layer 2, and attention layer connected in series.

5. The mobile learning context prediction method based on ensemble neural networks according to claim 4, characterized in that: The Long Short-Term Memory 1 consists of 64 neurons, with the ReLU activation function; the Long Short-Term Memory Layer 2 consists of 32 neurons, with the ReLU activation function; and the Attention Layer consists of 16 neurons, with the softmax activation function.

6. The mobile learning context prediction method based on ensemble neural networks according to claim 1, characterized in that: In step (4) of constructing a combined neural network, the noise-long short-term memory neural network is composed of a Gaussian noise layer, a long short-term memory layer, and a random deactivation layer connected in series.

7. The mobile learning context prediction method based on ensemble neural networks according to claim 6, characterized in that: The standard deviation of the Gaussian noise layer is 0.2; the long short-term memory layer consists of 64 neurons and the activation function is the ReLU activation function; the random inactivation rate of the random inactivation layer is 0.

4.

8. The mobile learning context prediction method based on ensemble neural networks according to claim 1, characterized in that: In step (4) of constructing a combined neural network, the fully connected neural network 1 is composed of a fully connected layer 1 connected in series with a random deactivation layer and a fully connected layer 2.

9. The mobile learning context prediction method based on ensemble neural networks according to claim 8, characterized in that: The fully connected layer 1 consists of 32 neurons and uses the leaky_relu activation function; the random inactivation rate of the random inactivation layer is 0.2; the fully connected layer 2 consists of 16 neurons and uses the softmax activation function.

10. The mobile learning context prediction method based on ensemble neural networks according to claim 1, characterized in that: In step (4) of constructing the combined neural network, the structure of the fully connected neural network 2 is the same as that of the fully connected neural network 1, and the activation function used is the sigmoid activation function.

Images