Memory state prediction method based on contrastive learning and stabilized long short-term memory network and application thereof
By combining contrastive learning with a stabilized long short-term memory network, the problems of noise robustness and stable temporal modeling in memory state prediction of deep learning models are solved, and higher accuracy memory state prediction is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- YUNNAN NORMAL UNIV
- Filing Date
- 2026-04-10
- Publication Date
- 2026-06-26
AI Technical Summary
Existing deep learning models struggle to handle noise and temporal fluctuations in learning behavior when predicting memory states, resulting in insufficient prediction accuracy and a lack of stable temporal modeling capabilities.
A memory state prediction method based on contrastive learning and stabilized long short-term memory networks is adopted. Through data preprocessing, data augmentation and sequence feature extraction, a loss function is constructed to train the model, thereby achieving noise robustness and stable temporal modeling.
It improves the accuracy and stability of memory state prediction, enhances the model's generalization ability in the face of noise and time fluctuations, and realizes the dynamic characterization of learners' memory states and the prediction of future retention.
Smart Images
Figure CN122020070B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of deep learning technology, and in particular to a method for predicting memory states based on contrastive learning and stabilizing long short-term memory networks and its application. Background Technology
[0002] Spaced repetition, based on the forgetting curve, enhances long-term retention by scheduling reviews at appropriate times. For example, in language learning, learners need to review the same word multiple times to form long-term memory. Spaced repetition doesn't involve repeatedly presenting the word in a short period; instead, it distributes reviews across different time points, gradually increasing the time interval between adjacent reviews. This reduces ineffective repetition while enhancing long-term memory retention.
[0003] Deep learning models typically predict learners' future memory states based on their past review behavior in order to generate personalized review plans. The learner's past review behavior, as input, usually includes recall results, review intervals, and historical recall probabilities; the future memory state, as output, includes memory half-life and recall probability, which are used to characterize the rate of memory decay and determine the timing of review.
[0004] In current technical solutions, most common methods rely on heuristic rules or traditional regression models, which struggle to handle noise and temporal fluctuations in the learning process, resulting in insufficient prediction accuracy. Deep sequence models also tend to suffer from gradient instability in long sequence modeling and lack effective noise resistance mechanisms. Therefore, this application proposes a memory state prediction method that simultaneously possesses noise robustness and stable temporal modeling capabilities. Summary of the Invention
[0005] The main purpose of this application is to provide a memory state prediction method based on contrastive learning and stabilized long short-term memory networks, aiming to solve the problem of how to achieve memory state prediction with noise robustness and stable temporal modeling capabilities.
[0006] To achieve the above objectives, this application provides a method for predicting memory states based on contrastive learning and stabilized long short-term memory networks, the method comprising:
[0007] S10, Obtain the preprocessed review sequence dataset, wherein the review sequence dataset consists of multiple review sequences including recall results, review time intervals, and historical recall probabilities;
[0008] S20, take a portion of the review sequences in the review sequence dataset, and perform recall result perturbation enhancement, review time interval perturbation enhancement, and historical recall probability perturbation enhancement respectively to obtain the first positive sample, the second positive sample, and the third positive sample. Take the other review sequences in the review sequence dataset that do not belong to the positive samples as negative samples.
[0009] S30, based on a stabilized long short-term memory network, extract sequence representations from the review sequence dataset, extract the first positive sample, the second positive sample, and the third positive sample to obtain positive sample representations, and extract the negative sample representations corresponding to the negative samples;
[0010] S40, a loss function is constructed based on the sequence representation, the positive sample representation, and the negative sample representation to guide the training of a preset large model;
[0011] S50, acquire the review sequence data to be predicted, and perform memory state prediction on the review sequence data to be predicted based on the trained large model.
[0012] Optionally, S20 includes:
[0013] S21, extract the review sequence dataset according to the first preset ratio to obtain the first target review sequence, and perform label flipping enhancement on the recall results in the first target review sequence to obtain the first positive sample;
[0014] S22, extract the review sequence dataset according to the second preset ratio to obtain the second target review sequence, and shorten / extend the corresponding review time interval according to the value of the recall result in the second target review sequence to obtain the second positive sample;
[0015] S23, extract the third target review sequence from the review sequence dataset according to the third preset ratio, inject Gaussian noise and random offset into the historical recall probability of the third target review sequence to obtain the third positive sample;
[0016] S24, take a portion of the review sequences other than the first positive sample, the second positive sample, and the third positive sample in the review sequence dataset as negative samples.
[0017] Optionally, S21 specifically includes:
[0018] S211, the label-flipping enhancement of the first target review sequence is performed using the following formula:
[0019]
[0020] In the formula, For uniformly distributed random variables, To preset the flip probability, For the enhanced recall results, This is the initial recall result;
[0021] S212, the first target review sequence after label flipping enhancement is used as the first positive sample.
[0022] Optionally, S22 specifically includes:
[0023] S221, when the value of the recall result of the second target review sequence meets the first preset value, the first preset increment is summed with the corresponding review time interval to shorten the review time interval, wherein the first preset increment is a negative value;
[0024] S222, when the value of the recall result of the second target review sequence meets the second preset value, the second preset increment is summed with the corresponding review time interval to extend the review time interval, wherein the second preset increment is a positive value;
[0025] S223, the second target review sequence with shortened / extended review intervals is used as the second positive sample.
[0026] Optionally, S23 specifically includes:
[0027] S231, Inject Gaussian noise and random offset into the historical recall probability of the third target review sequence:
[0028]
[0029] In the formula, Let be the probability of perturbation history recall after Gaussian noise is injected. The initial probability of historical recall; This is a random perturbation term introduced to the probability of historical recall. This indicates that the expression follows a pattern with a mean of 0 and a variance of . Gaussian random noise;
[0030] S232, Probability of Perturbing Historical Recall The value of is restricted to the interval Inside:
[0031]
[0032] S233, the third target review sequence after being injected with Gaussian noise and random offset is used as the third positive sample.
[0033] Optionally, S40 specifically includes:
[0034] S41, a multilayer perceptron regression network is used to predict the sequence representation. Corresponding memory half-life value :
[0035]
[0036] In the formula, , These represent the learnable weight parameters of each layer in the multilayer perceptron regression network; For activation functions; , These represent the learnable bias parameters corresponding to each layer;
[0037] S42, based on predicted half-life and the first in the current training batch The true memory half-life label of a review sequence Determine the mean square error loss function :
[0038]
[0039] In the formula, This represents the number of samples in the current training batch that participate in the regression loss calculation.
[0040] S43, Determine sequence characterization , respectively with positive sample characterization Negative sample characterization Cosine similarity between , Based on cosine similarity , Determine the contrastive learning loss function :
[0041]
[0042] In the formula, For batch size, For temperature parameters;
[0043] S44, based on the mean square error loss function Comparative learning loss function The weighted sum is used to determine the loss function. :
[0044]
[0045] In the formula, , These are hyperparameters that control the weights of the half-life regression loss and the contrast loss, respectively.
[0046] S45, based on loss function Backpropagation training is performed on the trainable parameters in the pre-defined large model.
[0047] Optionally, in S10, the step of preprocessing the review sequence dataset includes:
[0048] S11, Let the i-th review action of the same user on the same knowledge item be... Sort all review activities in chronological order to obtain the initial review sequence S:
[0049]
[0050] In the formula, Indicates the result of recall; Indicates the time interval between adjacent review sessions; Represents the probability of historical recall. Indicates the sequence length;
[0051] S12, the time interval in the initial review sequence S Perform scaling and normalization to obtain the transformation time interval The probability of historical recall The probability of transformation history recall is obtained by performing centralization and scaling transformation. :
[0052]
[0053]
[0054] In the formula, , , All are coefficients of variation derived from empirical values;
[0055] S13, constructing the result of recall , change time interval and the probability of changing historical recall The preprocessed review sequence dataset consists of... :
[0056] .
[0057] Optionally, in S50, the memory state includes a predicted half-life value. and recall probability value The steps of predicting the memory state of the review sequence data to be predicted based on the trained preset model include:
[0058] S51, based on a large trained model Calculate the review sequence data to be predicted half-life prediction :
[0059]
[0060] S52, based on half-life predictions , with the review sequence data to be predicted The time interval Calculate the probability value of recall :
[0061] .
[0062] In addition, to achieve the above objectives, this application also provides a computer system comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the memory state prediction method based on contrastive learning and stabilized long short-term memory networks as described in any of the preceding claims.
[0063] In addition, to achieve the above objectives, this application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the memory state prediction method based on contrastive learning and stabilized long short-term memory networks as described in any of the preceding claims.
[0064] This application has at least the following beneficial effects:
[0065] 1. In the data preprocessing stage, this application proposes to scale and normalize the time interval of the review sequence data, and to center and scale the historical recall probability to form a standardized input sequence that can reflect the learner's memory change trajectory.
[0066] 2. In the data augmentation stage, this application proposes to apply three types of perturbations to the sequence: label flipping, time perturbation, and probability shift. These three perturbations simulate the situations that may occur during long-term learning, so that multiple positive samples of the same sequence can be robust to noisy labels, have generalization ability under different review time shifts, and still converge stably when observation errors and recording biases occur.
[0067] 3. In the sequence feature extraction stage, this application proposes to use a stabilized long short-term memory network (sLSTM) with exponential gating and normalization factor to extract features from the review sequence and the positive and negative samples after data augmentation, which can effectively ensure numerical stability when modeling long sequences.
[0068] 4. This application proposes to jointly calculate the loss and optimize the model by combining the features of positive and negative samples with the overall features of the sequence. A fully connected layer is used as a regression prediction module to predict the memory half-life. The MSE loss between the actual half-life and the predicted half-life is calculated as the regression loss of the model. Then, the contrastive learning loss is calculated based on the representation of positive and negative samples. This loss increases the cosine similarity between the sequence and positive samples and decreases the cosine similarity between the sequence and negative samples, so that the model can bring the positive sample pairs closer and push the negative sample pairs further apart in the embedding space.
[0069] 5. Input the review sequence to be predicted into the long short-term memory network and regression prediction module, output the memory half-life prediction value, and further calculate the recall probability at any time point based on the half-life, so as to realize the dynamic characterization of the learner's memory state and the prediction of future retention. Attached Figure Description
[0070] Figure 1 This is a flowchart illustrating the memory state prediction method based on contrastive learning and stabilized long short-term memory networks involved in the embodiments of this application.
[0071] Figure 2 This is a schematic diagram of the overall framework of the memory state prediction method based on contrastive learning and stabilized long short-term memory networks involved in the embodiments of this application;
[0072] Figure 3 This is a schematic diagram of the hardware operating environment of the computer system involved in the embodiments of this application.
[0073] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation
[0074] To better understand the above technical solutions, exemplary embodiments of this disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of this disclosure to those skilled in the art.
[0075] First Embodiment
[0076] Reference Figure 1 The illustrated method for predicting memory states based on contrastive learning and stabilized long short-term memory networks includes the following steps:
[0077] S10, Obtain the preprocessed review sequence dataset, wherein the review sequence dataset consists of multiple review sequences including recall results, review time intervals, and historical recall probabilities;
[0078] In this embodiment, the original review sequence data is first preprocessed to form a standardized input sequence that can reflect the learner's memory change trajectory.
[0079] Optionally, the preprocessing includes scaling and normalizing the time intervals, and centering and scaling the historical recall probabilities, specifically including the following steps:
[0080] S11, Let the i-th review action of the same user on the same knowledge item be... Sort all review activities in chronological order to obtain the initial review sequence S:
[0081]
[0082] In the formula, Indicates the result of recall; Indicates the time interval between adjacent review sessions; Represents the probability of historical recall. Indicates the sequence length;
[0083] S12, the time interval in the initial review sequence S Perform scaling and normalization to obtain the transformation time interval The probability of historical recall The probability of transformation history recall is obtained by performing centralization and scaling transformation. :
[0084]
[0085]
[0086] In the formula, , , All are coefficients of variation derived from empirical values;
[0087] In some alternative implementations, α is typically 100, μ = 0.5, and σ = 0.5.
[0088] S13, constructing the result of recall , change time interval and the probability of changing historical recall The preprocessed review sequence dataset consists of... :
[0089] .
[0090] S20, take a portion of the review sequences in the review sequence dataset, and perform recall result perturbation enhancement, review time interval perturbation enhancement, and historical recall probability perturbation enhancement respectively to obtain the first positive sample, the second positive sample, and the third positive sample. Other review sequences in the review sequence dataset that were not extracted are taken as negative samples.
[0091] The processed review sequence dataset Next, data augmentation is performed on the input sequence to construct positive and negative samples for contrastive learning.
[0092] As an innovative step proposed in this embodiment, this step aims to enhance the resistance of large models to noise and temporal fluctuations in their learning behavior. It proposes to establish a contrastive learning objective by label flipping, temporal perturbation, and probabilistic noise injection, allowing the model to iteratively update parameters based on the contrastive learning objective.
[0093] The specific details of this step are explained below:
[0094] The purpose of perturbation enhancement for recall results is to simulate occasional errors or unexpected successes that may occur during real learning, thereby improving the model's robustness to noisy labels. The sequence after flipping enhancement is similar to the original sequence, therefore it is used as the first positive sample, denoted as [sample name missing]. .
[0095] The purpose of enhancing the review interval perturbation is to simulate the review rhythm of different learners and reflect the time fluctuations that learners may experience during recall. The principle is to improve the model's generalization ability under different review time shifts by introducing a conditional distribution perturbation mechanism. The sequence enhanced by the interval perturbation is similar to the original sequence, therefore it is used as the second positive sample, denoted as [sample name missing]. .
[0096] For probability perturbation enhancement of historical recall, it is equivalent to label smoothing on the probability labels, ensuring stable convergence of the model even with observation errors and recording biases. The sequence after probability shift enhancement is similar to the original sequence, therefore it is used as the third positive sample, denoted as [sample name missing]. .
[0097] It should be noted that the samples of the three positive samples are all sampled independently. Therefore, there may be overlap between the first positive sample, the second positive sample, and the third positive sample, or there may be no overlap.
[0098] In addition to the three positive sample features, a portion of other sequences from the same training batch are sampled as sequences. Negative samples.
[0099] S30, based on a stabilized long short-term memory network, extract sequence representations from the review sequence dataset, extract the first positive sample, the second positive sample, and the third positive sample to obtain positive sample representations, and extract the negative sample representations corresponding to the negative samples;
[0100] After obtaining three sets of positive and negative samples, a stabilized long short-term memory network (sLSTM) is used to extract the features of the samples.
[0101] It should be noted that, through comparative experiments, the sample features constructed based on the stabilized long short-term memory network can effectively ensure numerical stability when modeling long sequences. Therefore, this network was selected as the basic architecture for feature extraction.
[0102] Specifically, each review activity As The input is sequentially passed to the input gate and the forget gate, then the candidate state and the cell state are updated, and finally the hidden state is obtained through the output gate.
[0103] Specifically, it includes:
[0104] Input gate for
[0105]
[0106] Forgotten Gate for
[0107]
[0108] Candidate state The update is
[0109]
[0110] The cell state is updated as follows
[0111]
[0112] Output gate and hidden state are
[0113]
[0114] In the formula, Use the Sigmoid activation function; Indicates element-wise multiplication; and Let be the learning parameters of the model, where and For input gate Learning parameters, and Forgotten Gate Learning parameters, and Candidate state Learning parameters; For the first The hidden state at any given moment; For the first The hidden state at any given moment.
[0115] In the above, the exponential activation function ensures that the input gate value is positively scaled to avoid gradient decay; the state update part uses normalization constraints to ensure numerical stability when modeling long sequences.
[0116] In this step, the obtained , and Positive sample representations are obtained through the aforementioned stabilization long short-term memory network processing steps. , and Other sequences from the same training batch are sampled as sequences. Negative samples are processed using the same stabilization long short-term memory network method to obtain negative sample representations. .
[0117] Additionally, it should be noted that for sequences... After the complete input sequence is obtained, the result of the last time step is taken. As a sequence representation, it is denoted as .
[0118] S40, a loss function is constructed based on the sequence representation, the positive sample representation, and the negative sample representation to guide the training of a preset large model;
[0119] After obtaining sequence characterization Positive sample characterization , and and negative sample characterization Then, a loss function is constructed based on the three representations to optimize the pre-defined large model.
[0120] Specifically, the loss function consists of two parts: the mean squared error loss function. and contrastive learning loss function .
[0121] Mean squared error loss function The half-life regression loss, used in the model, is employed to predict the memory half-life value corresponding to the review sequence. Specifically, it includes:
[0122] S41, a multilayer perceptron regression network is used to predict the sequence representation. Corresponding memory half-life value :
[0123]
[0124] In the formula, , These represent the learnable weight parameters of each layer in the multilayer perceptron regression network; For activation functions; , These represent the learnable bias parameters corresponding to each layer;
[0125] S42, based on predicted half-life and the first in the current training batch The true memory half-life label of a review sequence Determine the mean square error loss function :
[0126]
[0127] In the formula, This represents the number of samples in the current training batch that participate in the regression loss calculation.
[0128] On the other hand, the contrastive learning loss function This is used to calculate the similarity between positive and negative sample pairs, aiming to enable the model to shorten the distance between positive sample pairs and widen the distance between negative sample pairs in the embedding space, thereby improving the discriminative power of the memorized representation. Specifically, it includes:
[0129] S43, Determine sequence characterization , respectively with positive sample characterization Negative sample characterization Cosine similarity between , Based on cosine similarity , Determine the contrastive learning loss function :
[0130]
[0131] In the formula, For batch size, For temperature parameters;
[0132] In this embodiment, the expression for calculating cosine similarity is:
[0133]
[0134] Furthermore, S44, according to the mean squared error loss function Comparative learning loss function The weighted sum is used to determine the loss function. :
[0135]
[0136] In the formula, , These are hyperparameters that control the weights of the half-life regression loss and the contrast loss, respectively.
[0137] S45, based on loss function Backpropagation training is performed on the trainable parameters in the pre-defined large model.
[0138] S50, acquire the review sequence data to be predicted, and perform memory state prediction on the review sequence data to be predicted based on the trained large model.
[0139] After completing the model training in the aforementioned steps, for the review sequence data that needs to be predicted, i.e. the review sequence data to be predicted, the trained large model is used to predict the review sequence data to be predicted and output the memory state prediction result.
[0140] Specifically, and optionally, in this embodiment, the memory state includes a half-life prediction value. and recall probability value The specific calculation steps include:
[0141] S51, based on a large trained model Calculate the review sequence data to be predicted half-life prediction :
[0142]
[0143] S52, based on half-life predictions , with the review sequence data to be predicted The time interval Calculate the probability value of recall :
[0144] .
[0145] After the above steps, the memory state prediction is completed.
[0146] In the technical solution provided in this embodiment, during the data preprocessing stage, the time interval of the review sequence data is scaled and normalized, and the historical recall probability is centered and scaled to form a standardized input sequence that reflects the learner's memory change trajectory, which is used as training data. During the data augmentation stage, the sequence is subjected to three perturbations: label flipping, time perturbation, and probability shifting, to construct multiple similar samples for the same sequence. These similar samples are designated as positive samples, and other samples are randomly sampled as negative samples. During the sequence feature extraction stage, a stabilized long short-term memory network (sLSTM) with exponential gating and a normalization factor is used to extract features from the review sequence and the data-augmented positive and negative samples. The positive and negative sample features are then compared with the sequence... The overall characteristics are combined for loss calculation and model optimization. A fully connected layer is used as a regression prediction module to predict the memory half-life. The MSE loss between the actual half-life and the predicted half-life is calculated as the regression loss of the model. Then, the contrastive learning loss is calculated based on the representation of positive and negative samples. This loss increases the cosine similarity between the sequence and positive samples and decreases the cosine similarity between the sequence and negative samples. Finally, the half-life regression loss and the contrastive learning loss are combined to optimize the model parameters. Finally, based on the optimized model, the review sequence to be predicted is input into the long short-term memory network and the regression prediction module, and the memory half-life prediction value is output. Furthermore, the recall probability at any time point is calculated based on the half-life, so as to realize the dynamic characterization of the learner's memory state and the prediction of future retention.
[0147] Second Embodiment
[0148] Based on the first embodiment, this embodiment further explains the selection of positive and negative samples in step S20.
[0149] The perturbation enhancement of the recall results for the first positive sample specifically includes the following steps:
[0150] S21, extract the review sequence dataset according to a first preset ratio to obtain a first target review sequence, and perform label flipping enhancement on the recall results in the first target review sequence to obtain a first positive sample; further, S21 specifically includes:
[0151] S211, the label-flipping enhancement of the first target review sequence is performed using the following formula:
[0152]
[0153] In the formula, For uniformly distributed random variables, To preset the flip probability, For the enhanced recall results, This is the initial recall result;
[0154] S212, the first target review sequence after label flipping enhancement is used as the first positive sample.
[0155] It should be noted that this operation is in Simulates the memory transition from "success" to "failure" in real time. The simulation of "failure → success" corrective recall forms a dynamic uncertainty data augmentation process. The sequence after flipping augmentation is similar to the original sequence, therefore it is used as the first positive sample, denoted as . .
[0156] The enhancement of the review time interval perturbation for the second positive sample specifically includes the following steps:
[0157] S22, extract the review sequence dataset according to the second preset ratio to obtain the second target review sequence, and shorten / extend the corresponding review time interval according to the value of the recall result in the second target review sequence to obtain the second positive sample;
[0158] Furthermore, S22 includes:
[0159] S221, when the value of the recall result of the second target review sequence meets the first preset value, the first preset increment is summed with the corresponding review time interval to shorten the review time interval, wherein the first preset increment is a negative value;
[0160] For example, let the first preset ratio be 10%, the first preset value be 1, and the review interval be... First preset increment Shortened review interval :
[0161]
[0162] When recalling the results At that time, the disturbance satisfies It is used to simulate the stable memory caused by "pre-study".
[0163] S222, when the value of the recall result of the second target review sequence meets the second preset value, the second preset increment is summed with the corresponding review time interval to extend the review time interval, wherein the second preset increment is a positive value;
[0164] For example, let the first preset ratio be 10%, the second preset value be 0, and the review interval be... First preset increment The increased review interval :
[0165]
[0166] When recalling the results At that time, the disturbance satisfies It is used to simulate the enhanced forgetting caused by "delayed review".
[0167] S223, the second target review sequence with shortened / extended review intervals is used as the second positive sample.
[0168] Similarly, the sequence enhanced by the interval perturbation is similar to the original sequence, and therefore serves as the second positive sample, denoted as . .
[0169] The perturbation of the historical recall probability of the third positive sample is enhanced, specifically including the following steps:
[0170] S23, extract the third target review sequence from the review sequence dataset according to the third preset ratio, inject Gaussian noise and random offset into the historical recall probability of the third target review sequence to obtain the third positive sample;
[0171] Furthermore, S23 specifically includes:
[0172] S231, Inject Gaussian noise and random offset into the historical recall probability of the third target review sequence:
[0173]
[0174] In the formula, Let be the probability of perturbation history recall after Gaussian noise is injected. The initial probability of historical recall; This is a random perturbation term introduced to the probability of historical recall. This indicates that the expression follows a pattern with a mean of 0 and a variance of . Gaussian random noise;
[0175] S232, Probability of Perturbing Historical Recall The value of is restricted to the interval Inside:
[0176]
[0177] S233, the third target review sequence after being injected with Gaussian noise and random offset is used as the third positive sample.
[0178] Similarly, the sequence enhanced by probability shift is similar to the original sequence, and therefore serves as the third positive sample, denoted as . .
[0179] In this embodiment, the sampling method for negative samples can be to uniformly and randomly extract half of the remaining sequences after removing the positive samples from the same training batch as negative samples; other conventional methods can also be used, which will not be elaborated in this embodiment.
[0180] Verification of Examples
[0181] To verify the effectiveness of the above method (referred to as CL-HLR in this embodiment), experiments were conducted on a real language learning behavior dataset, and its performance was compared with several baseline methods. The main comparison models included: traditional heuristics such as Pimsleur and Leitner systems, classic half-life regression models HLR and DHP, the improved recurrent neural network model GRU-HLR, and the time-enhanced half-life regression model THLR. The experiments were evaluated on recall probability prediction and half-life prediction tasks, using metrics including the mean absolute error of recall probability prediction (MAO). ) and the symmetric mean absolute percentage error of half-life prediction ( Table 1 shows the performance comparison results of different methods on the test set.
[0182] Table 1. Performance of each method on recall probability prediction and half-life prediction tasks.
[0183]
[0184] The results in Table 1 demonstrate that, compared with various memory half-life prediction models, the method of this invention achieves the best performance in both recall probability prediction and half-life prediction tasks. In the recall probability prediction task, the method of this invention achieves an error reduction of approximately 15.7% compared to the best baseline model, while in the half-life prediction task, it achieves a performance improvement of 16.4% compared to the best baseline model, fully verifying the significant advantages of the method of this invention in terms of noise robustness, sequence modeling ability, and overall prediction accuracy.
[0185] As one implementation scheme, Figure 3 This is a schematic diagram of the hardware operating environment of the computer system involved in the embodiments of this application.
[0186] like Figure 3As shown, the computer system may include: a processor 1001, such as a CPU; a memory 1005; a user interface 1003; a network interface 1004; and a communication bus 1002. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.
[0187] Those skilled in the art will understand that Figure 3 The computer system architecture shown does not constitute a limitation on the computer system and may include more or fewer components than shown, or combine certain components, or have different component arrangements.
[0188] like Figure 3 As shown, the memory 1005, as a storage medium, may include an operating system, a network communication module, a user interface module, and computer programs. The operating system is a program that manages and controls the hardware and software resources of the computer system, as well as the operation of the computer programs and other software or programs.
[0189] exist Figure 3 In the computer system shown, the user interface 1003 is mainly used to connect to the terminal and communicate with the terminal; the network interface 1004 is mainly used to communicate with the backend server; and the processor 1001 can be used to call the computer program stored in the memory 1005.
[0190] In this embodiment, the computer system includes: a memory 1005, a processor 1001, and a computer program stored in the memory and executable on the processor, wherein:
[0191] When processor 1001 calls a computer program stored in memory 1005, it performs the following operations:
[0192] S10, Obtain the preprocessed review sequence dataset, wherein the review sequence dataset consists of multiple review sequences including recall results, review time intervals, and historical recall probabilities;
[0193] S20, take a portion of the review sequences in the review sequence dataset, and perform recall result perturbation enhancement, review time interval perturbation enhancement, and historical recall probability perturbation enhancement respectively to obtain the first positive sample, the second positive sample, and the third positive sample. Take the other review sequences in the review sequence dataset that do not belong to the positive samples as negative samples.
[0194] S30, based on a stabilized long short-term memory network, extract sequence representations from the review sequence dataset, extract the first positive sample, the second positive sample, and the third positive sample to obtain positive sample representations, and extract the negative sample representations corresponding to the negative samples;
[0195] S40, a loss function is constructed based on the sequence representation, the positive sample representation, and the negative sample representation to guide the training of a preset large model;
[0196] S50, acquire the review sequence data to be predicted, and perform memory state prediction on the review sequence data to be predicted based on the trained large model.
[0197] Furthermore, those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program includes program instructions and can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in a computer system to implement the process steps of the embodiments of the above methods.
[0198] Therefore, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the various steps of the memory state prediction method based on contrastive learning and stabilized long short-term memory networks as described in the above embodiments.
[0199] The computer-readable storage medium can be any computer-readable storage medium capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), magnetic disk, or optical disk.
[0200] It should be noted that, since the storage medium provided in the embodiments of this application is the storage medium used to implement the methods of the embodiments of this application, those skilled in the art can understand the specific structure and variations of the storage medium based on the methods described in the embodiments of this application, and therefore will not be repeated here. All storage media used in the methods of the embodiments of this application fall within the scope of protection of this application.
[0201] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0202] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0203] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0204] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0205] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.
[0206] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.
Claims
1. A method for predicting memory states based on contrastive learning and stabilized long short-term memory networks, characterized in that, The method includes the following steps: S10, Obtain the preprocessed review sequence dataset, wherein the review sequence dataset consists of multiple review sequences including recall results, review time intervals, and historical recall probabilities; S20, take a portion of the review sequences in the review sequence dataset, and perform recall result perturbation enhancement, review time interval perturbation enhancement, and historical recall probability perturbation enhancement respectively to obtain the first positive sample, the second positive sample, and the third positive sample. Take the other review sequences in the review sequence dataset that do not belong to the positive samples as negative samples. S30, based on a stabilized long short-term memory network, extract sequence representations from the review sequence dataset, extract the first positive sample, the second positive sample, and the third positive sample to obtain positive sample representations, and extract the negative sample representations corresponding to the negative samples; S40, a loss function is constructed based on the sequence representation, the positive sample representation, and the negative sample representation to guide the training of a preset large model; S50, acquire the review sequence data to be predicted, and perform memory state prediction on the review sequence data to be predicted based on the trained large model; S20 includes: S21, extract the review sequence dataset according to the first preset ratio to obtain the first target review sequence, and perform label flipping enhancement on the recall results in the first target review sequence to obtain the first positive sample; S22, extract the review sequence dataset according to the second preset ratio to obtain the second target review sequence, and shorten / extend the corresponding review time interval according to the value of the recall result in the second target review sequence to obtain the second positive sample; S23, extract the third target review sequence from the review sequence dataset according to the third preset ratio, inject Gaussian noise and random offset into the historical recall probability of the third target review sequence to obtain the third positive sample; S24, take a portion of the review sequences other than the first positive sample, the second positive sample, and the third positive sample in the review sequence dataset as negative samples.
2. The memory state prediction method based on contrastive learning and stabilized long short-term memory networks as described in claim 1, characterized in that, S21 specifically includes: S211, the label-flipping enhancement of the first target review sequence is performed using the following formula: ； In the formula, For uniformly distributed random variables, To preset the flip probability, For the enhanced recall results, This is the initial recall result; S212, the first target review sequence after label flipping enhancement is used as the first positive sample.
3. The memory state prediction method based on contrastive learning and stabilized long short-term memory networks as described in claim 1, characterized in that, S22 specifically includes: S221, when the value of the recall result of the second target review sequence meets the first preset value, the first preset increment is summed with the corresponding review time interval to shorten the review time interval, wherein the first preset increment is a negative value; S222, when the value of the recall result of the second target review sequence meets the second preset value, the second preset increment is summed with the corresponding review time interval to extend the review time interval, wherein the second preset increment is a positive value; S223, the second target review sequence with shortened / extended review intervals is used as the second positive sample.
4. The memory state prediction method based on contrastive learning and stabilized long short-term memory networks as described in claim 1, characterized in that, S23 specifically includes: S231, Inject Gaussian noise and random offset into the historical recall probability of the third target review sequence: ； In the formula, Let be the probability of perturbation history recall after Gaussian noise is injected. The initial probability of historical recall; This is a random perturbation term introduced to the probability of historical recall. This indicates that the expression follows a pattern with a mean of 0 and a variance of . Gaussian random noise; S232, Probability of Perturbing Historical Recall The value of is restricted to the interval Inside: ； S233, the third target review sequence after being injected with Gaussian noise and random offset is used as the third positive sample.
5. The memory state prediction method based on contrastive learning and stabilized long short-term memory networks as described in claim 1, characterized in that, S40 specifically includes: S41, a multilayer perceptron regression network is used to predict the sequence representation. Corresponding memory half-life value : ； In the formula, , These represent the learnable weight parameters of each layer in the multilayer perceptron regression network; For activation functions; , These represent the learnable bias parameters corresponding to each layer; S42, based on predicted half-life and the first in the current training batch The true memory half-life label of a review sequence Determine the mean squared error loss function : ； In the formula, This represents the number of samples in the current training batch that participate in the regression loss calculation. S43, Determine sequence characterization , respectively with positive sample characterization Negative sample characterization Cosine similarity between , Based on cosine similarity , Determine the contrastive learning loss function : ； In the formula, For batch size, For temperature parameters; S44, based on the mean square error loss function Comparative learning loss function The weighted sum is used to determine the loss function. : ； In the formula, , These are hyperparameters that control the weights of the half-life regression loss and the contrastive loss, respectively. S45, based on loss function Backpropagation training is performed on the trainable parameters in the pre-defined large model.
6. The memory state prediction method based on contrastive learning and stabilized long short-term memory networks as described in claim 1, characterized in that, In S10, the preprocessing step for the review sequence dataset includes: S11, Let the i-th review action of the same user on the same knowledge item be... Sort all review activities in chronological order to obtain the initial review sequence S: ； In the formula, Indicates the result of recall; Indicates the time interval between adjacent review sessions; Represents the probability of historical recall. Indicates the sequence length; S12, the time interval in the initial review sequence S Perform scaling and normalization to obtain the transformation time interval The probability of initial historical recall The probability of transformation history recall is obtained by performing centralization and scaling transformation. : ； ； In the formula, , , All are coefficients of variation derived from empirical values; S13, constructing the result of recall , change time interval and the probability of changing historical recall The preprocessed review sequence dataset consists of... : 。 7. The memory state prediction method based on contrastive learning and stabilized long short-term memory networks as described in claim 1, characterized in that, In S50, the memory state includes a half-life prediction value. and recall probability value The steps of predicting the memory state of the review sequence data to be predicted based on the trained preset model include: S51, based on a large trained model Calculate the review sequence data to be predicted half-life prediction : ； S52, based on half-life predictions , with the review sequence data to be predicted The time interval Calculate the probability value of recall : 。 8. A computer system, characterized in that, The computer system includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, it implements the steps of the memory state prediction method based on contrastive learning and stabilization of long short-term memory networks as described in any one of claims 1 to 7.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the memory state prediction method based on contrastive learning and stabilization of long short-term memory networks as described in any one of claims 1 to 7.
Citation Information
Patent Citations
CN121166972A
CN121211914A