A method, device and equipment for generating an attention map based on weight attenuation

Abstract

This specification discloses a method, apparatus, and device for generating attention maps based on weight decay. By determining the feature matrix corresponding to the knowledge point sequence, the feature matrix includes a key matrix K, a query matrix Q, and a value matrix V; the feature value Q of the i-th knowledge point in the query matrix Q is obtained. i And, obtain the eigenvalue K of the j-th knowledge point in the key matrix K. j Determine the time difference Δ(t) between the i-th knowledge point and the j-th knowledge point. i ,t j ), determine the decay function γ(t) that includes the time difference and the decay parameter θ. i ,t j ,θ), where θ is a learnable decay parameter; based on the eigenvalue Q i eigenvalue K j The attenuation function γ(t) i ,t j The attention map is generated using the value matrix V, which in turn reinforces performance information of recently relevant questions and other questions by taking into account the time interval factor of learner forgetting behavior.

Description

Technical Field

[0001] This specification relates to the field of Internet technology, and in particular to a method, apparatus, and device for generating attention graphs based on weight decay. Background Technology

[0002] Currently, online learning has become the primary learning method for learners. Knowledge tracking models, as a key component of online learning, mainly predict learners' performance in future practice by tracking changes in their knowledge mastery during the learning process. Traditional knowledge tracking models often ignore factors such as learners' potential forgetting behavior during the learning process, and do not focus on the correlation between recently related questions and other questions. This makes it difficult for the attention maps generated during model training and prediction to accurately capture relevant features.

[0003] Therefore, a scheme for generating attention maps that contain accurate features in knowledge tracing models is needed. Summary of the Invention

[0004] This specification provides a method, apparatus, device, and storage medium for generating attention graphs based on weight decay, in order to solve the following technical problem: the need for a scheme for generating attention graphs that contain accurate features in a knowledge tracing model.

[0005] To solve the above-mentioned technical problems, one or more embodiments of this specification are implemented as follows:

[0006] In a first aspect, embodiments of this specification provide a method for generating an attention map based on weight decay, applied to a sequence of knowledge points containing knowledge points and time. The method includes: determining a feature matrix corresponding to the sequence of knowledge points, the feature matrix including a key matrix K, a query matrix Q, and a value matrix V; and obtaining the feature value Q of the i-th knowledge point in the query matrix Q. i And, obtain the eigenvalue K of the j-th knowledge point in the key matrix K. j Wherein, the j-th knowledge point is the preceding knowledge point related to the i-th knowledge point; the time difference Δ(t) between the i-th knowledge point and the j-th knowledge point is determined. i ,t j ), determine the decay function γ(t) that includes the time difference and the decay parameter θ. i ,t j ,θ), where θ is a learnable decay parameter; based on the eigenvalue Q i eigenvalue K j The attenuation function γ(t) i ,t j The attention map is generated using the value matrix V, θ).

[0007] In a second aspect, embodiments of this specification provide an apparatus for generating attention maps based on weight decay, applied to a sequence of knowledge points containing knowledge points and time. The apparatus includes: a feature determination module for determining a feature matrix corresponding to the knowledge point sequence, the feature matrix including a key matrix K, a query matrix Q, and a value matrix V; and a feature value acquisition module for acquiring the feature value Q of the i-th knowledge point in the query matrix Q. i And, obtain the eigenvalue K of the j-th knowledge point in the key matrix K. j Wherein, the j-th knowledge point is the preceding knowledge point related to the i-th knowledge point; the attenuation module determines the time difference Δ(t) between the i-th knowledge point and the j-th knowledge point. i ,t j ), determine the decay function γ(t) that includes the time difference and the decay parameter θ. i ,t j ,θ), where θ is a learnable decay parameter; the attention map generation module, based on the feature value Q i eigenvalue K j The attenuation function γ(t) i ,t j The attention map is generated using the value matrix V, θ).

[0008] In a third aspect, one or more embodiments of this specification provide an electronic device, comprising:

[0009] At least one processor; and,

[0010] A memory communicatively connected to the at least one processor; wherein,

[0011] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method as described in the first aspect.

[0012] In a fourth aspect, embodiments of this specification provide a non-volatile computer storage medium storing computer-executable instructions, which, when read by a computer from the storage medium, cause one or more processors to perform the method described in the first aspect.

[0013] The above-described at least one technical solution adopted in one or more embodiments of this specification can achieve the following beneficial effects: by determining the feature matrix corresponding to the knowledge point sequence, the feature matrix includes a key matrix K, a query matrix Q, and a value matrix V; and obtaining the feature value Q of the i-th knowledge point in the query matrix Q. i And, obtain the eigenvalue K of the j-th knowledge point in the key matrix K. jWherein, the j-th knowledge point is the preceding knowledge point related to the i-th knowledge point; the time difference Δ(t) between the i-th knowledge point and the j-th knowledge point is determined. i ,t j ), determine the decay function γ(t) that includes the time difference and the decay parameter θ. i ,t j ,θ), where θ is a learnable decay parameter; based on the eigenvalue Q i eigenvalue K j The attenuation function γ(t) i ,t j The attention map is generated by the matrix V and the value matrix V, thereby strengthening the performance information of recently related questions and other questions by taking into account the time interval factor of learner forgetting behavior. This results in the calculation of an attention map containing accurate features, which is beneficial for training a high-precision model and subsequent prediction. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 A flowchart illustrating a method for generating an attention map based on weight decay, provided in an embodiment of this specification.

[0016] Figure 2 This is a schematic diagram illustrating the relevance of a learner's answers to knowledge points, provided as an embodiment of this specification.

[0017] Figure 3 A schematic diagram of a forgetting curve provided for an embodiment of this specification;

[0018] Figure 4 A schematic diagram of a device for generating an attention map based on weight decay, provided in an embodiment of this specification;

[0019] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this specification. Detailed Implementation

[0020] This specification provides a method, apparatus, device, and storage medium for generating attention maps based on weight decay.

[0021] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application.

[0022] like Figure 1 As shown, Figure 1 This specification provides a flowchart illustrating a method for generating attention maps based on weight decay, applicable to a sequence of knowledge points containing knowledge points and time. The method includes:

[0023] S101: Determine the feature matrix corresponding to the knowledge point sequence, wherein the feature matrix includes a key matrix K, a query matrix Q, and a value matrix V.

[0024] First, let S = {(C1,t1,R1),...,(C n ,t n .R n R represents a sequence of knowledge points containing a series of knowledge points and times, where ti represents the timestamp when the learner answers the i-th question, and 1 ≤ i ≤ n. Ci refers to the question information of the learner's answer to the i-th question, such as the knowledge concept type to which the question belongs, indicating that the concept type encoding can be represented using one-hot encoding such as numbers or letters. i ∈{0,1} refers to the learner's response information to the i-th question, where 0 indicates an incorrect response and 1 indicates a correct response.

[0025] For example, a typical knowledge point in a knowledge point sequence might take the form (1, ti, 0), which means that the learner takes ti to answer the i-th question, the question type is encoded as "1", and the answer is "incorrect".

[0026] The corresponding learnable mapping function can be pre-set. Mapping function The corresponding knowledge points are mapped to d-dimensional space to obtain the corresponding basic features X. The basic features may include the mapping sub-matrices corresponding to the aforementioned C, t, and R.

[0027] After obtaining the basic characteristics X of the event sequence, W can be used. k W q and W vThe three weight matrices are multiplied by X to obtain the corresponding feature matrix for acquiring the event sequence. For example, the basic feature X is multiplied by the aforementioned weight matrix W. k Multiplying them yields the corresponding key matrix K; the basic feature X and the aforementioned weight matrix W q Multiplying the two matrices yields the corresponding query matrix Q; multiplying the basic feature X with the aforementioned weight matrix Wv yields the corresponding value matrix V. Each row in the generated key matrix K and query matrix Q actually corresponds to a knowledge point in the sequence of knowledge points.

[0028] S103, Obtain the feature value Q of the i-th knowledge point in the query matrix Q. i And, obtain the eigenvalue K of the j-th knowledge point in the key matrix K. j .

[0029] Attention maps require calculating the attention score between any two knowledge points to characterize the correlation between them. That is, for any current knowledge point, it can be designated as the i-th knowledge point, and the correlation between it and the j-th knowledge point can be calculated.

[0030] However, conventional calculations generally do not consider the memory decline that learners may experience during the learning process. In other words, to better simulate the changes in a learner's knowledge state over time, it is necessary to emphasize performance information from recent relevant questions.

[0031] Therefore, when calculating the correlation between the j-th knowledge point and the i-th knowledge point, we first identify the preceding knowledge point that is related to the i-th knowledge point and determine it as the j-th knowledge point. Here, correlation can refer to the knowledge points being of the same or similar types.

[0032] like Figure 2 As shown, Figure 2 This diagram illustrates the relevance of a learner's answers to knowledge points, as provided in an embodiment of this specification. In this diagram, the i-th knowledge point is taken as the current knowledge point. There may be other knowledge points unrelated to the i-th knowledge point (or none may exist) between the i-th and j-th knowledge points. Furthermore, there are no other knowledge points related to the i-th knowledge point between the i-th and j-th knowledge points.

[0033] In other words, among the knowledge points that are related to the i-th knowledge point, the j-th knowledge point is the one that is closest to the i-th knowledge point, that is, the related knowledge point that was last learned in the most recent time.

[0034] The correlation between two knowledge points can be determined by the corresponding codes in the knowledge point sequence. For example, if the correlation criterion requires that the two knowledge points be the same, then for the knowledge points (1, ti, 0) and (1, tj, 1) in the sequence, since the code for the knowledge point type is "1", it can be determined that the two are related knowledge points.

[0035] If the criterion for determining relevance requires that two knowledge points be similar, then similar knowledge points can be encoded using adjacent numerical codes during encoding. Therefore, if the distance between two codes does not exceed a preset distance (e.g., a preset distance of 2), the two knowledge points can be considered similar and thus related. For example, for the knowledge points (1, ti, 0) and (3, tj, 1) contained in the sequence, since the distance between their knowledge point type codes is 2 and does not exceed the preset distance, the two knowledge points can be considered similar and thus related.

[0036] For example, the Pearson correlation coefficient (r) can be used to characterize the correlation between two numerical sequences (i.e., columns, lists, sequences, etc.). The r value is a number between -1 and 1. The correlation (r) is determined by calculating the r values ​​of the codes for two knowledge point types. The closer r is to 1, the stronger the positive correlation; the closer r is to -1, the stronger the negative correlation (i.e., the more "opposite" the columns); and the closer r is to 0, the weaker the correlation.

[0037] S105, determine the time difference Δ(t) between the i-th knowledge point and the j-th knowledge point. i ,t j ), determine the decay function γ(t) that includes the time difference and the decay parameter θ. i ,t j ,θ), where θ is a learnable attenuation parameter.

[0038] As mentioned earlier, memory decline occurs during the learning process, and the longer the time interval, the greater the degree of forgetting. Figure 3 As shown, Figure 3 This is a schematic diagram of a forgetting curve provided for an embodiment of this specification. Statistics show that only 58% of the knowledge or experience is retained within 20 minutes of learning, while this figure drops to 33.7% within one day.

[0039] Therefore, when predicting learners' responses to the current problem, too early a question related to the current problem is not helpful in predicting their response. Instead, it is necessary to emphasize the performance information from more recent related questions. That is, we need to focus on the correlation between the i-th knowledge point and the j-th knowledge point mentioned above.

[0040] Based on this, a corresponding attenuation function γ(t) is introduced in the embodiments of this specification.i ,t j The function contains the time difference Δ(t) between the i-th knowledge point and the j-th knowledge point. i ,t j ) and the attenuation parameter θ, so that the time difference Δ(t) can be used to determine the attenuation parameter θ. i ,t j The decay of memory is characterized by γ(t), where θ is a learnable decay parameter in matrix form. i ,t j The specific form of θ can be set according to actual needs. For example, it can be in exponential form, power function form, absolute value form, etc.

[0041] S107, based on the characteristic value Q i eigenvalue K j The attenuation function γ(t) i ,t j The attention map is generated using the value matrix V, θ).

[0042] Determine the attenuation function γ(t) i ,t j After θ), we can first determine the eigenvalue Q. i eigenvalue K j Generate raw attention scores s i,j .

[0043] For example, determine the transpose vector Kj of the eigenvalue Kj. T Thus, the Q can be i and transpose vector K j T The original attention score is obtained by multiplication. And based on the original attention score s i,j and the decay function γ(t) i ,t j ,θ) to perform memory decay, thereby obtaining a new attention score s i,j,γ .

[0044] Then, the new attention score can be multiplied by the value matrix V to generate the attention map. For example, the attention map can be calculated using the following formula:

[0045]

[0046] The τ here represents any one of the knowledge points in the current encoded sequence, α i,jThis represents the correlation between the i-th knowledge point and the j-th knowledge point. The element in the i-th row and j-th column of the generated attention graph represents the influence weight of the j-th knowledge point in the knowledge point sequence on the i-th knowledge point.

[0047] By determining the feature matrix corresponding to the knowledge point sequence, the feature matrix includes a key matrix K, a query matrix Q, and a value matrix V; the feature value Q of the i-th knowledge point in the query matrix Q is obtained. i And, obtain the eigenvalue K of the j-th knowledge point in the key matrix K. j Wherein, the j-th knowledge point is the preceding knowledge point related to the i-th knowledge point; the time difference Δ(t) between the i-th knowledge point and the j-th knowledge point is determined. i ,t j ), determine the decay function γ(t) that includes the time difference and the decay parameter θ. i ,t j ,θ), where θ is a learnable decay parameter; based on the eigenvalue Q i eigenvalue K j The attenuation function γ(t) i ,t j The attention map is generated by the matrix V and the value matrix V, thereby strengthening the performance information of recently related questions and other questions by taking into account the time interval factor of learner forgetting behavior. This results in the calculation of an attention map containing accurate features, which is beneficial for training a high-precision model and subsequent prediction.

[0048] In one embodiment, the original attention scores s are fused. i,j and the decay function γ(t) i ,t j Generate new attention scores s, θ) i,j,γ At that time, different methods can be adopted depending on the actual situation.

[0049] For example, the original attention score s can be... i,j With the decay function γ(t) i ,t j Multiplying θ and θ together generates a new attention score s. i,j,γ The new attention score s can be generated in the following way. i,j,γ :

[0050]

[0051] For example, the original attention score s can also be... i,j With the decay function γ(t) i ,t j Subtracting θ from θ generates a new attention score s.i,j,γ The new attention score s can be generated in the following way. i,j,γ :

[0052]

[0053] Furthermore, it should be noted that the aforementioned method provides one specific form. However, in practical applications, the function form containing the time difference and attenuation parameter θ can be set in various styles according to actual needs, including exponential function form, power function form, and absolute value ratio form, etc.

[0054] For example, the decay function γ(t) in the form of an exponential function that includes the time difference and the decay parameter θ. i ,t j In the equation γ(t,θ), the time difference and the decay parameter θ are the decay function γ(t) in the form of an exponential function. i ,t j The independent variable in (θ) can be in the form described above.

[0055] For example, in the power function form of the decay function γ(t) which includes the time difference and the decay parameter θ i ,t j In the equation γ(t,θ), the time difference and the attenuation parameter θ are the attenuation function γ(t) in power function form. i ,t j When the independent variable is in (θ), its form can be as follows:

[0056] For example, the decay function γ(t) i ,t j (θ) can also be expressed as the ratio of the time difference to the attenuation parameter θ, as shown below:

[0057] In practical applications, besides considering the impact of the time interval between the current problem and the previous related problem on weight decay, we also need to consider the impact of other problems between the current problem and the previous related problem on weight decay, such as... Figure 2 As shown, other problems (i.e., knowledge points corresponding to time tk) may exist between the i-th problem and the related j-th problem. Therefore, in one embodiment, the present application also considers whether the i-th knowledge point and the j-th knowledge point are adjacent in the sequence. When they are not adjacent, other knowledge points existing between the i-th knowledge point and the j-th knowledge point are determined, and the decay function γ(t) is affected by these other knowledge points. i ,t j Corrections are made for θ).

[0058] Since prolonged periods without studying can lead to memory decline, when the time interval between the i-th question and the previously related j-th question is long, the attenuation of attention weight is greater if the learner has not studied other questions during this period. Conversely, if the learner is still studying other k-th questions (which can be multiple) during this period, the attenuation of attention weight is smaller.

[0059] Based on this, the time difference Δ(t) between the i-th knowledge point and the other knowledge points can be determined. i ,t k ), where Δ(t) i ,t k )=|t i -t k |, and thus determine the Δ(t) i ,t k ) and the Δ(t) i ,t j The ratio P of ) is based on the ratio of P to the decay function γ(t) i ,t j Corrections are made for θ).

[0060] The ratio P actually reflects whether learners are continuously learning other problems within the time interval of learning the same related problem. Therefore, if there are no other knowledge points between the i-th problem and the j-th problem, the corresponding attenuation coefficient remains unchanged. Conversely, if there are other knowledge points between the i-th problem and the j-th problem, the attenuation coefficient should be increased to make the attenuation of attention weight smaller.

[0061] Based on this, according to the stated Δ(t) i ,t k ) and the Δ(t) i ,t j The ratio of ) to the decay function γ(t) i ,t j When correcting ,θ), the following method can be used: determine the attenuation function γ(t) containing λ. i ,t j ,θ) and the correction term φ of P, where λ is a learnable correction parameter; the correction term φ and the decay function γ(t) i ,t j ,θ) and γ(t) i ,t j ,t k The decay function γ(t,θ,λ) is determined to be the attenuation function. i ,t jThe correction value of φ is given by φ, where the magnitude of the correction term φ is positively correlated with the number of other knowledge points between the i-th knowledge point and the j-th knowledge point, and is also positively correlated with the ratio P.

[0062] For example, a correction term could take the form of:

[0063] t k It refers to the timestamp of other questions between the learner's answer to the current question and the previous related question, and λ is a learnable correction parameter.

[0064] Accordingly, based on the correction term φ, the attenuation function γ(t) i ,t j When correcting ,θ), the following method can be used:

[0065] That is, when j and i are adjacent, no correction is made; when j and i are not adjacent, the attenuation coefficient is corrected based on the number of other problems between them and the relative time ratio P. This can more accurately reflect the impact of memory attenuation caused by whether learners have learned other knowledge points during the interval of learning the same related knowledge point.

[0066] During the training phase, after obtaining the attention map, the model can generate an output vector based on the attention map, decode the output vector to obtain a prediction vector, calculate the data residual, and train the trainable parameters in the model (including parameters such as θ and λ in the decay function) based on the data residual. After the model training is completed and used for prediction, the output vector can be generated based on the attention map, and the output vector can be decoded to predict the learner's mastery of the current knowledge point.

[0067] Based on the same idea, one or more embodiments of this specification also provide apparatus and devices corresponding to the above methods, such as... Figure 4 , Figure 5 As shown.

[0068] In the second aspect, such as Figure 4 As shown, Figure 4 A schematic diagram of a weight decay-based attention map generation device provided in this specification, applied to a knowledge point sequence containing knowledge points and time, the device comprising:

[0069] The feature determination module 401 determines the feature matrix corresponding to the knowledge point sequence, wherein the feature matrix includes a key matrix K, a query matrix Q, and a value matrix V;

[0070] Feature value acquisition module 403 acquires the feature value Q of the i-th knowledge point in the query matrix Q. iAnd, obtain the eigenvalue K of the j-th knowledge point in the key matrix K. j Wherein, the j-th knowledge point is the previous knowledge point related to the i-th knowledge point;

[0071] Attenuation module 405 determines the time difference Δ(t) between the i-th knowledge point and the j-th knowledge point. i ,t j ), determine the decay function γ(t) that includes the time difference and the decay parameter θ. i ,t j ,θ), where θ is a learnable decay parameter;

[0072] Attention map generation module 407, based on the feature value Q i eigenvalue K j The attenuation function γ(t) i ,t j The attention map is generated using the value matrix V, θ).

[0073] Optionally, the attention map generation module 407, based on the feature value Q i eigenvalue K j Generate raw attention scores s i,j ; Fuse the original attention scores s i,j and the decay function γ(t) i ,t j Generate new attention scores s, θ) i,j,γ .

[0074] Optionally, the attention map generation module 407 generates the original attention score s i,j With the decay function γ(t) i ,t j Multiplying θ and θ together generates a new attention score s. i,j,γ Alternatively, the original attention score s i,j With the decay function γ(t) i ,t j Subtracting θ from θ generates a new attention score s. i,j,γ .

[0075] Optionally, the attenuation module 405 determines an attenuation function γ(t) in the form of an exponential function, which includes the time difference and the attenuation parameter θ. i ,t j ,θ), where the time difference and attenuation parameter θ are the attenuation function γ(t) in exponential form. i ,t j The independent variable in ,θ); or, determine the decay function γ(t) in power function form that includes the time difference and the decay parameter θ. i,t j ,θ), where the time difference and attenuation parameter θ are the attenuation function γ(t) in power function form. i ,t j The independent variable in ,θ).

[0076] Optionally, the device further includes a correction module 409, which determines whether the i-th knowledge point and the j-th knowledge point are adjacent in the sequence; if they are not adjacent, it determines other knowledge points existing between the i-th knowledge point and the j-th knowledge point, and adjusts the decay function γ(t) according to the other knowledge points. i ,t j Corrections are made for θ).

[0077] Optionally, the correction module 409 determines the time difference Δ(t) between the i-th knowledge point and the other knowledge points. i ,t k ); Determine the Δ(t) i ,t k ) and the Δ(t) i ,t j The ratio P of ) is based on the ratio of P to the decay function γ(t) i ,t j Corrections are made for θ).

[0078] Optionally, the correction module 409 determines that λ and the attenuation function γ(t) are included. i ,t j ,θ) and the correction term φ of P, where λ is a learnable correction parameter; the correction term φ and the decay function γ(t) i ,t j ,θ) and γ(t) i ,t j ,t k The decay function γ(t,θ,λ) is determined to be the attenuation function. i ,t j The correction value for ,θ).

[0079] Optionally, the device further includes a training module 411 for training the attenuation parameter θ based on the attention map.

[0080] In the third aspect, such as Figure 5 As shown, Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this specification. The device includes:

[0081] At least one processor; and,

[0082] A memory communicatively connected to the at least one processor; wherein,

[0083] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method as described in the first aspect.

[0084] In a fourth aspect, based on the same idea, embodiments of this specification also provide a non-volatile computer storage medium corresponding to the above method, storing computer-executable instructions, which, when read by a computer from the storage medium, cause one or more processors to execute the method described in the first aspect.

[0085] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0086] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0087] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0088] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware.

[0089] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the embodiments of this specification can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0090] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0091] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0092] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0093] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0094] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0095] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0096] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0097] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0098] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, devices, and non-volatile computer storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0099] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0100] The above description is merely one or more embodiments of this specification and is not intended to limit this specification. Various modifications and variations can be made to the one or more embodiments of this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of one or more embodiments of this specification should be included within the scope of the claims of this specification.

Claims

1. A method for generating attention maps based on weight decay, applied to a sequence of knowledge points containing knowledge points and time; time represents the timestamp when the learner answers a question; The knowledge points include: the learner's question information, the timestamp of the learner's answer, and the learner's response information; the method includes: Determine the feature matrix corresponding to the knowledge point sequence, wherein the feature matrix includes a key matrix K, a query matrix Q, and a value matrix V; Obtain the eigenvalue Q of the i-th knowledge point in the query matrix Q. i And, obtain the eigenvalue K of the j-th knowledge point in the key matrix K. j Wherein, the j-th knowledge point is the previous knowledge point related to the i-th knowledge point; Determine the time difference between the i-th knowledge point and the j-th knowledge point. Determine whether the time difference and attenuation parameters are included. decay function , wherein The decay parameter is learnable; According to the characteristic value Q i eigenvalue K j The attenuation function Generating an attention map from the value matrix V includes: determining the eigenvalues ​​K. j transpose vector According to the characteristic value Q i and the transpose vector Generate raw attention score ; Fuse the original attention scores and the decay function To induce memory decay and generate new attention scores By using the new attention score Multiplying the value matrix V with the attention map generates an attention map.

2. The method as described in claim 1, wherein, Fusion of the original attention scores and the decay function To induce memory decay and generate new attention scores ,include: The original attention score With the decay function Multiplication generates new attention scores ;or, The original attention score With the decay function Subtraction generates a new attention score .

3. The method as described in claim 1, wherein, Determine whether the time difference and attenuation parameters are included. decay function ,include: Determine whether the time difference and attenuation parameters are included. Decreasing function in exponential form Wherein, the time difference and attenuation parameter The decay function is of the form of the exponential function. The independent variable in; or, Determine whether the time difference and attenuation parameters are included. decay function in power form Wherein, the time difference and attenuation parameter The decay function is of the power function form. The independent variable in the equation.

4. The method of claim 1, wherein, The method further includes: Determine whether the i-th knowledge point and the j-th knowledge point are adjacent in the sequence; When they are not adjacent, determine the other knowledge points that exist between the i-th knowledge point and the j-th knowledge point, and apply the decay function based on the other knowledge points. Make corrections.

5. The method as described in claim 4, wherein, Based on the other knowledge points mentioned above, the attenuation function Make corrections, including: Determine the time difference between the i-th knowledge point and the other knowledge points. ; Determine the With the The ratio P, based on the P with respect to the decay function Make corrections.

6. The method of claim 5, wherein, According to the above With the The ratio of the decay function Make corrections, including: Determine if it contains The attenuation function and the correction term of P , wherein These are learnable correction parameters; The correction item and the decay function and Determined as the attenuation function The correction value.

7. The method of claim 1, further comprising: The attenuation parameter is based on the attention map. Conduct training.

8. A device for generating attention maps based on weight decay, applied to a sequence of knowledge points containing knowledge points and time; the time represents the timestamp when the learner answers a question; The knowledge points include: question information of the learner's answer, timestamp of the learner's answer, and response information of the learner's answer; the device includes: The feature determination module determines the feature matrix corresponding to the knowledge point sequence, wherein the feature matrix includes a key matrix K, a query matrix Q, and a value matrix V; The feature value acquisition module acquires the feature value Q of the i-th knowledge point in the query matrix Q. i And, obtain the eigenvalue K of the j-th knowledge point in the key matrix K. j Wherein, the j-th knowledge point is the previous knowledge point related to the i-th knowledge point; The attenuation module determines the time difference between the i-th knowledge point and the j-th knowledge point. Determine whether the time difference and attenuation parameters are included. decay function , wherein The decay parameter is learnable; The attention map generation module, based on the feature value Q i eigenvalue K j The attenuation function Generating an attention map from the value matrix V includes: determining the eigenvalues ​​K. j transpose vector According to the characteristic value Q i and the transpose vector Generate raw attention score ; Fuse the original attention scores and the decay function To induce memory decay and generate new attention scores By using the new attention score Multiplying the value matrix V with the attention map generates an attention map.

9. An electronic device, comprising: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, such that the at least one processor can perform the method as described in any one of claims 1 to 7.

Images