Robust federated knowledge tracking incentive distribution-based academic record prediction method and medium

Abstract

The invention relates to an academic record prediction method based on robust federated knowledge tracking incentive distribution and a medium, and the method comprises the following steps: taking a school as a participant, and carrying out federated knowledge tracking training; according to the training time delay of the participants, making a bandwidth allocation decision by using a greedy algorithm, so that all the participants synchronously complete model updating and uploading, and all the participants execute federal knowledge tracking training in parallel; after federated knowledge tracking training is completed, the contribution level of each participant to the global knowledge tracking model is evaluated, a reinforcement learning method is used to generate an incentive distribution strategy, a feedback utility total reward of all participants under the incentive distribution strategy is calculated, a learning network is updated, and finally the incentive distribution strategy with the highest feedback utility total reward is selected. And using the global knowledge tracking model to predict the future answering accuracy of each student of each school in the course. Compared with the prior art, the method has the advantages of high overall effectiveness, high accuracy, high practicability, high robustness and the like.

Description

Technical Field

[0001] The present invention relates to the field of federated learning technology, and in particular to a learning performance prediction method and medium based on robust federated knowledge tracking incentive allocation. Background Art

[0002] In recent years, intelligent education has received widespread attention. As an important branch of intelligent education, the knowledge tracking model can predict the accuracy of students' answers to new knowledge points based on their historical learning data, provide teachers with accurate teaching decision support, and improve learning outcomes.

[0003] Traditional knowledge tracking concentrates the data of various schools in a data center for centralized training, which brings the risk of data privacy leakage. In order to protect privacy, schools are reluctant to share local data, resulting in data silos. Later, federated learning, as a distributed collaborative training paradigm, alleviates the problem of data privacy leakage and breaks the data silos by uploading model parameters instead of original data.

[0004] However, in federated knowledge tracking, schools as participants will consume their own resources when participating in federated knowledge tracking training. If there is no appropriate reward compensation, participants are unwilling to continue to participate in training. In addition, for privacy reasons, participants are unwilling to disclose their resource information to the data center. For the design of incentives for federated knowledge tracking, optimizing the overall utility of participants and ensuring the performance of the global model are the main goals. However, in addition to considering the formulation of the reward allocation strategy, the design of the incentive mechanism is also constrained by malicious or unfair factors that may exist during the training process. First, malicious participants will interfere with the performance of the global model and tend to grab more rewards; second, in the case of federated knowledge tracking using synchronous training, some participants will enter the waiting time. The existence of waiting time will make participants expect higher rewards, affecting the design of subsequent incentive mechanisms; finally, in the case of unknown specific information of participants, considering the participant's cost and contribution level, the exploration of reward allocation strategy is a complex optimization problem. In recent years, although some related work has studied the design of incentive mechanisms in federated learning, they have not considered the information asymmetry between the data center and participants, and cannot effectively achieve incentives under incomplete information, and the overall utility of participants is poor. Summary of the invention

[0005] The purpose of the present invention is to overcome the defect of poor overall utility of the above-mentioned prior art and to provide a learning performance prediction method and medium based on robust federated knowledge tracking incentive allocation.

[0006] The purpose of the present invention can be achieved by the following technical solutions:

[0007] A learning performance prediction method based on robust federated knowledge tracking incentive allocation includes the following steps:

[0008] Collect the learning and answering data of each student of the same course in each school, and use each school as a participant in the federal knowledge tracking training to conduct federal knowledge tracking training;

[0009] Based on the training latency of the participants, a greedy algorithm is used to make bandwidth allocation decisions, so that all participants can complete model updates and uploads synchronously, and each participant can perform federated knowledge tracking training in parallel;

[0010] After the federated knowledge tracking training is completed, the contribution level of each participant to the global knowledge tracking model is evaluated, and the incentive allocation strategy is generated using the reinforcement learning method. The total feedback utility reward of all participants under the incentive allocation strategy is calculated, the learning network is updated, and this step is repeated. Finally, the incentive allocation strategy with the highest total feedback utility reward is selected, and the global knowledge tracking model is used as the prediction model;

[0011] Use the prediction model to predict the future answer accuracy of each student in this course in each school.

[0012] Furthermore, the greedy algorithm includes the following steps:

[0013] According to the total bandwidth allocated to each participant, the total delay of all participants is calculated to obtain the shortest delay T min ;

[0014] At the shortest delay T min Under the constraint of , solve the bandwidth resources of each participant; if the sum of the bandwidth resources of each participant exceeds the total bandwidth of the system, update the current shortest delay to T min +δ, where δ is the increment, and the bandwidth resources are recalculated until the sum of the bandwidth resources is less than or equal to the total system bandwidth, and the bandwidth allocation decision and the current shortest delay are obtained.

[0015] Furthermore, the total delay includes the training delay and communication delay The calculation expression of the total delay is:

[0016]

[0017]

[0018]

[0019]

[0020]

[0021]

[0022] Where η is the number of CPU cycles required by the participant to process a single data sample, is the size of the local dataset of participant j, f j is the CPU frequency used by participant j in training, M is the size of the uploaded model parameters, and the unit of measurement of M is bit. Rate j is the uplink data rate achievable by participant j, P is the communication power of the participant, B is the total system bandwidth, μ j is the bandwidth resource obtained by participant j, c j is the channel gain between participant j and the data center, N 0 is the power spectral density of additive white Gaussian noise during wireless transmission, is the total delay of the participants.

[0023] Furthermore, the calculation expression of the bandwidth resources obtained by the participants is:

[0024]

[0025] In the formula, μ j is the bandwidth resource obtained by participant j.

[0026] Furthermore, the process of generating an incentive allocation strategy includes the following steps:

[0027] Calculate the contribution level β of participant j to the global knowledge tracking model j ;

[0028] Calculate the total cost E required for participant j to train locally j , the total cost includes the CPU computation cost of a local update of participant j and communication costs

[0029] Calculate the total reward budget for the data center;

[0030] The sum of the participants’ local utilities is calculated based on the total reward budget, contribution level, and total cost.

[0031] Furthermore, the calculation expression of the total local utility of participants is:

[0032]

[0033] Where I is the total local utility, I j is the utility value of participant j, e is the natural logarithm, Δl j is the difference between the local knowledge tracking model and the global knowledge tracking model, m is the adjustment parameter, and E jis the total cost required for participant j to train locally, R j The incentive for participant j to participate in the training.

[0034] Furthermore, the reinforcement learning method includes setting the states, actions, and rewards of the incentive allocation strategy;

[0035] State: The current state at time t is defined as R j,t-1 represents the incentives allocated by the data center to participants at time t-1, r t-1 It represents the environmental reward after the incentive allocation strategy is executed at time t-1. At the beginning, all parameters are 0;

[0036] Action: Action

[0037] Reward: When the agent performs action a t After that, the environment reward is fed back to the agent. If the utility value of a participant is negative, the environment reward r t =0; otherwise, the environment rewards I j is the utility value of participant j.

[0038] Furthermore, at time t, the current state of the agent is s t = {R 1,t-1 ,R 2,t-1 ,…,R J,t-1 ,r t-1}, generate and execute incentive allocation strategy a t = {R 1,t ,R 2,t ,…,R J,t}, when executing the incentive allocation strategy a t After that, the environment takes the overall utility of the participants as the environmental reward r t Feedback is given to the agent, and then the agent changes from the current state s t Transfer to the next state s t+1 , the incentive mechanism enters the next time t+1 until it reaches the termination state.

[0039] Furthermore, the method further comprises the following steps:

[0040] Before the start of federated knowledge tracking training, multiple rounds of pre-training are carried out. According to the loss information of the candidate participants in each round of pre-training, the candidate participants are marked. By comparing the number of markings of the candidate participants with a preset threshold, the candidate participants whose markings exceed the threshold are excluded to obtain the participants.

[0041] A second aspect of the present invention is a storage medium having a program stored thereon, which, when executed, implements any of the above learning performance prediction methods based on robust federated knowledge tracking incentive allocation.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] 1) The present invention uses a bandwidth allocation decision based on a greedy algorithm to solve the shortest delay solution, so that all participants can synchronously complete the model update and upload, ensuring the synchronization of resource allocation of each participant, and then by calculating the utility and overall utility of each participant, using reinforcement learning to ensure that the participants can obtain non-negative utility while maximizing the overall utility, so that the data center can explore incentive allocation strategies under incomplete information, optimize the overall utility of participants, and improve the accuracy of the model.

[0044] 2) The present invention distributes incentives by calculating the total reward budget of the data center, the contributions and costs of the participants, which is more in line with the limiting factors of the actual application environment, and the obtained incentive distribution results have better practicality.

[0045] 3) The present invention uses pre-training to determine whether there are malicious participants among the candidate participants who interfere with the performance of the global model, and excludes malicious participants in advance to ensure the robustness of incentive allocation in subsequent processes. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a schematic diagram of the method of the present invention.

[0047] Figure 2 It is a flow chart of the present invention.

[0048] Figure 3 Comparison chart of accuracy on different data sets.

[0049] Figure 4 The figure below is a comparison chart of the total latency results on different data sets.

[0050] Figure 5 Comparison chart of incentives and overall utility results for different reward budgets and different data sets. DETAILED DESCRIPTION

[0051] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0052] Example 1

[0053] The training process of federated knowledge tracking is that the data center distributes the current global knowledge tracking model to participants (i.e., schools), and participants use local data for training and upload the trained local knowledge tracking model parameters to the data center for aggregation to generate a new global knowledge tracking model. In this process, a loss function needs to be defined, and the loss function is minimized by using the local data sets of the participants to achieve the learning purpose. Assume that the current federated knowledge tracking system consists of a data center and N schools as participants. The loss function is composed of To represent, where 1≤i≤N, the local data set owned by participant i is Its size is The loss function for this participant is:

[0054]

[0055] Among them, θ i is the local knowledge tracking model, represents the j-th data sample of participant i.

[0056] Global loss function It can be expressed as:

[0057]

[0058] The optimal Θ is obtained through the global loss function, and the federated average algorithm is used to approximate the optimal Θ. The average gradient of the local model aggregated by the data center is used for update:

[0059]

[0060] Among them, θ i,k represents the training parameters of participant i in the kth iteration, α represents the learning rate, and finally the data center globally aggregates the training parameters of the participants.

[0061] The present invention is a learning performance prediction method based on robust federated knowledge tracking incentive allocation, comprising the following steps:

[0062] S1: In the pre-training stage, the candidate participants are marked according to their loss information in each round of training, and the number of marks of the candidate participants is compared with the preset threshold, and the candidate participants whose marks exceed the threshold are excluded to obtain the participants;

[0063] In the τth round of pre-training, the current global knowledge tracking model is Θ τ , the loss on the data center test set is l τ , the local knowledge tracking model trained by candidate participant i is θ τ+1, the loss value on the data center test set is Ideally, the performance improvement of the global knowledge tracking model should be accompanied by a decrease in its loss value. If it is greater than the loss value lτ obtained by the global knowledge tracking model, we mark it as a suspicious malicious participant. Finally, the set of suspicious malicious participants marked in this round can be expressed as After T rounds of pre-training, we can get the number of times all participants are marked as suspicious malicious participants as {w 1 ,w 2 ,…,w N}, if the number of times it is marked exceeds the threshold ω we set in advance, it will be identified as a malicious participant. The final set of malicious participants can be expressed as And get the participants after excluding the malicious participants

[0064] S2: When the training data of the participants is unknown, the data center uses a greedy algorithm-based bandwidth allocation decision to solve the shortest delay solution of the total delay according to the training delay of the participants, so that all participants can synchronously complete the model update and upload and perform federated knowledge tracking training;

[0065] The data center uses a greedy method to explore the shortest latency solution that allows all participants to synchronously complete model updates and uploads, and obtains the corresponding bandwidth allocation decision.

[0066] Total delay T of participant j j Including training delay and communication delay Two parts, expressed as:

[0067]

[0068] The training delay is determined by the amount of local data and computing power of the participant. The CPU training delay of a local update by the participant is expressed as:

[0069]

[0070] Where η represents the number of CPU cycles required by a participant to process a single data sample, represents the size of the local dataset of participant j, f j It represents the computing resources of participant j when participating in the local knowledge tracking model training, that is, the CPU cycle frequency used to perform the task; after the local knowledge tracking model is trained, participant j uploads the model parameters to the data center. The required communication delay is related to the number of parameters. It can be expressed as:

[0071]

[0072] Where M represents the size of the uploaded model parameters (in bits), Rate j represents the uplink data rate that can be achieved by participant j, which is expressed according to the Shannon formula as follows:

[0073]

[0074] In the formula, P represents the communication power of the participant, B represents the total bandwidth of the system, represents the proportion of resource blocks allocated to participant j, c j represents the channel gain between participant j and the data center, N 0 Represents the power spectral density of additive white Gaussian noise during wireless transmission.

[0075] The existing most efficient resource management solution points out that when all participants complete the model update and upload synchronously and there are no redundant communication resources, the optimal resource management solution with the lowest latency is obtained.

[0076] We transform the high-dimensional bandwidth resource decision problem into a one-dimensional shortest delay problem. When the specific information of the participants is unknown, we use a greedy algorithm to solve it.

[0077] Assume that each participant gets all the bandwidth (μ j =1), and solve for the total delay of each participant in this case It can be expressed as:

[0078]

[0079] At this point, our shortest delay is Right now The bandwidth allocation decision is {μ 1 B,μ 2 B,…,μ J B}, the bandwidth allocated to the participants is μ j , which can be expressed as:

[0080]

[0081] If the total bandwidth allocated to participants exceeds our total bandwidth resources (i.e. ), we increase the time increment δ to the current shortest delay and obtain the new delay T′ min =T min +δ, then we calculate the resource block ratio μ required by participant j under the current delay constraint j ,if If it is not satisfied, we continue to increase the time increment δ for the current shortest delay and repeat the process until the proportion of resource blocks allocated to participants meets the total resource constraint. We get the shortest delay that allows all participants to complete model update and upload at the same time. And the bandwidth allocation decision under the current delay constraint

[0082] The bandwidth allocation decision-making solution based on the greedy algorithm is as follows. The algorithm performs the following steps in sequence:

[0083] 1) Assuming that the bandwidth resources allocated to each participant are the total bandwidth, calculate the total delay of all participants and get the shortest delay T min ;

[0084] 2) In the time delay T min Under the constraint of

[0085]

[0086] 3) If the bandwidth resources required by the participants exceed the total bandwidth of the system, that is, The current shortest delay is updated to T min ←T min +δ;

[0087] 4) Repeat the above two steps until the bandwidth allocation decision that does not exceed the total system bandwidth and the current shortest delay are obtained.

[0088] S3: After the federated knowledge tracking training is completed, the contribution level of each participant to the global knowledge tracking model is evaluated, and the incentive allocation strategy is generated using the reinforcement learning method. The total feedback utility reward of all participants under the incentive allocation strategy is calculated, the learning network is updated, and this step is repeated. Finally, the incentive allocation strategy with the highest total feedback utility reward is selected, and the global knowledge tracking model is used as the prediction model;

[0089] Under the condition of information asymmetry between participants and data centers, considering the costs and contribution levels of participants, and ensuring the individual rationality of participants, in order to obtain the incentive allocation strategy of participants in a continuous high-dimensional space, the actor-critic based deep reinforcement learning technology is used to design the incentive method, so that the data center can explore the incentive allocation strategy under incomplete information and optimize the overall utility of participants.

[0090] The utility I of participant j in participating in federated knowledge tracing training j It is the product of the difference between the incentives obtained by the participants from participating in the training and the cost of the training and the contribution level, expressed as

[0091] I j=β j (R j -E j )

[0092] Among them, β j R represents the contribution level of participant j to the global knowledge tracking model. The evaluation of the contribution level is related to the participant’s loss information. j represents the incentive obtained by participant j for participating in the training; E j represents the cost consumed by participant j in training, including computational cost and communication cost;

[0093] The contribution level of participant j to the global knowledge tracking model β j It is expressed as:

[0094]

[0095] Where e is the natural logarithm, Δl j is the difference between the local knowledge tracking model and the global knowledge tracking model, and m is the adjustment parameter.

[0096] The total cost E required for participant j to train locally j is the sum of computational cost and communication cost, which can be expressed as:

[0097]

[0098] The CPU computation cost of a local update by participant j It is expressed as:

[0099]

[0100] Where λ represents the effective capacitance coefficient of the participant, η represents the number of CPU cycles required by the participant to process a single data sample, represents the size of the local dataset of participant j, f j It represents the computing power resources allocated to participant j when participating in the local knowledge tracking model training, that is, the CPU frequency used to perform the task; after the local model is trained, the participant uploads the model parameters to the data center. The communication cost required is related to the upload time and power. The communication cost It can be expressed as:

[0101]

[0102] Where P represents the communication power of the participant, Represents the communication delay of the participants.

[0103] The total reward budget C of the data center is related to the amount of data of the participants and is expressed as:

[0104]

[0105] In the formula, It is the sum of the data volume of the local data sets of all participants participating in the training, and ρ is a tuning parameter.

[0106] The goal of the incentive mechanism is to maximize the overall utility of participants within a limited budget, where the overall utility of participants is the sum of the local utilities of all participants:

[0107]

[0108] In the formula, the total reward allocated to participants shall not exceed the reward budget of the data center, that is, In order to ensure individual rationality, the utility value of each participant must be non-negative, that is, I j ≥0.

[0109] Reinforcement learning provides an effective heuristic method for solving the approximate optimal solution of the incentive allocation strategy. In order to solve the problem of optimizing the overall utility of the participants, the present invention uses reinforcement learning to explore the incentive allocation strategy under the constraints of limited incentives, and then optimizes the overall utility of the participants. For the problem of solving the incentive allocation strategy in a high-dimensional continuous space, the reinforcement learning method based solely on value or strategy will have the problem of dimensionality explosion. The actor-critic method that integrates value-based value estimation and strategy-based policy optimization can effectively handle the incentive allocation problem in a high-dimensional continuous space. In this way, the optimization of the overall utility of the participants can be effectively solved in a high-dimensional continuous space.

[0110] The design details of the state, action, and reward of the incentive allocation strategy explored using reinforcement learning are as follows:

[0111] 1) State: In the case of unknown specific information of the participants, the current state of the agent (i.e., the data center) at time t is the observation of the agent in the environment (i.e., the participants), that is, at time t-1, the data center executes the incentive allocation action a t-1 and its feedback reward r t-1 , therefore, the current state at time t can be defined as Where R j,t-1 represents the incentives assigned by the agent to the participants at time t-1, r t-1 Indicates the incentive allocation strategy {R 1,t-1 ,R 2,t-1 ,…,R J,t-1}. In particular, at the beginning, Each parameter is 0.

[0112] 2) Action: According to the current state s at time t t, the agent generates incentive allocation actions under the total reward budget

[0113] 3) Reward: When the agent performs action a t = {R 1,t ,R 2,t ,…,R J,t}, the environment will feed back the environment reward to the agent. If the current incentive is assigned to action a t If the individual rationality of the participants cannot be guaranteed and the constraints are met, the environmental reward is 0, that is, r t = 0; if the individual rationality of the participants can be guaranteed and all constraints are met, the environmental reward is the overall utility of the participants, that is,

[0114] The method for solving the incentive allocation strategy based on reinforcement learning includes the following steps:

[0115] S31: The agent updates the actor-critic network;

[0116] S32: Actor networks in agents generate incentive allocation decisions;

[0117] S33: The agent executes the incentive allocation decision and allocates incentives to participants;

[0118] S34: The environment provides environmental reward feedback according to the incentive allocation strategy executed;

[0119] S35: The critic network in the agent estimates the value of the action;

[0120] S36: Repeat the above steps until the termination state is reached;

[0121] S37: Repeat the above steps until a predetermined number of training rounds is reached.

[0122] The incentive allocation strategy obtained according to the present invention completes the federated knowledge tracking training process of the model.

[0123] S4: Use the prediction model to predict the future answer accuracy of each student in each school in this course.

[0124] The global knowledge tracking model obtained through federated knowledge tracking training is used as a prediction model. Based on students' historical learning data, the accuracy rate / answering status of students facing new knowledge points is predicted, providing teachers with accurate teaching decision support and improving learning outcomes. In order to promote the application of knowledge tracking technology, multiple schools are jointly conducting federated knowledge tracking learning, which can improve the training quality and generalization ability of the model while ensuring privacy, and enhance its adaptability in different environments, thereby providing teachers with more accurate and personalized teaching decision support and optimizing teaching strategies in a targeted manner.

[0125] This embodiment further illustrates the technical effect of the present invention by means of simulation experiment comparison.

[0126] In the simulation experiment of this embodiment, the default settings of the system parameters are as follows. The federated knowledge tracking system includes 1 data center and 15 participants (i.e., N = 15), including 10 honest participants and 5 malicious participants. Three classic knowledge tracking datasets collected from the online education platform ASSISTments are used: ASSISTments2009, ASSISTments2015, and ASSISTments2017. For the computing power of the participant devices, the CPU frequency of the participant devices is uniformly set to [1.0, 2.0] GHz, and the number of CPU cycles required to process a data sample is η = 1×10 5 cycles / sample, the effective capacitance coefficient is λ=1×10 -28 For the communication between participants, the total system bandwidth B is set to 2 MHz and the channel gain c i Following the independent Rayleigh fading model, the power spectral density N of the additive Gaussian white noise during wireless transmission is 0 1×10 -9 W, and the communication power P is set to 1W.

[0127] The accuracy of the task, the total latency, and the overall utility of the participants are used as the comparison criteria for performance. First, in the comparison of task accuracy, the two comparison methods used as comparison criteria are traditional federated knowledge tracing training and local knowledge tracing training. In traditional federated knowledge tracing training, the data center assumes that each candidate participant is an honest participant and performs ordinary federated knowledge tracing training. In local knowledge tracing training, participants only perform local training without uploading or aggregating models, and the experimental accuracy result takes the average of the accuracy of all participants. Second, in the comparison of total latency, one of the comparison methods used as a standard is the traditional uniform bandwidth allocation method, which distributes the total bandwidth evenly to the participants. Finally, in the comparison of the overall utility of the participants, the five comparison methods used as comparison criteria are equal incentives, contribution incentives, data volume incentives, contract incentives, and random incentives. In equal incentives, the data center distributes the total reward evenly to the participants. In contribution incentives, the data center distributes the total reward according to the proportion of the participants' contribution to the global knowledge tracing model. In data volume incentives, the data center distributes the total reward according to the proportion of the data volume of the participants participating in the training. In contract incentives, data centers are clustered according to the contribution and data volume of participants, and corresponding incentives are allocated according to the contract items of different clusters. In random incentives, data centers randomly allocate incentives to participants under the total reward budget.

[0128] Comparison of the accuracy of the global knowledge tracking model of the three methods in the presence of malicious participants, such as Figure 3 As shown. Malicious participants will generate malicious interference information to interfere with the global knowledge tracking model, causing the model performance to degrade. The accuracy curves of the global knowledge tracking model of traditional federated knowledge tracking training and local knowledge tracking training fluctuate greatly, and the optimal accuracy is still lower than that of the present invention, among which the model accuracy curve of traditional federated knowledge tracking training fluctuates greatly. Comparison of the accuracy of the method of the present invention and traditional federated knowledge tracking training and local knowledge tracking training shows that the performance of the federated knowledge tracking system in the method of the present invention is more robust.

[0129] like Figure 4 As shown, the delays of completing training by the two methods are compared. Based on the optimal resource management solution, when all participants complete the model update and upload at the same time and there are no extra communication resources, the optimal bandwidth resource management solution can be obtained and the delay is the shortest. The high-dimensional bandwidth allocation problem is converted into a one-dimensional shortest delay problem. When the shortest delay is obtained, the optimal bandwidth allocation is also obtained. Compared with the traditional unified bandwidth allocation decision, the delay difference between the greedy bandwidth allocation method and the traditional unified bandwidth allocation decision shows that the method of the present invention can obtain the shortest total delay for task completion, and then obtain the optimal bandwidth allocation decision.

[0130] like Figure 5 As shown, the overall utility of participants under different reward budgets of the six methods is compared. The utility of participants takes into account the contribution and cost of the participants. Under the premise of ensuring the personal rationality of the participants, higher incentives are given to participants with high contributions, which brings higher benefits, that is, higher overall utility of the participants. Compared with other incentive methods, under different reward budgets, the method of the present invention can achieve better overall utility of participants. Since the method of the present invention adopts reinforcement learning, it continuously "trial and error" and learns through reinforcement learning, explores incentive allocation strategies, and optimizes the overall utility of participants. Therefore, the method of the present invention performs better than other methods.

[0131] In general, the method proposed in the present invention takes into account the limited reward budget of the data center and the information asymmetry between it and the participants, blocks the attacks of malicious participants, synchronously completes model updates and uploads, and explores incentive allocation strategies, thereby ensuring global model performance and optimizing the overall utility of participants.

[0132] In some embodiments, methods S1 to S4 may be implemented as a computer software program that is tangibly contained in a machine-readable medium, which may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or apparatus. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0133] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A learning performance prediction method based on robust federated knowledge tracking incentive allocation, characterized in that: The following steps are involved: Collect the learning and answering data of each student of the same course in each school, and use each school as a participant in the federal knowledge tracking training to conduct federal knowledge tracking training; Based on the training latency of the participants, a greedy algorithm is used to make bandwidth allocation decisions, so that all participants can complete model updates and uploads synchronously, and each participant can perform federated knowledge tracking training in parallel; After the federated knowledge tracking training is completed, the contribution level of each participant to the global knowledge tracking model is evaluated, and the incentive allocation strategy is generated using the reinforcement learning method. The total feedback utility reward of all participants under the incentive allocation strategy is calculated, the learning network is updated, and this step is repeated. Finally, the incentive allocation strategy with the highest total feedback utility reward is selected, and the global knowledge tracking model is used as the prediction model; Use the prediction model to predict the future answer accuracy of each student in this course in each school.

2. A learning performance prediction method based on robust federated knowledge tracking incentive allocation according to claim 1, characterized in that: The greedy algorithm includes the following steps: According to the total bandwidth allocated to each participant, the total delay of all participants is calculated to obtain the shortest delay T min ; At the shortest delay T min Under the constraint of , solve the bandwidth resources of each participant; if the sum of the bandwidth resources of each participant exceeds the total bandwidth of the system, update the current shortest delay to T min +δ, where δ is the increment, and the bandwidth resources are recalculated until the sum of the bandwidth resources is less than or equal to the total system bandwidth, and the bandwidth allocation decision and the current shortest delay are obtained.

3. A learning performance prediction method based on robust federated knowledge tracking incentive allocation according to claim 2, characterized in that: The total delay includes the training delay and communication delay The calculation expression of the total delay is: Where η is the number of CPU cycles required by a participant to process a single data sample, is the size of the local dataset of participant j, f j is the CPU frequency used by participant j in training, M is the size of the uploaded model parameters, and the unit of measurement of M is bit. Rate j is the uplink data rate achievable by participant j, P is the communication power of the participant, B is the total system bandwidth, μ j is the bandwidth resource obtained by participant j, c j is the channel gain between participant j and the data center, N0 is the power spectral density of additive Gaussian white noise during wireless transmission, is the total delay of the participants.

4. The learning performance prediction method based on robust federated knowledge tracking incentive allocation according to claim 3 is characterized in that: The calculation expression of the bandwidth resources obtained by the participant is: In the formula, μ j is the bandwidth resource obtained by participant j.

5. The learning performance prediction method based on robust federated knowledge tracking incentive allocation according to claim 1 is characterized in that: The process of generating the incentive allocation strategy comprises the following steps: Calculate the contribution level β of participant j to the global knowledge tracking model j ; Calculate the total cost E required for participant j to train locally j , the total cost includes the CPU computation cost of a local update of participant j and communication costs Calculate the total reward budget for the data center; The sum of the participants’ local utilities is calculated based on the total reward budget, contribution level, and total cost.

6. A learning performance prediction method based on robust federated knowledge tracking incentive allocation according to claim 5, characterized in that: The calculation expression of the total local utility of the participants is: Where I is the total local utility, I j is the utility value of participant j, e is the natural logarithm, Δl j is the difference between the local knowledge tracking model and the global knowledge tracking model, m is the adjustment parameter, and E j is the total cost required for participant j to train locally, R j The incentive for participant j to participate in the training.

7. The learning performance prediction method based on robust federated knowledge tracking incentive allocation according to claim 1 is characterized in that: The reinforcement learning method includes setting the states, actions, and rewards of the incentive allocation strategy; State: The current state at time t is defined as R j,t-1 represents the incentives allocated by the data center to participants at time t-1, r t-1 It represents the environmental reward after the incentive allocation strategy is executed at time t-1. At the beginning, all parameters are 0; Action: Action Reward: When the agent performs action a t After that, the reward is fed back to the agent. If there is a participant with a negative utility value, the reward r t =0; otherwise, reward I j is the utility value of participant j.

8. The learning performance prediction method based on robust federated knowledge tracking incentive allocation according to claim 7 is characterized in that: At time t, the agent’s current state is s t = {R 1,t-1 ,R 2,t-1 ,…,R J,t-1 ,r t-1 }, generate and execute incentive allocation strategy a t = {R 1,t ,R 2,t ,…,R J,t }, when executing the incentive allocation strategy a t After that, the environment takes the overall utility of the participants as the environmental reward r t Feedback is given to the agent, and then the agent changes from the current state s t Transfer to the next state s t+1 , the incentive mechanism enters the next time t+1 until it reaches the termination state.

9. The learning performance prediction method based on robust federated knowledge tracking incentive allocation according to claim 1 is characterized in that: The following steps are also included: Before the start of federated knowledge tracking training, multiple rounds of pre-training are carried out. According to the loss information of the candidate participants in each round of pre-training, the candidate participants are marked. By comparing the number of markings of the candidate participants with a preset threshold, the candidate participants whose markings exceed the threshold are excluded to obtain the participants.

10. A storage medium having a program stored thereon, characterized in that: When the program is executed, it implements a learning performance prediction method based on robust federated knowledge tracking incentive allocation as described in any one of claims 1-9.

Images