Student behavior detection method based on personalized federal learning

Through personalized federated learning methods, the collaborative training of multiple schools is realized in student behavior detection, and a personalized model adapted to the specific data distribution of each school is generated, which solves the problems of data dispersion and heterogeneity, ensures privacy protection, and improves teaching quality .

CN118981673AInactive Publication Date: 2024-11-19JIANGSU SECOND NORMAL UNIVERSITY
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410949492.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-15
Publication Date
2024-11-19
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing student behavior detection model requires a large amount of manual annotation of data during the training process, resulting in high cost and inconsistent data quality. It also has privacy and ethical problems, making it difficult to obtain high-quality annotation data in large-scale applications.

Method used

Using a method based on personalized federated learning, through distributed data training and adaptive fusion of personalized methods, schools allow each school to train models locally, just share model parameters instead of raw data, to generate personalized personalized ones that are adapted to the distribution of specific data in each school. To ensure privacy protection and improve the generalization capabilities of the model.

Benefits of technology

It effectively reduces the adverse effects of data heterogeneity, improves the accuracy and robustness of the model, improves the effectiveness of educators in monitoring students' attention and learning situations, and improves the quality of teaching.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118981673A_ABST
    Figure CN118981673A_ABST
Patent Text Reader

Abstract

The invention discloses a student behavior detection method based on personalized federal learning, and relates to the technical field of behavior detection, and the method comprises the following steps: a participating school collects and marks student behavior data to construct a local data set; the central server distributes a global encoder and a global classifier; the participating schools train the model by using a self-adaptive fusion personalized method; the participating schools upload the updated local encoders and classifiers to a central server, and the central server calculates a global encoder and a global classifier in the next round; and repeating the steps S2-S4 until the model converges. According to the student behavior detection method based on personalized federal learning, cooperative training of a plurality of school student behavior detection models is realized, so that a personalized model adapted to specific data distribution of each school is generated, privacy protection is ensured, adverse effects caused by data isomerism are effectively reduced, and the student behavior detection efficiency is improved. The monitoring effect of educators on the attention and learning conditions of students is improved, and the teaching quality is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of behavior detection, and in particular to a student behavior detection method based on personalized federated learning. Background Art

[0002] With the rapid development of artificial intelligence technology, the education field is gradually applying artificial intelligence technology to student behavior detection, a key link in educational data analysis, to provide real-time feedback on students' attention, participation, and emotional state, helping teachers to adjust teaching strategies in a timely manner. However, the current training of student behavior detection models requires a large amount of manually labeled data, which faces considerable challenges in practical applications.

[0003] First, manually labeling data requires a lot of manpower and time. The process of collecting and labeling student behavior data is cumbersome and complicated, requiring professionals to process and verify the data, resulting in high overall costs. Especially in large-scale application scenarios, the labeling workload is huge, and it is difficult to achieve continuous and effective labeling. In addition, the quality of manually labeled data varies, and subjective factors and human errors may lead to inconsistent labeling results, affecting the accuracy and robustness of model training.

[0004] In addition, obtaining manually annotated data also faces privacy and ethical issues. Student behavior data is sensitive information, and the collection and use of data must strictly comply with relevant laws, regulations and ethical standards to ensure that student privacy is protected. These restrictions increase the difficulty of data collection and make it more complicated to obtain high-quality annotated data on a large scale in actual teaching environments.

[0005] In order to solve the above problems, a student behavior detection method based on federated learning is proposed to solve the data dispersion problem through distributed data training. In order to overcome the problem of data heterogeneity: differences in acquisition equipment (the cameras and sensor equipment used in different schools may be different, resulting in different image resolutions, color depths, etc.), physical environment differences (such as classroom layout, lighting conditions, background complexity, etc.), and student characteristics (appearance, clothing, behavior patterns, etc.), a personalized federated learning method is used to enhance the global model learning effect. Personalized federated learning allows each school to train the model locally, and only needs to share model parameters instead of raw data, thereby protecting student privacy while aggregating diverse data from different schools, improving the generalization ability of the model, and learning a personalized model suitable for the school's data, greatly improving educators' monitoring of students' attention and learning, and improving teaching quality. Summary of the invention

[0006] The purpose of the present invention is to provide a student behavior detection method based on personalized federated learning, which overcomes the problem of poor model performance caused by data dispersion and data heterogeneity in the student behavior detection scenario. By adopting a personalized federated training framework, collaborative training of multiple schools is achieved to generate a personalized model that adapts to the specific data distribution of each school, while ensuring privacy protection and effectively reducing the adverse effects of data heterogeneity.

[0007] To achieve the above object, the present invention provides a student behavior detection method based on personalized federated learning, comprising the following steps:

[0008] S1, participating schools collect and annotate student behavior data to build local datasets;

[0009] S2, the central server distributes the global encoder and global classifier;

[0010] S3,participating schools use adaptive fusion personalized method to train the model;

[0011] S4, the participating schools upload the updated local encoders and classifiers to the central server, and the central server calculates the next round of global encoders and global classifiers;

[0012] S5. Repeat steps S2 to S4 until the model converges.

[0013] Preferably, the operation steps of step S3 are as follows:

[0014] S31, using the global encoder With the global classifier Override local encoder With local classifier

[0015] S32, update the initial local encoder With the initial local classifier Get the first local encoder With the first local classifier

[0016] S33, update fusion weight Get the first fusion weight To learn a global encoder With the first local encoder The optimal combination of . ;

[0017] Preferably, the specific steps of step S31 include: in communication round t, participating school k first uses the global encoder and global classifier to cover the local encoder and local classifier of the previous round to obtain the initial local encoder of this round With the initial local classifier

[0018] Preferably, the specific steps of step S32 include:

[0019] Sample x first passes through the initial local encoder Get Features Then the features Input local classifier Get classification results The first classification loss is then calculated by the cross entropy loss function And update the initial local encoder with this With the initial local classifier Get the first local encoder With the first local classifier And upload to the central server;

[0020] First classification loss The calculation process is as follows:

[0021]

[0022] in, is the local data set of participating user k; l is the sample number; is the sample image corresponding to the sample number l in the local dataset; is the corresponding category label; (x, y) represents the image and label pair in the local dataset; y is the label; N is the number of behavior categories; 1 is the indicator function; is the initial local classifier The output probability distribution vector The probability value corresponding to category c;

[0023] The updating process of the model is:

[0024]

[0025] Among them, α is the learning rate; represents a differential operator.

[0026] Preferably, the specific steps of step S33 include:

[0027] School k fixes the first local classifier parameters to fusion weights For the global encoder With the first

[0028] Ground Encoder Perform model interpolation, where the fusion weights The shape is consistent with the global encoder and the local encoder, and the element range is between 0 and 1. The fusion encoder obtained after interpolation is used to calculate the second classification loss again with the cross entropy function. And update the fusion weights accordingly Get the first fusion weight

[0029] Second category loss The calculation process is as follows:

[0030]

[0031] in, is the fusion encoder obtained after interpolation, O is an all-one matrix, and its shape is consistent with the shape of the encoder. ⊙ is the Hadamard product, which means the multiplication of corresponding positions. Represents the probability value corresponding to category c in the model output probability distribution vector.

[0032] The fusion weight update process is:

[0033]

[0034] Among them, α is the learning rate; represents a differential operator.

[0035] Preferably, in step S4, the specific operation of the central server calculating the next round of global encoder and classifier is as follows:

[0036]

[0037] Among them, K is the total number of participating schools; and The global encoder and global classifier of round t+1 calculated for the server.

[0038] Therefore, the present invention adopts the above-mentioned student behavior detection method based on personalized federated learning and adopts personalized federated learning technology to realize the collaborative training of student behavior detection models of multiple schools to generate a personalized model adapted to the specific data distribution of each school, while ensuring privacy protection, effectively reducing the adverse effects of data heterogeneity, and improving educators' monitoring of students' attention and learning conditions, thereby improving teaching quality.

[0039] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 The present invention is a flowchart of an embodiment of a method for detecting student behavior based on personalized federated learning. DETAILED DESCRIPTION

[0041] The technical solution of the present invention is further described below through the accompanying drawings and embodiments.

[0042] Unless otherwise defined, the technical terms or scientific terms used in the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0043] Example

[0044] See also Figure 1 , the present invention provides a student behavior detection method based on personalized federated learning, comprising the following steps:

[0045] S1. Participating schools collect and annotate student behavior data to build a local data set. The local data set may include video and audio data collected by sensor devices, which carry features such as students' movements, expressions, and speeches that can detect their behaviors.

[0046] S2. The central server distributes the global encoder and the global classifier.

[0047] S3. Participating schools use adaptive fusion personalized methods to train models.

[0048] The operation steps of the above step S3 are as follows:

[0049] S31, using the global encoder With the global classifier Override local encoder With local classifier In communication round t, participating school k first uses the global encoder and global classifier to cover the local encoder and local classifier of the previous round to obtain the initial local encoder of this round With the initial local classifier

[0050] S32, update the initial local encoder With the initial local classifier Get the first local encoder

[0051] With the first local classifier School k updates the initial local encoder on local data With initial

[0052] Local Classifier Get the first local encoder With the first local classifier And upload to the central server.

[0053] The specific steps of the above step S32 include:

[0054] Sample x first passes through the initial local encoder Get Features Then the features Input local classifier Get classification results Then, according to the corresponding specific behavior category label, the first classification loss is calculated by the cross entropy loss function And update the initial local encoder with this With initial

[0055] Local Classifier Get the first local encoder With the first local classifier And upload to the central server.

[0056] First classification loss The calculation process is as follows:

[0057]

[0058] in, is the local data set of participating user k; l is the sample number; is the sample image corresponding to the sample number l in the local dataset; is the corresponding category label; (x, y) represents the image and label pair in the local dataset; y is the label; N is the number of behavior categories; 1 is the indicator function; is the initial local classifier The output probability distribution vector The probability value corresponding to category c;

[0059] The updating process of the model is:

[0060]

[0061] Among them, α is the learning rate; represents a differential operator.

[0062] S33, update fusion weight Get the first fusion weight To learn a global encoder With the first

[0063] Local encoder The optimal combination of school k fixes the first local classifier parameters to fusion weights

[0064] For the global encoder With the first local encoder Perform model interpolation, where the fusion weights The shape is consistent with the global encoder and the local encoder, and the element range is between 0 and 1, and the interpolation model is used.

[0065] The second classification loss is calculated again using the cross entropy function And update the fusion weights accordingly Get first

[0066] Fusion weight

[0067] Second category loss The calculation process is as follows:

[0068]

[0069] in, is the fusion encoder obtained after interpolation, O is an all-one matrix, and its shape is consistent with the shape of the encoder. ⊙ is the Hadamard product, which means the multiplication of corresponding positions. Represents the probability value corresponding to category c in the model output probability distribution vector.

[0070] The fusion weight update process is:

[0071]

[0072] Among them, α is the learning rate; represents a differential operator.

[0073] S4. The participating schools upload the updated local encoders and classifiers to the central server, and the central server calculates the next round of global encoders and global classifiers. The specific operations are as follows:

[0074]

[0075] Among them, K is the total number of participating schools; and The global encoder and global classifier of round t+1 calculated for the server.

[0076] S5. Repeat steps S2 to S4 until the model converges.

[0077] Therefore, the present invention adopts the above-mentioned student behavior detection method based on personalized federated learning, which can realize the collaborative training of student behavior detection models of multiple schools to generate a personalized model adapted to the specific data distribution of each school, while ensuring privacy protection, effectively reducing the adverse effects of data heterogeneity, and improving educators' monitoring of students' attention and learning conditions, thereby improving teaching quality.

[0078] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A student behavior detection method based on personalized federated learning, It is characterized in that The following steps are involved: S1, participating schools collect and annotate student behavior data to build local datasets; S2, the central server distributes the global encoder and global classifier; S3,participating schools use adaptive fusion personalized method to train the model; S4, the participating schools upload the updated local encoders and classifiers to the central server, and the central server calculates the next round of global encoders and global classifiers; S5. Repeat steps S2 to S4 until the model converges.

2. According to claim 1, a method for detecting student behavior based on personalized federated learning, It is characterized in that The operation steps of step S3 are as follows: S31, using the global encoder With the global classifier Override local encoder With local classifier S32, update the initial local encoder With the initial local classifier Get the first local encoder With the first local classifier S33, update fusion weight Get the first fusion weight To learn a global encoder With the first local encoder The optimal combination.

3. According to claim 2, a method for detecting student behavior based on personalized federated learning, It is characterized in that The specific steps of step S31 include: in communication round t, participating school k first uses the global encoder and global classifier to cover the local encoder and local classifier of the previous round to obtain the initial local encoder of this round With the initial local classifier 4. According to claim 3, a method for detecting student behavior based on personalized federated learning, It is characterized in that The specific steps of step S32 include: Sample x first passes through the initial local encoder Get Features Then the features Input local classifier Get classification results The first classification loss is then calculated by the cross entropy loss function And update the initial local encoder with this With the initial local classifier Get the first local encoder With the first local classifier And upload to the central server; First classification loss The calculation process is as follows: in, is the local data set of participating user k; l is the sample number; is the sample image corresponding to the sample number l in the local dataset; is the corresponding category label; (x, y) represents the image and label pair in the local dataset; y is the label; N is the number of behavior categories; 1 is the indicator function; is the initial local classifier The output probability distribution vector The probability value corresponding to category c; The updating process of the model is: Among them, α is the learning rate; represents a differential operator.

5. According to claim 4, a method for detecting student behavior based on personalized federated learning, It is characterized in that The specific steps of step S33 include: fixing the first local classifier parameter k to fuse the weight For the global encoder With the first local encoder Perform model interpolation, where the fusion weights The shape is consistent with the global encoder and the local encoder, and the element range is between 0 and 1. The fusion encoder obtained after interpolation is used to calculate the second classification loss again with the cross entropy function. And update the fusion weights accordingly Get the first fusion weight Second category loss The calculation process is as follows: in, is the fusion encoder obtained after interpolation, O is an all-one matrix, and its shape is consistent with the shape of the encoder. ⊙ is the Hadamard product, which means the multiplication of corresponding positions. Represents the probability value corresponding to category c in the model output probability distribution vector. The fusion weight update process is: Among them, α is the learning rate; θ represents the differential operator.

6. A method for detecting student behavior based on personalized federated learning according to claim 5, It is characterized in that In step S4, the specific operation of the central server calculating the next round of global encoder and classifier is as follows: Among them, K is the total number of participating schools; and The global encoder and global classifier of round t+1 calculated for the server.