Personalized federated edge learning method, device, storage medium and computer equipment

Abstract

The personalized federated edge learning method, device, storage medium and computer device provided by the application can collect video data through a monitoring terminal when detecting abnormal crowd behavior, optimize an initial model into a local model, adapt to different monitoring scenes, aggregate the local models output by multiple client devices into an edge model based on an aggregation threshold and using an asynchronous cross-device model aggregation strategy through an edge server, reduce aggregation delay caused by device problems, periodically aggregate the edge models output by multiple edge servers into a global model through a cloud server using an asynchronous model aggregation strategy, improve global performance, and finally distribute the global model as the latest initial model back to each device to re-optimize and aggregate the model until the global model meets a preset iteration end condition, thereby ensuring that the entire system can continuously learn and adapt to new data and changes while maintaining the real-time performance and efficiency of the model.

Description

Technical Field

[0001] This application relates to the field of federated learning technology, and in particular to a personalized federated edge learning method, apparatus, storage medium and computer equipment. Background Technology

[0002] In the field of public safety, Abnormal Crowd Detection (ACBD) plays a crucial role. Its goal is to identify unusual behaviors, such as suspicious activities or potential security threats, from surveillance video. Therefore, with the acceleration of urbanization and the growth of public safety demands, ACBD technology is playing an increasingly important role in ensuring social stability and preventing crime.

[0003] However, in real-world monitoring scenarios, traditional ACBD methods face challenges due to the diversity of populations and the complexity of the scenarios, particularly in terms of data heterogeneity and personalized requirements. These methods not only require large amounts of labeled data for training but also lack personalized learning model design for specific population scenarios, making it difficult to meet the personalized detection needs of different monitoring scenarios. Consequently, their adaptability is poor, limiting detection accuracy and resulting in unsatisfactory detection effects. Summary of the Invention

[0004] The purpose of this application is to at least address one of the aforementioned technical deficiencies, particularly the technical deficiency that existing methods for detecting abnormal human behavior are unable to meet the personalized detection needs of different monitoring scenarios, have poor adaptability, and thus limit detection accuracy and result in unsatisfactory detection effects.

[0005] This application provides a personalized federated edge learning method, the method comprising:

[0006] Video data is collected through a monitoring terminal, and a local model is generated by optimizing a preset initial model based on the video data using a client device.

[0007] The edge server obtains local models output by multiple client devices based on an aggregation threshold, and uses an asynchronous cross-device model aggregation strategy to aggregate the local models to generate an edge model.

[0008] By periodically employing an asynchronous model aggregation strategy on cloud servers, edge models output from multiple edge servers are globally aggregated to generate a global model.

[0009] The global model is distributed back to each edge server and each client device as the latest initial model, and the video data collected through the monitoring terminal and subsequent steps are returned until the global model meets the preset iteration end condition.

[0010] Optionally, the step of using the client device to train and optimize a preset initial model based on the video data to generate a local model includes:

[0011] The client device is used to decompose the preset initial model into a network, resulting in a general feature extraction subnetwork and a personalized label mapping subnetwork.

[0012] The video data is subjected to anomaly detection through the general feature extraction sub-network, and multiple behavioral features are extracted based on the detection results.

[0013] The personalized label mapping subnetwork is used to map labels to each behavioral feature to obtain the behavioral label corresponding to each behavioral feature.

[0014] The initial model is optimized based on each behavioral feature and the corresponding behavioral label to obtain a local model.

[0015] Optionally, the step of optimizing the initial model based on each behavioral feature and the corresponding behavioral label to obtain a local model includes:

[0016] Each behavioral feature and its corresponding behavioral label are input into the initial model to obtain the predicted label output by the initial model based on each behavioral label.

[0017] The gradient results of the initial model are obtained by backpropagating each predicted label using the loss function.

[0018] The network parameters of the general feature extraction subnetwork and the personalized label mapping subnetwork are updated based on the gradient results using stochastic gradient descent and hyperparameter control methods.

[0019] Optionally, the calculation formula for backpropagation includes:

[0020]

[0021] In the formula, Indicates the use of loss function The calculated gradient results; where, Represents the gradient. This represents the general feature extraction subnetwork in the i-th client device. The network parameters in the (t-1)th iteration update; This represents the personalized tag mapping subnetwork in the i-th client device. The network parameters in the (t-1)th iteration update; Represents the loss function right and The partial derivatives of .

[0022] Optionally, the calculation formula for the stochastic gradient descent method includes:

[0023]

[0024]

[0025] In the formula, This represents the general feature extraction subnetwork in the i-th client device. Network parameters in the t-th update iteration; Indicates the learning rate; This represents the general feature extraction subnetwork. The gradient; Indicates from the loss function The calculated gradient results; This represents the personalized tag mapping subnetwork in the i-th client device. Network parameters in the t-th update iteration; This represents the general feature extraction subnetwork. The gradient.

[0026] Optionally, the calculation formula of the hyperparameter control method includes:

[0027]

[0028] In the formula, Represents the norm of the parameter vector; Regularization is used to control the personalized label mapping subnetwork. The intensity of personalized adjustments.

[0029] Optionally, the calculation formula for the asynchronous cross-device model aggregation strategy includes:

[0030]

[0031]

[0032] In the formula, and This represents the general feature extraction subnetwork in the edge model. and the personalized tag mapping subnetwork The network parameters in the (t+1)th iteration update; and This represents the general feature extraction subnetwork in the i-th client device. and the personalized tag mapping subnetwork Network parameters in the t-th iteration; j represents the j-th client device; This represents the set of client devices participating in the current aggregation. Indicates the number of parameters.

[0033] Optionally, the calculation formula for the asynchronous model aggregation strategy includes:

[0034]

[0035]

[0036] In the formula, and These represent the general feature extraction subnetworks in the global model. and the personalized tag mapping subnetwork Network parameters; Let i represent the set of all edge servers participating in the aggregation, and let i and j represent the i-th and j-th edge servers in the set of all edge servers participating in the aggregation, respectively.

[0037] This application also provides a personalized federal edge learning device, including:

[0038] The model optimization module is used to collect video data through the monitoring terminal and use the client device to optimize the preset initial model based on the video data to generate a local model.

[0039] The first aggregation module is used to obtain local models output by multiple client devices based on the aggregation threshold by the edge server, and to aggregate the local models using an asynchronous cross-device model aggregation strategy to generate the edge model.

[0040] The secondary aggregation module is used to periodically aggregate edge models output by multiple edge servers using an asynchronous model aggregation strategy employed by the cloud server to generate a global model.

[0041] The model iteration module is used to distribute the global model as the latest initial model back to each edge server and each client device, and return the video data collected through the monitoring terminal and its subsequent steps until the global model meets the preset iteration end conditions.

[0042] This application also provides a storage medium storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the personalized federated edge learning method as described in any of the above embodiments.

[0043] This application also provides a computer device, including: one or more processors, and memory;

[0044] The memory stores computer-readable instructions that, when executed by the one or more processors, perform the steps of the personalized federated edge learning method as described in any of the above embodiments.

[0045] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:

[0046] The personalized federated edge learning method, apparatus, storage medium, and computer equipment provided in this application can collect video data through monitoring terminals during abnormal crowd behavior detection. Client devices can then optimize a preset initial model based on the video data to generate a local model, enabling the model to adapt to different monitoring scenarios and improving detection accuracy and adaptability. Next, edge servers can acquire local models output by multiple client devices based on aggregation thresholds, and an asynchronous cross-device model aggregation strategy can be used to aggregate these local models to generate an edge model. This reduces aggregation latency caused by device issues and improves the efficiency and flexibility of edge model aggregation. Furthermore, cloud servers can periodically use an asynchronous model aggregation strategy to globally aggregate the edge models output by multiple edge servers, generating a global model. This allows the model to learn a wider range of crowd behavior patterns and features, improving overall performance. Finally, the global model can be distributed back to each edge server and client device as the latest initial model, and the system returns to the monitoring terminal to collect video data and perform subsequent steps until the global model meets the preset iteration termination conditions. This ensures that the entire system can continuously learn and adapt to new data and changes while maintaining the model's real-time performance and efficiency. Furthermore, the method described in this application can reduce reliance on a central server, lower data transmission overhead, and thus enhance the scalability and robustness of the entire monitoring system. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 A schematic diagram of the architecture of a personalized federated edge learning framework provided for an embodiment of this application;

[0049] Figure 2 A flowchart illustrating a personalized federated edge learning method provided in this application embodiment;

[0050] Figure 3A strategy flowchart for an initial model optimization process provided in this application embodiment;

[0051] Figure 4 An interactive schematic diagram of a personalized federated edge learning method provided in an embodiment of this application;

[0052] Figure 5 A strategy flowchart for a two-stage model aggregation process provided in an embodiment of this application;

[0053] Figure 6 A schematic diagram of the structure of a personalized federated edge learning device provided in this application embodiment;

[0054] Figure 7 This is a schematic diagram of the internal structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0055] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0056] In real-world surveillance scenarios, traditional ACBD methods face challenges due to the diversity of populations and the complexity of the environment, particularly in terms of data heterogeneity and personalized requirements. These methods not only require large amounts of labeled data for training but also lack personalized learning model design for specific population scenarios. This makes them ill-suited to meet the personalized detection needs of different surveillance environments, resulting in poor adaptability, limited detection accuracy, and unsatisfactory detection performance.

[0057] Based on this, this application proposes the following technical solution, as detailed below:

[0058] In some embodiments, such as Figure 1 As shown, Figure 1 This application provides an architectural diagram of a personalized federated edge learning framework as an embodiment of the present application; by Figure 1 It is understood that the personalized federated edge learning method of this application can be applied to different monitoring scenarios. The personalized federated edge learning framework it adopts mainly consists of cloud servers, edge servers, client devices and monitoring terminals.

[0059] The system consists of several layers: monitoring terminals deployed in public areas to collect critical video surveillance data; client devices connected to these terminals to receive video stream data and perform initial model training and inference locally; edge servers acting as an intermediate layer to collect model updates from client devices and perform model aggregation operations; and cloud servers serving as the system's central hub to create and update the global model and coordinate the entire system's operation.

[0060] In one embodiment, such as Figure 2 As shown, Figure 2 This is a flowchart illustrating a personalized federated edge learning method provided in an embodiment of this application. This application provides a personalized federated edge learning method, specifically including the following:

[0061] S110: Collects video data through the monitoring terminal and uses the client device to optimize the preset initial model based on the video data to generate a local model.

[0062] In this step, when it is necessary to detect the behavior of abnormal groups in different monitoring scenarios, this application can collect video data of the corresponding scenario through the monitoring terminal in the personalized federated edge learning framework, and then use the client device to optimize the preset initial model based on the video data to generate a local model. Therefore, the model can adapt to different monitoring scenarios, thereby improving the accuracy and adaptability of detection.

[0063] The initial model refers to a general behavior detection model pre-stored in the client device. This model is trained on a large-scale dataset and has basic behavior recognition capabilities, but its recognition accuracy for specific monitoring scenarios is not high enough. Therefore, the client device needs to further optimize it for specific monitoring scenarios.

[0064] Specifically, monitoring terminals deployed in public places can collect video data in real time for corresponding scenarios, ensuring that the system can comprehensively and accurately acquire behavioral data under various monitoring scenarios. After data collection is completed, the monitoring terminal can transmit the video data to the client device connected to it. This client device undertakes the core task of local intelligent analysis, and can preprocess, extract features, and train the model based on a preset initial model to obtain a local model.

[0065] Understandably, by combining local environmental characteristics and historical behavioral data, client devices can personalize and optimize the initial model, generating a local model adapted to the current monitoring scenario. Compared to traditional centralized training methods, this approach can better capture behavioral patterns in different scenarios and improve the accuracy of detecting abnormal behavior.

[0066] S120: The edge server obtains local models output by multiple client devices based on the aggregation threshold, and uses an asynchronous cross-device model aggregation strategy to aggregate the local models to generate an edge model.

[0067] In this step, after the local model is obtained through client device optimization in step S110, the edge server can obtain the local models output by multiple client devices connected to it based on the aggregation threshold, and aggregate each local model using an asynchronous cross-device model aggregation strategy to generate an edge model. This reduces aggregation latency caused by device issues and improves the efficiency and flexibility of edge model aggregation.

[0068] The aggregation threshold refers to the conditions or standards set by the edge server before executing local model aggregation to determine when to trigger the model aggregation process, thereby ensuring that model aggregation is completed within a reasonable timeframe. This guarantees the effectiveness of model aggregation while avoiding excessively long waiting times that could lead to system response delays. It should be noted that the aggregation threshold in this application can be either a local model submission threshold or a waiting time threshold; no restriction is imposed here.

[0069] Understandably, after collecting local models, the edge server can employ an asynchronous cross-device model aggregation strategy to efficiently merge the local models submitted by various client devices. This asynchronous cross-device model aggregation strategy is a model aggregation method used in federated learning edge computing environments. It allows the edge server to receive and process model updates from client devices at different times without waiting for all client devices to submit synchronously, thus effectively reducing aggregation latency caused by factors such as inconsistent device states, network fluctuations, or limited computing resources. Therefore, the edge server in this application not only improves the efficiency of edge model aggregation but also enhances the system's flexibility, enabling it to adapt to dynamically changing computing environments and communication conditions.

[0070] Furthermore, during model aggregation, edge servers can employ weighted averaging, personalized aggregation, or meta-learning-based methods to perform more intelligent aggregation optimization based on the training contribution of each client device, data distribution characteristics, and model performance feedback. Ultimately, the aggregated edge model will have greater generalization capabilities while retaining the personalized characteristics of each client device, enabling it to more accurately adapt to the abnormal behavior detection needs in different monitoring scenarios. This lays a solid foundation for further cloud optimization and global model updates.

[0071] S130: Utilize cloud servers to periodically perform global aggregation of edge models output by multiple edge servers using an asynchronous model aggregation strategy to generate a global model.

[0072] In this step, after obtaining the edge model through edge server aggregation in step S120, the cloud server can periodically use an asynchronous model aggregation strategy to globally aggregate the edge models output by multiple edge servers connected to it, generating a global model. This enables the model to learn a wider range of population behavior patterns and characteristics, thereby improving global performance.

[0073] Specifically, after completing local model aggregation, the edge server can generate an edge model and upload it to the cloud server. The cloud server, as the central coordinating node of the entire system framework, can periodically execute asynchronous model aggregation strategies to merge models from multiple edge servers in order to continuously optimize the global model.

[0074] Understandably, since the monitoring areas covered by different edge servers may differ, the data distribution, feature patterns, and types of crowd behavior collected will also vary. Therefore, cloud aggregation through cloud servers can fully integrate the feature information of each region, enabling the model to learn a wider range of diverse crowd behavior patterns and improve the model's adaptability and generalization ability in various monitoring scenarios.

[0075] Furthermore, the asynchronous model aggregation strategy adopted in this application can avoid the latency problem caused by the cloud server waiting for all edge servers to submit models synchronously, enabling the system to perform global model updates more efficiently. Even if some edge servers fail to submit models in time due to network conditions, computing resources, or other factors, it will not affect the continuous optimization of the global model. Therefore, the system can maintain high detection accuracy and stability in different monitoring environments, thereby improving the overall performance and intelligence level of abnormal behavior detection.

[0076] S140: Distribute the global model as the latest initial model back to each edge server and each client device, and return the video data collected through the monitoring terminal and its subsequent steps until the global model meets the preset iteration end conditions.

[0077] In this step, after the global model is obtained by cloud server aggregation in step S130, the cloud server can also distribute the global model as the latest initial model back to each edge server and each client device, so that each client device can continue to train and optimize the model to adapt to new data and changes, while maintaining the model's real-time performance and efficiency. The model iteration process can end when the global model meets the preset iteration end conditions.

[0078] Understandably, through a continuous iterative update mechanism, this application ensures the real-time performance of the model, enabling client devices to always use the latest optimized model for inference and decision-making, preventing the model from gradually becoming ineffective due to changes in data distribution. Simultaneously, while distributing the model, the cloud server continuously monitors the model's training performance and, based on preset iteration termination conditions such as achieving stable detection accuracy or a set number of training epochs, decides whether to terminate the entire model iteration process. This ensures that while maintaining model quality, computational and communication efficiency is improved, unnecessary resource consumption is avoided, and the entire personalized federated edge learning framework maintains high efficiency while possessing stronger adaptability and intelligence.

[0079] It should be noted that, in addition to distributing the global model back to various client devices for iterative training, the cloud server can also distribute it to various edge servers. Specifically, in the personalized federated edge learning framework, the edge servers are geographically closer to the client devices. Compared to the long-distance communication between edge servers and the cloud server, the communication latency between edge servers is relatively low. Furthermore, in applications with high real-time requirements, such as abnormal crowd behavior detection, rapid data transmission and model updates are crucial. Therefore, this application allows local model updates to be sent to the edge server for preliminary processing, significantly shortening data transmission time, accelerating model iteration, enabling the system to respond more promptly to newly emerging abnormal behavior patterns, improving the system's timeliness and usability, and enhancing its monitoring and early warning capabilities for public safety incidents.

[0080] Furthermore, in large-scale monitoring networks, numerous client devices may simultaneously perform task computations and data transmissions. If all data flows directly to the cloud server, it will place enormous pressure on network bandwidth and the cloud server's computing resources, easily leading to data congestion and processing delays. Edge servers, situated between client devices and the cloud server, act as an intermediary layer, effectively offloading and preprocessing data and tasks. They can temporarily store and filter data and model updates from client devices, processing and forwarding them according to certain rules and priorities. This reduces the burden on the cloud server, optimizes the entire system's data transmission and processing flow, and ensures the system's efficient and stable operation.

[0081] In the above embodiments, during abnormal crowd behavior detection, video data can be collected through a monitoring terminal, and client devices can optimize a preset initial model based on the video data to generate a local model. This allows the model to adapt to different monitoring scenarios, improving detection accuracy and adaptability. Then, edge servers can obtain local models output by multiple client devices based on aggregation thresholds, and an asynchronous cross-device model aggregation strategy can be used to aggregate these local models to generate an edge model. This reduces aggregation latency caused by device issues and improves the efficiency and flexibility of edge model aggregation. Next, cloud servers can periodically use an asynchronous model aggregation strategy to globally aggregate the edge models output by multiple edge servers to generate a global model. This allows the model to learn a wider range of crowd behavior patterns and features, improving overall performance. Finally, the global model can be distributed back to each edge server and each client device as the latest initial model, and the system returns to collecting video data through the monitoring terminal and subsequent steps until the global model meets the preset iteration termination conditions. This ensures that the entire system can continuously learn and adapt to new data and changes while maintaining the model's real-time performance and efficiency. Furthermore, the method described in this application can reduce reliance on a central server, lower data transmission overhead, and thus enhance the scalability and robustness of the entire monitoring system.

[0082] In one embodiment, step S110, which involves using a client device to train and optimize a preset initial model based on video data to generate a local model, may include:

[0083] S111: Use the client device to decompose the preset initial model into a network, resulting in a general feature extraction subnetwork and a personalized label mapping subnetwork.

[0084] S112: Anomaly detection is performed on video data through a general feature extraction subnetwork, and multiple behavioral features are extracted based on the detection results.

[0085] S113: The personalized label mapping subnetwork is used to map labels to each behavioral feature to obtain the behavioral label corresponding to each behavioral feature.

[0086] S114: Optimize the initial model based on each behavioral feature and the corresponding behavioral label to obtain a local model.

[0087] In this embodiment, when optimizing the initial model, the client device can first decompose the preset initial model into a network to obtain a general feature extraction subnetwork and a personalized label mapping subnetwork. Then, after detecting abnormal features in the video data through the general feature extraction subnetwork, multiple behavioral features are extracted based on the detection results. Each behavioral feature is then labeled through the personalized label mapping subnetwork to obtain the behavioral label corresponding to each behavioral feature. Finally, the initial model can be optimized based on each behavioral feature and the behavioral label corresponding to each behavioral feature to obtain a local model.

[0088] Specifically, the general feature extraction subnetwork and the personalized label mapping subnetwork perform different functions, working together to optimize the model and achieve personalized adaptation. The general feature extraction subnetwork is primarily responsible for extracting general features related to abnormal behavior detection from the input video data. These features are shared across multiple monitoring scenarios and can capture common abnormal behavior patterns in different scenarios. Through this subnetwork, the system can extract low-dimensional, representative features from high-dimensional video data. These features reflect key information in the video, such as motion trajectories and posture changes, providing a foundation for subsequent abnormal behavior detection. The personalized label mapping subnetwork is mainly responsible for mapping the features extracted by the general feature extraction subnetwork to specific behavior labels. This subnetwork is trained based on the characteristics of the local dataset, enabling it to adapt to the needs of specific monitoring scenarios and achieve personalized abnormal behavior recognition. Through this subnetwork, the system can associate the extracted general features with specific behavior labels, such as "running," "gathering," and "fighting," thereby achieving accurate identification of abnormal behavior in specific scenarios.

[0089] In a specific embodiment, the general feature extraction subnetwork and the personalized label mapping subnetwork obtained after the initial model decomposition can be represented as follows:

[0090]

[0091]

[0092] In the formula, and Let each represent a general feature extraction subnetwork in the local model on the i-th client device. and personalized tag mapping subnetwork Parameters; and These represent the general feature extraction subnetworks in the initial model. and personalized tag mapping subnetwork Global parameters.

[0093] Among them, the general feature extraction subnetwork It can be represented as a mapping function that maps high-dimensional input video data. Mapped to a lower-dimensional feature space This allows us to extract common features from the monitoring scenario. The mapping function is specifically represented as follows:

[0094]

[0095] In the formula, d represents the dimension of the video data; t represents the dimension of the hidden features.

[0096] Personalized Tag Mapping Subnetwork It can also be represented as a mapping function that maps the feature space output by the general feature extraction subnetwork. Mapping to behavior label space This allows for the assignment of corresponding behavior labels to each feature, and the mapping function is specifically expressed as follows:

[0097]

[0098] In the formula, y represents the dimension of the label feature.

[0099] Based on this, the client device can combine all extracted behavioral features and their corresponding behavioral labels to further optimize and train the initial model, enabling the model to more accurately adapt to the characteristics of the current monitoring environment, ultimately resulting in an optimized local model. Through this optimization process, the client device can not only improve the model's recognition accuracy in local scenarios but also enhance its adaptability to different monitoring environments, providing a higher-quality model foundation for subsequent edge model aggregation and global model optimization.

[0100] In one embodiment, the process of optimizing the initial model based on each behavioral feature and the behavioral label corresponding to each behavioral feature to obtain a local model in step S114 may include:

[0101] S1141: Input each behavioral feature and the corresponding behavioral label into the initial model to obtain the predicted label output by the initial model based on each behavioral label.

[0102] S1142: Backpropagate the loss function to each predicted label to obtain the gradient results of the initial model.

[0103] S1143: Using stochastic gradient descent and hyperparameter control methods, the network parameters of the general feature extraction subnetwork and the personalized label mapping subnetwork are updated based on the gradient results.

[0104] In this embodiment, the client device can input each behavioral feature and the behavioral label corresponding to each behavioral feature into the initial model to obtain the predicted label output by the initial model based on each behavioral label. Then, the loss function is used to backpropagate each predicted label to obtain the gradient result of the initial model. The stochastic gradient descent method and the hyperparameter control method are used to update the network parameters of the general feature extraction subnetwork and the personalized label mapping subnetwork based on the gradient result.

[0105] Stochastic gradient descent is a gradient descent algorithm used to optimize neural network parameters. Compared to standard gradient descent (Batch Gradient Descent, BGD), it uses only one or a mini-batch of samples to calculate gradients and update parameters in each iteration, instead of using the entire dataset. This reduces computational cost, increases training speed, and allows the model to escape local optima more quickly, improving its generalization ability. Hyperparameter control methods refer to the reasonable adjustment of key hyperparameters that affect training performance during model training to optimize training efficiency and improve the final performance of the model. These hyperparameters can include learning rate, batch size, and momentum, and are not limited here.

[0106] Specifically, the initial model can calculate the label for each behavior based on the input behavioral features and output the corresponding predicted label, thus providing the model with an understanding and classification of the current data. After obtaining the predicted labels, the client device can use a preset loss function to calculate the error between each predicted label and its corresponding true label to measure the model's accuracy on the current training data. Subsequently, this error information can be passed to the parameters of each layer of the model through the backpropagation algorithm, enabling the client device to calculate the gradient information of the initial model and clarify the direction in which the model needs to be optimized. On this basis, the client device can use the stochastic gradient descent method, combined with a hyperparameter control strategy, to iteratively update the network parameters of the general feature extraction subnetwork and the personalized label mapping subnetwork to gradually optimize the model performance. This allows the model to adapt to local monitoring data while improving its ability to understand complex behavioral patterns, ultimately resulting in a more accurate and efficient local model.

[0107] Indicatively, such as Figure 3 As shown, Figure 3 A strategy flowchart for an initial model optimization process provided in this application embodiment; Figure 3In this system, the monitoring terminal is responsible for collecting data and transmitting it to data participants, i.e., client devices. These client devices can then divide the data into multiple batches (Batch 1, Batch 2, ..., Batch k) for processing. The data then flows through the system via forward (FP) and backward (BP) transmissions. Forward (FP) transmits data from the monitoring terminal to the data processing nodes, while backward (BP) transmits processing results or model updates from the data processing nodes to other nodes, i.e., edge servers. This allows the edge servers to receive model updates from multiple client devices and perform aggregation or further optimization.

[0108] all in all, Figure 3 The process of data collection, transmission, processing, and model training for a monitoring terminal is described in detail. Data flows through the system through forward and reverse transmission. After multiple batches of processing and model training, the trained model is finally generated and the results are output. The output results include soft labels (probability distribution) and true values ​​(hard labels), which are mainly used to evaluate model performance.

[0109] In one embodiment, the calculation formula for backpropagation in step S1242 may include:

[0110]

[0111] In the formula, Indicates the use of loss function The calculated gradient results; where, Represents the gradient. Denotes the general feature extraction subnetwork in the i-th client device. The network parameters in the (t-1)th iteration update; Represents the personalized tag mapping subnetwork in the i-th client device. The network parameters in the (t-1)th iteration update; Represents the loss function right and The partial derivatives of .

[0112] In this embodiment, backpropagation guides the model on how to adjust parameters to reduce loss. Its formula is primarily used for gradient calculations when the client device optimizes the model locally, ensuring that the model can learn specific abnormal behavior patterns in the monitoring scenario through personalized training. It plays a central role in personalized federated edge learning, enabling the model to maintain high detection accuracy while adapting to different monitoring environments, and ultimately improving the overall system's generalization ability through the aggregation of edge servers and cloud servers.

[0113] In one embodiment, the calculation formula for the stochastic gradient descent method in step S1143 may include:

[0114]

[0115]

[0116] In the formula, Denotes the general feature extraction subnetwork in the i-th client device. Network parameters in the t-th update iteration; Indicates the learning rate; Represents the general feature extraction subnetwork The gradient; Indicates from the loss function The calculated gradient results; Represents the personalized tag mapping subnetwork in the i-th client device. Network parameters in the t-th update iteration; Represents the general feature extraction subnetwork The gradient.

[0117] In this embodiment, the calculation formula of the stochastic gradient descent method is mainly used for local model training on the client device. By continuously optimizing the general feature extraction subnetwork and the personalized label mapping subnetwork through stochastic gradient descent, the model's ability to detect abnormal behavior is improved, and it adapts to the personalized needs of different monitoring scenarios. Finally, the optimized local model is uploaded to the edge server for asynchronous aggregation, further enhancing the overall abnormal behavior detection capability of the system.

[0118] In one embodiment, the calculation formula for the hyperparameter control method in step S1243 may include:

[0119]

[0120] In the formula, Represents the norm of the parameter vector; This indicates regularization, used to control the personalized label mapping subnetwork. The intensity of personalized adjustments.

[0121] In this embodiment, to balance the absorption of global knowledge and the preservation of local data features, this application introduces hyperparameters. This hyperparameter adjusts the degree of learning from the global model. It allows client devices to adjust the degree of personalization based on the characteristics of local data, balancing the personalization and stability of the model during local training. This ensures that the optimized model does not undergo drastic parameter changes while adapting to new data, thereby improving the model's generalization ability and stability.

[0122] Specifically, the calculation formula for the hyperparameter control method is mainly used in the personalized label mapping subnetwork. The optimization utilizes regularization terms to control the magnitude of parameter updates, thereby balancing the model's individual adaptability with training stability. This is achieved by appropriately adjusting the hyperparameters. This application ensures that the model can learn new behavioral features without affecting its generalization ability and convergence due to large parameter adjustments.

[0123] In one embodiment, the calculation formula for the asynchronous cross-device model aggregation strategy in step S120 may include:

[0124]

[0125]

[0126] In the formula, and Represents the general feature extraction subnetwork in the edge model and personalized tag mapping subnetwork The network parameters in the (t+1)th iteration update; and Denotes the general feature extraction subnetwork in the i-th client device. and personalized tag mapping subnetwork Network parameters in the t-th iteration; j represents the j-th client device; This represents the set of client devices participating in the current aggregation. Indicates the number of parameters.

[0127] In this embodiment, the formula is mainly used to aggregate local models of client devices on the edge server to generate new edge models. An asynchronous aggregation mechanism is employed, allowing different client devices to submit their local models at different times without affecting the overall model aggregation process. This strategy uses a weighted average method to fuse the models from client devices, improving the accuracy and adaptability of the edge models while reducing aggregation waiting time caused by client device offlineness or latency, thus increasing the update efficiency of the edge models and adapting to the data distribution characteristics of different client devices.

[0128] In one embodiment, the calculation formula for the asynchronous model aggregation strategy in step S130 may include:

[0129]

[0130]

[0131] In the formula, and These represent the general feature extraction subnetworks in the global model. and personalized tag mapping subnetwork Network parameters; Let i represent the set of all edge servers participating in the aggregation, and let i and j represent the i-th and j-th edge servers in the set of all edge servers participating in the aggregation, respectively.

[0132] In this embodiment, the formula is mainly used by the cloud server to asynchronously aggregate edge server models, ultimately generating a global model to improve the model's generalization ability and stability. Here, through a weighted averaging strategy, the cloud server can integrate model parameters from different edge servers, enabling the global model to learn a wider range of behavioral patterns, adapt to data distributions in different environments, and simultaneously improve the model's generalization ability and robustness. This ensures the model can adapt to the data characteristics of different edge environments, achieving more accurate predictions and decisions.

[0133] To better explain the personalized federated edge learning approach of this application, the following will be conducted through... Figure 4 and Figure 5 To further illustrate, illustratively, such as Figure 4 and Figure 5 As shown, Figure 4 An interactive schematic diagram of a personalized federated edge learning method provided in an embodiment of this application; Figure 5 This is a flowchart illustrating a two-stage model aggregation process provided in an embodiment of this application.

[0134] Figure 4 In this process, firstly, multiple client devices collect and transmit data through monitoring equipment, and then each trains a local model to generate a local model. After training, the client devices upload their local models to the corresponding edge servers. The edge servers receive the local models from multiple client devices and aggregate them to generate edge models. Next, the edge servers upload the aggregated edge models to the cloud server, which is responsible for global model aggregation, further integrating the edge models from multiple edge servers to generate a global model. After global model aggregation is complete, the cloud server distributes the updated global model back to each edge server and client device through a FEL task, achieving model sharing and enabling the next round of local model training and updates. This entire process iterates continuously until the model reaches the expected performance metrics or completes a predetermined number of training rounds.

[0135] Figure 5In this system, the edge server acts as the core node, responsible for data interaction and model updates with multiple client devices. First, the edge server retrieves the global model from the cloud server or other upstream nodes and distributes it to each client device. Upon receiving the model, each client device processes the data and trains the model locally, generating an updated local model. After training, the client device uploads the updated model to the edge server. The edge server receives model updates from multiple client devices and performs aggregation to generate a new edge model. The aggregated edge model can then be uploaded to the cloud server for global model updates or used directly for local inference and task execution. This entire process, coordinated by the edge server, achieves efficient collaboration between distributed model training and updates, ensuring the system's real-time performance and scalability.

[0136] The personalized federated edge learning device provided in the embodiments of this application is described below. The personalized federated edge learning device described below can be referred to in correspondence with the personalized federated edge learning method described above.

[0137] In one embodiment, such as Figure 6 As shown, Figure 6 This application provides a schematic diagram of the structure of a personalized federated edge learning device according to an embodiment of the present application; the present application also provides a personalized federated edge learning device, including a model optimization module 210, a primary aggregation module 220, a secondary aggregation module 230, and a model iteration module 240, specifically including the following:

[0138] The model optimization module 210 is used to collect video data through the monitoring terminal and use the client device to optimize the preset initial model based on the video data to generate a local model.

[0139] The first aggregation module 220 is used to obtain local models output by multiple client devices based on the aggregation threshold using the edge server, and to aggregate the local models using an asynchronous cross-device model aggregation strategy to generate an edge model.

[0140] The secondary aggregation module 230 is used to periodically aggregate edge models output by multiple edge servers using an asynchronous model aggregation strategy employed by the cloud server to generate a global model.

[0141] The model iteration module 240 is used to distribute the global model as the latest initial model back to each edge server and each client device, and return the video data collected through the monitoring terminal and its subsequent steps until the global model meets the preset iteration end condition.

[0142] In the above embodiments, during abnormal crowd behavior detection, video data can be collected through a monitoring terminal, and client devices can optimize a preset initial model based on the video data to generate a local model. This allows the model to adapt to different monitoring scenarios, improving detection accuracy and adaptability. Then, edge servers can obtain local models output by multiple client devices based on aggregation thresholds, and an asynchronous cross-device model aggregation strategy can be used to aggregate these local models to generate an edge model. This reduces aggregation latency caused by device issues and improves the efficiency and flexibility of edge model aggregation. Next, cloud servers can periodically use an asynchronous model aggregation strategy to globally aggregate the edge models output by multiple edge servers to generate a global model. This allows the model to learn a wider range of crowd behavior patterns and features, improving overall performance. Finally, the global model can be distributed back to each edge server and each client device as the latest initial model, and the system returns to collecting video data through the monitoring terminal and subsequent steps until the global model meets the preset iteration termination conditions. This ensures that the entire system can continuously learn and adapt to new data and changes while maintaining the model's real-time performance and efficiency. Furthermore, the method described in this application can reduce reliance on a central server, lower data transmission overhead, and thus enhance the scalability and robustness of the entire monitoring system.

[0143] In one embodiment, the model optimization module 210 may include:

[0144] The network decomposition submodule is used to decompose the preset initial model using the client device to obtain a general feature extraction subnetwork and a personalized label mapping subnetwork.

[0145] The feature extraction submodule is used to detect abnormal features in video data through a general feature extraction subnetwork and extract multiple behavioral features based on the detection results.

[0146] The label mapping submodule is used to map labels to each behavioral feature through a personalized label mapping subnetwork, so as to obtain the behavioral label corresponding to each behavioral feature.

[0147] The model optimization submodule is used to optimize the initial model based on each behavioral feature and the corresponding behavioral label to obtain a local model.

[0148] In one embodiment, the model optimization submodule may include:

[0149] The label prediction unit is used to input each behavioral feature and the corresponding behavioral label into the initial model to obtain the predicted label output by the initial model based on each behavioral label.

[0150] The gradient calculation unit is used to backpropagate the loss function to each predicted label to obtain the gradient result of the initial model.

[0151] The parameter update unit is used to update the network parameters of the general feature extraction subnetwork and the personalized label mapping subnetwork based on the gradient results, using stochastic gradient descent and hyperparameter control methods, respectively.

[0152] In one embodiment, the gradient calculation unit may include:

[0153]

[0154] In the formula, Indicates the use of loss function The calculated gradient results; where, Represents the gradient. Denotes the general feature extraction subnetwork in the i-th client device. The network parameters in the (t-1)th iteration update; Represents the personalized tag mapping subnetwork in the i-th client device. The network parameters in the (t-1)th iteration update; Represents the loss function right and The partial derivatives of .

[0155] In one embodiment, the parameter update unit may include:

[0156]

[0157]

[0158] In the formula, Denotes the general feature extraction subnetwork in the i-th client device. Network parameters in the t-th update iteration; Indicates the learning rate; Represents the general feature extraction subnetwork The gradient; Indicates from the loss function The calculated gradient results; Represents the personalized tag mapping subnetwork in the i-th client device. Network parameters in the t-th update iteration; Represents the general feature extraction subnetwork The gradient.

[0159] In one embodiment, the parameter update unit may further include:

[0160]

[0161] In the formula, Represents the norm of the parameter vector; This indicates regularization, used to control the personalized label mapping subnetwork. The intensity of personalized adjustments.

[0162] In one embodiment, the primary aggregation module 220 may include:

[0163]

[0164]

[0165] In the formula, and Represents the general feature extraction subnetwork in the edge model and personalized tag mapping subnetwork The network parameters in the (t+1)th iteration update; and Denotes the general feature extraction subnetwork in the i-th client device. and personalized tag mapping subnetwork Network parameters in the t-th iteration; j represents the j-th client device; This represents the set of client devices participating in the current aggregation. Indicates the number of parameters.

[0166] In one embodiment, the secondary aggregation module 230 may include:

[0167]

[0168]

[0169] In the formula, and These represent the general feature extraction subnetworks in the global model. and personalized tag mapping subnetwork Network parameters; Let i represent the set of all edge servers participating in the aggregation, and let i and j represent the i-th and j-th edge servers in the set of all edge servers participating in the aggregation, respectively.

[0170] In one embodiment, this application also provides a storage medium storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the personalized federated edge learning method as described in any of the above embodiments.

[0171] In one embodiment, this application also provides a computer device storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the personalized federated edge learning method as described in any of the above embodiments.

[0172] Indicatively, such as Figure 7 As shown, Figure 7 This is a schematic diagram of the internal structure of a computer device 300 provided in an embodiment of this application. The computer device 300 can be provided as a server. (Refer to...) Figure 7 The computer device 300 includes a processing component 302, which further includes one or more processors, and memory resources represented by memory 301 for storing instructions, such as application programs, that can be executed by the processing component 302. The application programs stored in memory 301 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 302 is configured to execute instructions to perform the personalized federated edge learning method of any of the above embodiments.

[0173] The computer device 300 may also include a power supply component 303 configured to perform power management of the computer device 300, a wired or wireless network interface 304 configured to connect the computer device 300 to a network, and an input / output (I / O) interface 305. The computer device 300 may operate on an operating system stored in memory 301, such as Windows Server™, Mac OS X™, Unix™, Linux™, Free BSD™, or similar.

[0174] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0175] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0176] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.

[0177] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A personalized federated edge learning method, characterized in that, The method includes: Video data is collected through a monitoring terminal, and a local model is generated by optimizing a preset initial model based on the video data using a client device. The edge server obtains local models output by multiple client devices based on an aggregation threshold, and uses an asynchronous cross-device model aggregation strategy to aggregate the local models to generate an edge model. By periodically employing an asynchronous model aggregation strategy on cloud servers, edge models output from multiple edge servers are globally aggregated to generate a global model. The global model is distributed back to each edge server and each client device as the latest initial model, and the video data collected through the monitoring terminal and its subsequent steps are returned until the global model meets the preset iteration end condition. The step of using the client device to train and optimize a preset initial model based on the video data to generate a local model includes: The client device is used to decompose the preset initial model into a network, resulting in a general feature extraction subnetwork and a personalized label mapping subnetwork. The video data is subjected to anomaly detection through the general feature extraction sub-network, and multiple behavioral features are extracted based on the detection results. The personalized label mapping subnetwork is used to map labels to each behavioral feature to obtain the behavioral label corresponding to each behavioral feature. The initial model is optimized based on each behavioral feature and the corresponding behavioral label to obtain a local model.

2. The personalized federated edge learning method according to claim 1, characterized in that, The process of optimizing the initial model based on each behavioral feature and the corresponding behavioral label to obtain a local model includes: Each behavioral feature and its corresponding behavioral label are input into the initial model to obtain the predicted label output by the initial model based on each behavioral label. The gradient results of the initial model are obtained by backpropagating each predicted label using the loss function. The network parameters of the general feature extraction subnetwork and the personalized label mapping subnetwork are updated based on the gradient results using stochastic gradient descent and hyperparameter control methods.

3. The personalized federated edge learning method according to claim 2, characterized in that, The calculation formula for backpropagation includes: ； In the formula, Indicates the use of loss function The calculated gradient results; where, Represents the gradient. This represents the general feature extraction subnetwork in the i-th client device. The network parameters in the (t-1)th iteration update; This represents the personalized tag mapping subnetwork in the i-th client device. The network parameters in the (t-1)th iteration update; Represents the loss function right and The partial derivatives of .

4. The personalized federated edge learning method according to claim 2, characterized in that, The calculation formula for the stochastic gradient descent method includes: ； ； In the formula, This represents the general feature extraction subnetwork in the i-th client device. The network parameters in the t-th update iteration; Indicates the learning rate; This represents the general feature extraction subnetwork. The gradient; Indicates from the loss function The calculated gradient results; This represents the personalized tag mapping subnetwork in the i-th client device. The network parameters in the t-th update iteration; This represents the general feature extraction subnetwork. The gradient.

5. The personalized federated edge learning method according to claim 2, characterized in that, The calculation formula for the hyperparameter control method includes: ； In the formula, Represents the norm of the parameter vector; Regularization is used to control the personalized label mapping subnetwork. The intensity of personalized adjustments; This represents the general feature extraction subnetwork in the i-th client device. The network parameters in the (t-1)th iteration update; This represents the personalized tag mapping subnetwork in the i-th client device. The network parameters in the (t-1)th iteration update.

6. The personalized federated edge learning method according to claim 1, characterized in that, The calculation formula for the asynchronous cross-device model aggregation strategy includes: ； ； In the formula, and This represents the general feature extraction subnetwork in the edge model. and the personalized tag mapping subnetwork The network parameters in the (t+1)th iteration update; and This represents the general feature extraction subnetwork in the i-th client device. and the personalized tag mapping subnetwork Network parameters in the t-th iteration; j represents the j-th client device; This represents the set of client devices participating in the current aggregation. This indicates the number of parameters; both g and d represent the number of samples in the video data.

7. The personalized federated edge learning method according to claim 1, characterized in that, The calculation formula for the asynchronous model aggregation strategy includes: ； ； In the formula, and These represent the general feature extraction subnetworks in the global model. and the personalized tag mapping subnetwork Network parameters; Let represent the set of all edge servers participating in the aggregation, where i and j represent the i-th and j-th edge servers in the set of all edge servers participating in the aggregation, respectively; s represents the s-th edge server; and g and d both represent the number of video data samples.

8. A personalized federated edge learning device, characterized in that, include: The model optimization module is used to collect video data through the monitoring terminal and use the client device to optimize the preset initial model based on the video data to generate a local model. The first aggregation module is used to obtain local models output by multiple client devices based on the aggregation threshold by the edge server, and to aggregate the local models using an asynchronous cross-device model aggregation strategy to generate the edge model. The secondary aggregation module is used to periodically aggregate edge models output by multiple edge servers using an asynchronous model aggregation strategy employed by the cloud server to generate a global model. The model iteration module is used to distribute the global model as the latest initial model back to each edge server and each client device, and return the video data collected through the monitoring terminal and its subsequent steps until the global model meets the preset iteration end conditions. The model optimization module includes: The client device is used to decompose the preset initial model into a network, resulting in a general feature extraction subnetwork and a personalized label mapping subnetwork. The video data is subjected to anomaly detection through the general feature extraction sub-network, and multiple behavioral features are extracted based on the detection results. The personalized label mapping subnetwork is used to map labels to each behavioral feature to obtain the behavioral label corresponding to each behavioral feature. The initial model is optimized based on each behavioral feature and the corresponding behavioral label to obtain a local model.

9. A storage medium, characterized in that: The storage medium stores computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the personalized federated edge learning method as described in any one of claims 1 to 7.

10. A computer device, characterized in that, include: One or more processors, and memory; The memory stores computer-readable instructions that, when executed by the one or more processors, perform the steps of the personalized federated edge learning method as described in any one of claims 1 to 7.

Images