Personalized federation learning-based violation behavior detection method

Abstract

The invention discloses a violation behavior detection method for personalized federal learning, and the method comprises the steps: training a basic feature extraction model through employing a violation behavior training sample library, deploying a global feature extraction layer at a server, distributing the basic feature extraction layer to each client, and generating a dynamic task description; the basic feature extraction layer of each client is trained through local data, and knowledge distillation learning is carried out through a teacher model issued by the server; each client extracts a mean feature of local data to form a local prototype vector, and uploads the local prototype vector and global parameters of each client to the server; the server performs dynamic clustering on the local prototype vector and the global parameter of each client and divides the local prototype vector and the global parameter into a plurality of client groups to obtain a clustering center, a clustering device parameter and a personalized model; and the server issues the updated clustering center, the updated clustering device parameter and the updated personalized model to the corresponding client, and continuously optimizes the feature space distribution and the model parameter of each client.

Description

Technical Field

[0001] The present invention relates to the technical field of traffic violation detection technology, and in particular to a personalized federated learning traffic violation detection method. Background Art

[0002] Traditional construction safety supervision relies heavily on centralized data collection and analysis technologies. For example, this involves capturing worker behavior data in real time through construction site cameras, sensors, and wearable devices, and uploading it to a central server for model training. However, this model faces multiple challenges: Data from different contractors and project teams in a construction scenario often forms strictly isolated "data silos" due to commercial confidentiality and worker privacy, making it difficult for a single institution to obtain sufficient violation samples to train a robust model. The highly dynamic nature of the construction site environment (such as changes in the layout of temporary facilities and the cross-operation of multiple trades) makes it difficult for traditional centralized models to adapt to the differences in data distribution across different project scenarios, making false detections and missed detections more likely. Furthermore, the computing power requirements of complex deep learning models for edge devices are significantly inconsistent with the low-power IoT terminals commonly found on construction sites, creating a conflict between the need for real-time detection and computing resource constraints.

[0003] The distributed collaborative computing framework of federated learning offers a breakthrough. Its "data remains static, model moves" approach ensures both personal privacy protection and compliance with construction worker biometric information requirements. It also enables knowledge sharing through the interaction of model parameters among multiple participants. However, the application of standard federated learning in construction scenarios still has limitations. A unified global model struggles to adapt to the individual characteristics of different construction phases and types of work (e.g., aerial work versus underground construction), potentially reducing sensitivity in identifying violations in specific scenarios (e.g., failure to use a safety harness or operating machinery beyond its designated boundaries).

[0004] The purpose of this invention is to design a personalized federated learning method for detecting traffic violations in response to the above-mentioned problems in the existing technology. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to propose a personalized federated learning violation detection method that can solve the above problems.

[0006] The present invention provides a personalized federated learning violation detection method, comprising: Use the violation training sample library to train the basic feature extraction model, deploy a global feature extraction layer on the server, distribute the basic feature extraction layer to each client, and generate dynamic task descriptions through a large language model; The basic feature extraction layer of each client is trained using local data and performs knowledge distillation learning using the teacher model sent by the server. Each client extracts the mean features of local data to form a local prototype vector, and uploads each client's local prototype vector and its global parameters to the server; The server dynamically clusters the local prototype vectors and global parameters of each client into several client groups, and obtains the cluster center, clusterer parameters and personalized model; The server sends the updated cluster centers, clusterer parameters, and personalized models to the corresponding clients, continuously optimizing the feature space distribution and model parameters of each client. The optimized personalized model is deployed to the client to detect real-time violations at the construction site, and the personalized model is continuously updated with real-time data.

[0007] Beneficial effects of the present invention: First, by decoupling local features and sharing parameters, we unify global feature extraction, enabling all clients to learn shared, high-quality basic representations. This improves generalization capabilities for new or small client-side applications. The model structure can be quickly migrated to new scenarios while also being fine-tuned and optimized for each client, ensuring both model personalization and synergy. KL divergence distillation is performed using the latest server-side global model (teacher) and local model (student). By jointly optimizing the distillation loss and the local classification loss, we address the problem of local overfitting and loss of global generalization caused by sparse local samples or distribution shifts on the client side.

[0008] Second, prototype vector extraction is performed using the local feature mean. This solves the technical problem of high communication consumption and privacy exposure associated with uploading all original samples or features. The mean prototype vector significantly reduces transmission volume and effectively protects sensitive user data. Generating local dual prototypes through task-specific MLP mapping avoids the inability of a single prototype to cover all complex business requirements and scenario feature distributions, ensuring that the prototype space on each end is closely aligned with the actual business task.

[0009] Third, using the Deep Embedding Clusterer (DEC), the local prototype vectors uploaded by the client are reduced in dimension and re-represented through a stacked denoising autoencoder. This produces a more robust and discriminative low-dimensional embedding feature. High-confidence samples are weighted by soft-assigning probabilities and target distributions (based on KL divergence). The server uses the distance between the low-dimensional embedding and the cluster center to measure the distance between each client and the group center. This weighted sum of the client-side model parameters is then used to output a personalized global model specific to the group.

[0010] Fourth, by dynamically distributing personalized models and cluster centers, we improve end-side personalization, adaptability, and security, and prevent local performance degradation caused by model / cluster center drift. Data on different ends is heterogeneously distributed. If it cannot be dynamically synchronized with the latest center, it will drift outside the global model or cluster, weakening collaboration. Therefore, by fine-tuning the feature extraction layer using the server's latest cluster center, the feature distribution of local data is automatically aligned with the group center. The global feature layer is frozen during the local training phase, and only the local decision head and personalized parameters are optimized to protect shared knowledge from extreme heterogeneity of end-to-end data. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0012] Figure 1 It is a flow chart of the method of this embodiment.

[0013] Figure 2 This is a flowchart of the method implementation of the client of this embodiment.

[0014] Figure 3 This is a flowchart of the method implementation on the server side of this embodiment.

[0015] Figure 4 This is a flowchart of the deep aggregation method implementation of this embodiment. DETAILED DESCRIPTION

[0016] To facilitate understanding by those skilled in the art, the structure of the present invention will now be further described in detail with reference to the embodiments and accompanying drawings. It should be understood that the steps mentioned in this embodiment, unless otherwise specified, can be adjusted in sequence according to actual needs, and can even be executed simultaneously or partially simultaneously.

[0017] like Figure 1 As shown, an embodiment of the present invention provides a personalized federated learning violation detection method, including: S1 uses the violation behavior training sample library to train the basic feature extraction layer, deploys the global feature extraction layer on the server, distributes the basic feature extraction layer to each client, and generates dynamic task descriptions through a large language model; S101 collects and cleans public traffic violation datasets, performs data enhancement, normalization, and annotation, and constructs a standardized training sample library; S102 is based on a pre-trained model and uses public datasets to train a global basic feature extraction layer through transfer learning. , optimize its feature representation capabilities and initialize the client's model weights ; In this step, the pre-trained model can be ResNet, BERT, etc.

[0018] S103 uses a large language model to describe the task type, generates a multi-dimensional soft label matrix, and assigns corresponding task soft labels based on the client's business scenario; In this step, a large language model (e.g., "illegal work at height" and "not wearing a helmet") is used to describe the task type (e.g., "working at height" or "not wearing a helmet"). Examples of large language models include ChatGPT, GPT-3 / 4 / 4o, ERNIE3.0, GLM, LLaMA, and ChatGLM. These textual descriptions are transformed into low-dimensional task feature vectors / soft label matrices after word encoding and embedding. Each task is grouped together, and each client receives the required soft labels (e.g., task distribution vectors) for subsequent model task adaptation and discriminant head parameter setting. This addresses the diverse task categories and non-standard descriptions found in real-world large-scale construction scenarios, as well as the error-prone and difficult-to-scalability issues of manually defined task spaces.

[0019] S104 extracts the basic feature layer , the global model architecture and task soft labels are distributed to each client to complete the initialization client deployment of the local model.

[0020] The basic feature extraction layer of each S2 client is trained using local data and performs knowledge distillation learning using the teacher model sent by the server. S201 Each client performs distributed model training through the feature decoupling network and task adapter combined with local data to calculate the local classification loss To optimize local model parameters ; S2011 Each client loads the local dataset , for local datasets Input data Perform standardization, denoising and enhancement, and associate task labels Generate training batches; In this step, it is assumed that clients, each client There is a unique private dataset, represented by ,in express size.

[0021] S2012 client based on the initial feature extraction layer sent by the server and task decision-making layer , initialize the client local model ,in, Represent local model parameters to ensure that the model architecture is suitable for the task; In this step, if Figure 2 As shown, the local parameters of the client model are decoupled into feature extraction layers and task decision-making layer To adapt to the client data distribution, the global model weight Share knowledge between all clients. Each client has its own model As the weight parameter of its model, Defined as the dimension of the model parameter vector, each client of Divide into local and global parts through personalization technology to form a client local model .

[0022] S2013 will local data Input to the client's feature extraction layer , generate high-dimensional feature representation , and passed to the client's task decision layer Make predictions; S2014 predicts the probability based on the output of the task decision layer Associated task tags , local classification loss is calculated by cross entropy Quantify the prediction error, where R is the number of all identifiable categories; In this step, C = 9, with a total of five major categories and nine subcategories. 1. Failure to wear safety protective equipment: not wearing a hard hat, working at height without a safety belt; 2. Clothing violations: entering the construction site in slippers, sandals, high heels, shorts, or skirts; 3. Behavioral violations: smoking in a no-smoking area, playing around on site, and standing on conveyor belts; 4. Inadequate protection: working on edges without guardrails or safety nets; 5. Operational violations: overloading, or standing or walking under a load.

[0023] S2015 uses the gradient descent algorithm to backpropagate local classification loss , optimize the feature extraction layer of the local model and task decision-making layer Parameters, update local model parameters ; S2016 will train the local model parameters and feature extraction layer weights Temporarily stored on the client.

[0024] In this step, if all client models only learn from their own data, the model can only adapt to a certain area and lack generalization ability. If global unified parameters are used, it is difficult to perfectly adapt to every complex and changing distribution. Therefore, parameter sharing is used to avoid transmitting the original training data, ensuring privacy while also ensuring the model training effect.

[0025] The S202 client uses the server's global model as the teacher model and calculates the distillation loss through KL divergence. , guiding the student model to train.

[0026] The S2021 client downloads the latest global model from the server As the teacher model, get its parameters Used for knowledge distillation; S2022 is based on the client's current local model Parameters , initialize the student model architecture to maintain input and output consistency with the teacher model; S2023 Input local data To Teacher Model , generate the softened probability distribution through forward propagation ,in, is the teacher model output, is the temperature coefficient; S2024 will be the same local data Enter the local student model , generating a softened probability distribution , and record the output features ; S2025 calculates the distillation loss of the teacher and student models based on KL divergence. The calculation formula is as follows: , in, Used to adjust the smoothness of probability distribution to avoid overfitting hard labels; In this step, according to the output of the task decision layer and the true label , the prediction error is quantified by the cross entropy loss.

[0027] S2026 will distill losses With local classification loss The weighted sum is used to get the total loss, which is calculated as follows:

[0028] in, is the distillation weight coefficient, which controls the strength of knowledge transfer; S2027 backpropagates the total loss through the gradient descent algorithm , update the local parameters of the student model , while freezing the global shared parameters To protect federated knowledge consistency.

[0029] In this step, if Figure 2 As shown in the figure, the local student model is guided by the soft target of the global teacher model and optimized in combination with the local classification loss using KL divergence distillation. On the one hand, it inherits global knowledge to enhance generalization, and on the other hand, it finely adapts to the local distribution. This is an important mechanism to improve the effectiveness and security of personalized federated scenario models.

[0030] Each S3 client extracts the mean features of local data to form a local prototype vector, and uploads each client's local prototype vector and its global parameters to the server; S301 client transfers local data set Divided into small batch groups, through the feature extraction layer Calculate the feature vector of each sample batch by batch ; S302 takes the mean of the feature vectors of all samples in the local data set to generate a global prototype vector. The calculation formula is as follows: , in, For the client The total number of samples, Characterize the overall feature distribution of local data; In this step, the global prototype vector can represent the overall data distribution information, greatly compressing the communication volume. It is more efficient and secure than uploading all samples. Taking the average can suppress the impact of extreme data and improve feature stability.

[0031] S303 based on the task tag issued by the server , load the corresponding task-specific mapping parameters , this parameter is defined by the task soft label generated by the server; S304 converts the global prototype vector Input task-specific MLP network to generate local dual prototype vectors , the calculation formula is as follows: ; In this step, if Figure 2 As shown in Figure 2, the prototype generator takes the mean of the feature vectors of all samples in the local dataset and performs nonlinear transformation and feature optimization of the features through the task-specific MLP network. The network contains fully connected layers and nonlinear activation functions, output dimensions and global prototype vectors Different businesses, scenarios, and tasks (e.g., different types of traffic violations) require different feature distribution spaces for discrimination. A single prototype cannot fully distinguish between multiple tasks. The MLP network has nonlinear projection capabilities and can be combined with task soft labels to enhance the distribution of high-order features across different tasks, helping to optimize the representation of task-related prototypes.

[0032] S305 local double prototype vector Perform L2 norm normalization processing, and the calculation formula is as follows: , In this step, the local double prototype vector L2 norm normalization is performed to ensure that prototype vectors from different clients are comparable within the same feature space. The scales of locally extracted feature vectors can vary significantly across devices and scenarios. Without normalization, prototypes are incomparable (making effective similarity / distance calculations impossible).

[0033] S306 Check global prototype vector With the local double prototype vector The numerical stability of the ,and its characterization capability was verified by local small sample ,tests; In this step, numerical stability is ensured, for example, by the presence of non-null values and the absence of abnormal outliers. In reality, many devices / clients have sparse local data, and including extreme values when averaging can significantly reduce the discriminative power of the aggregated prototype. This small-sample test prevents meaningless vectors (such as all zeros or infinite values) from affecting subsequent aggregation and task decisions.

[0034] S307 pairs of generated local double prototype vectors Perform lightweight encryption and temporarily store it in the local secure cache area, ready for uploading to the server.

[0035] In this step, lightweight encryption, such as differential privacy noise addition, ensures the security of subsequent data uploads. Although the prototype is safer than uploading the original data, it is possible for the attacker to guess some sensitive features of the original data from the prototype vector through inversion or inference attacks. Compared with heavy encryption or fully homomorphic encryption, "lightweight encryption" such as differential privacy / Gaussian noise has almost no pressure on communication / computation, can be uploaded in batches, and will not greatly hinder the effectiveness of federated aggregation. Because it is a public prototype, cracking it will not cause data leakage on a single client. The upload server only needs to locally double the prototype vector. Just upload it, the global prototype server has a backup.

[0036] The S4 server dynamically clusters the local prototype vectors and global parameters of each client into several client groups, and obtains the cluster center, clusterer parameters and personalized model; S401 The server receives the heterogeneous dual prototype features of the client and embeds the meta-clusterer Perform dynamic clustering and divide client groups , update cluster centers ; S4011 loads pre-trained deep embedding clusterer , where z represents the low-dimensional embedding representation of the double prototype vector, represents the clusterer parameters, the deep embedding clusterer Includes: Encoder and cluster assignment layer , and initialize the cluster centers As a result of K-means of random vectors based on local dual prototype vectors; In this step, if Figure 3 and Figure 4 As shown, in the encoder During the construction process, a stacked autoencoder (SAE) is used to initialize the deep embedding clusterer DEC. The SAE network is initialized layer by layer. Each layer is a denoised autoencoder that is trained to reconstruct the output of the previous layer after random destruction. The denoising autoencoder is a two-layer neural network defined as:

[0037] Among them, Dropout is a random mapping that randomly sets a part of its input dimension to 0. and are the activation functions used for the encoding and decoding layers respectively. By minimizing the least squares loss After training one layer, its output h is used as input to train the next layer. Each end-device scenario and data distribution vary, and directly clustering high-dimensional prototypes can be sparse and inefficient. Deep encoders significantly improve feature expression capabilities, reduce dimensionality, eliminate noise, and enhance discriminability.

[0038] S4012 decrypts the local double prototype vector Input encoder , generating a low-dimensional embedding representation , the calculation formula is as follows:

[0039] ; S4013 by clustering allocation layer Calculate each low-dimensional embedding representation Cluster centers The soft allocation probability is calculated as follows: , in, Represents the client Belong to a cluster group Soft assignment probability of ; S4014 is based on the soft allocation probability Calculating target distribution , enhance the weight of high confidence assignment, and the calculation formula is as follows:

[0040] , in, Cluster groups Frequency estimation of , used to balance the category distribution; S4015 calculates clustering loss based on KL divergence , and back-propagate to update the clusterer parameters and cluster centers , the calculation formula is as follows: ; S4016 uses gradient descent algorithm to update cluster centers , the calculation formula is as follows: , in, is the learning rate, ensuring that the center vector gradually approaches the true feature distribution; S4017 is based on the final soft allocation probability , divide the clients into C groups , the rules are: , Among them, argmax means that Traverse all cluster categories, Maximum The value of In this step, if Figure 3 As shown, each client belongs to the group with the highest probability, and further stores the updated cluster center , clusterer parameters The grouping results are then encrypted and sent to the corresponding client to guide subsequent personalized aggregation. Due to the varying data environments of each device, grouping and aggregation allows nodes with similar distributions and tasks to be clustered, resulting in more accurate personalized global weighting. Clustering balances the impact of samples, preventing large-scale nodes from dominating small-scale nodes, and improving overall model fairness. Outliers are grouped into marginal clusters, minimizing their impact on the overall model.

[0041] The updated cluster centers are sent to the client to guide the local model to adjust the feature extraction layer, and the teacher model is returned to the corresponding group client as follows: S402 Based on the clustering results, the server performs weighted aggregation on the client model parameters in the same group to generate a personalized global model and its parameters ; S4021 server loads client groups divided by dynamic clustering , verify the integrity and logical consistency of group division; In this step, the integrity and logical consistency of the group division can be verified by checking the uniqueness of the client ID.

[0042] S4022 for each group , receive local model parameters from the corresponding client and prototype embedding vector , filter invalid and timeout unresponsive client data; S4023 is represented by the client's low-dimensional embedding and group cluster centers The similarity is calculated by normalizing the weight. The calculation formula is as follows: , in, is the temperature coefficient, which controls the concentration of weight distribution; S4024 pairs of groups Model parameters for all clients within Perform weighted averaging according to the weights to generate personalized global model parameters. The calculation formula is as follows: ; In this step, the purpose is to ensure that the contribution of client models with similar feature distributions matches their similarity.

[0043] S4025 Testing the Personalization Model Using the Server The generalization performance of the model is calculated, and indicators such as precision and recall are calculated, and models with performance below the threshold are eliminated; In this step, the server can use the validation set or prototype vectors across groups to test the performance of the personalized model.

[0044] S4026 will optimize the personalized model parameters With the updated cluster center Pack and encrypt the data and send it to the clients in the corresponding group.

[0045] The S5 server sends the updated cluster centers, clusterer parameters, and personalized models to the corresponding clients, continuously optimizing the feature space distribution and model parameters of each client. S501 The server will update the personalized model parameters , cluster center and clusterer parameters Package by group, attach version identifier and metadata, and send to the corresponding client; In this step, metadata includes aggregation rounds, client lists, etc.

[0046] S502 After receiving the encrypted data packet, the client verifies the legitimacy of the digital signature and decrypts it to obtain the personalized model parameters, cluster center and related configuration information; S503: The personalized model parameters sent by the server Align with the local teacher model architecture and replace the local teacher model Parameters of the task decision layer Local optimization parameters of ; S504 based on new cluster centers , by fine-tuning the local teacher model The weight of , which minimizes the distance between the feature representation and the cluster center, is calculated as follows: ; In this step, the distance between the feature representation and the cluster center is calculated to ensure the consistency of feature distribution and group.

[0047] Further, if Figure 2 As shown in the figure, after the server sends the data, it can use three types of losses to optimize the local model to align with the latest optimization target of the server. Specifically: S505 combined with discriminant loss , distillation loss and prototype loss , jointly optimize local models; In this step, the local model and its local dual prototype vector are optimized through the loss function , to ensure that the feature space is adapted to the task requirements. The specific steps are as follows: S5051 based on local dual prototype vector Prototype center aggregated with the server , calculate the prototype alignment loss: , in, is the mean square error or cosine similarity loss, is the cluster center; S5052 integrates the three types of losses into the total loss according to their weights , the calculation formula is as follows: ; In this step, it is used to ensure the balance between task classification accuracy, knowledge transfer and feature space alignment.

[0048] S5053 calculates total loss through automatic differentiation Local model parameters The gradient of , and use the optimizer to update the parameters, while freezing the global shared parameters To maintain federal consistency.

[0049] In this step, you can track 、 and If a loss term dominates the training, the prototype loss is adaptively adjusted. Parameters or distillation loss Parameters , to prevent model bias.

[0050] S6 deploys the optimized personalized model to the client to detect real-time violations at the construction site, and continuously updates the personalized model with real-time data.

[0051] S601 will optimize the personalized model Convert to a lightweight format and perform model pruning or quantization based on the computing power and memory of the target edge device to ensure real-time inference efficiency; S602 deploys the model on edge devices at the construction site, integrates it into the existing violation detection system, and configures input and output interfaces; In this step, edge devices at the construction site include cameras, sensor nodes, etc. Input and output interfaces such as video stream parsing and alarm triggering are configured.

[0052] S603 collects image data of the construction scene in real time through edge devices, pre-processes it, and inputs it into the model for inference execution; In this step, preprocessing includes normalization and keyframe extraction.

[0053] S604 uses the deployed model to perform reasoning on the input data, outputs violation behavior detection results, and triggers graded alarms; In this step, the detection results include, for example, failure to fasten a safety rope, out-of-bounds operation, etc., and the alarms include, for example, sound and light prompts, and notifications from the management side.

[0054] S605 caches the detected abnormal data fragments locally on the edge device and generates incremental training samples by combining manual review or semi-automatic annotation tools. , used for subsequent model updates; S606 monitors the computing load, memory usage, and energy consumption of edge devices in real time, dynamically adjusting the model inference frequency or resolution to prevent device overload and system paralysis. S607 regularly generates deployment reports and optimizes model versions and update strategies through the server-edge feedback link to form a closed-loop optimization.

[0055] In this step, the deployment report may include response delay, detection accuracy, false alarm rate, etc.

[0056] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0057] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0058] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0059] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0060] It should be noted that in the claims, any reference signs placed between parentheses shall not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claim. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention can be implemented by means of hardware comprising several distinct components and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The use of the words first, second, third etc. does not indicate any order. These words may be interpreted as names.

[0061] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0062] Obviously, those skilled in the art may make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, if such modifications and variations fall within the scope of the claims and their equivalents, the present invention is intended to include such modifications and variations.

[0063] In the present invention, unless otherwise expressly specified or limited, the terms "mounted," "connected," "connect," "fixed," etc. should be understood broadly. For example, they may refer to fixed connection, detachable connection, or integration; mechanical connection or electrical connection; direct connection or indirect connection through an intermediate medium; internal communication between two components or interaction between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0064] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic expressions of the above terms should not be understood as necessarily referring to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

Claims

1. A personalized federated learning method for detecting traffic violations, characterized by: include: Use the violation training sample library to train the basic feature extraction model, deploy a global feature extraction layer on the server, distribute the basic feature extraction layer to each client, and generate dynamic task descriptions through a large language model; The basic feature extraction layer of each client is trained using local data and performs knowledge distillation learning using the teacher model sent by the server. Each client extracts the mean features of local data to form a local prototype vector, and uploads each client's local prototype vector and its global parameters to the server; The server dynamically clusters the local prototype vectors and global parameters of each client into several client groups, and obtains the cluster center, clusterer parameters and personalized model; The server sends the updated cluster centers, clusterer parameters, and personalized models to the corresponding clients, continuously optimizing the feature space distribution and model parameters of each client. The optimized personalized model is deployed to the client to detect real-time violations at the construction site, and the personalized model is continuously updated with real-time data.

2. A personalized federated learning violation detection method according to claim 1, characterized in that: The basic feature extraction layer of each client is trained using local data and knowledge distilled using the teacher model sent by the server, including: Each client performs distributed model training using a feature decoupling network and task adapter combined with local data to calculate local classification loss. To optimize local model parameters ; The client uses the server's global model as the teacher model and calculates the distillation loss through KL divergence. , guiding the student model to train.

3. A personalized federated learning violation detection method according to claim 2, characterized in that: Each client performs distributed model training by combining the feature decoupling network with the task adapter and local data to calculate the local classification loss. To optimize local model parameters include: Each client loads the local dataset , for local datasets Input data Perform standardization, denoising and enhancement, and associate task labels Generate training batches; The client extracts the initial feature based on the data sent by the server. and task decision-making layer , initialize the client local model ,in, Represent local model parameters to ensure that the model architecture is suitable for the task; Local data Input to the client's feature extraction layer , generate high-dimensional feature representation , and passed to the client's task decision layer Make predictions; Predict the probability based on the output of the task decision layer Associated task tags , local classification loss is calculated by cross entropy Quantify the prediction error, where C is the number of all identifiable categories; Using the gradient descent algorithm, backpropagate the local classification loss , optimize the feature extraction layer of the local model and task decision-making layer Parameters, update local model parameters ; The trained local model parameters and feature extraction layer weights Temporarily stored on the client.

4. The personalized federated learning violation detection method according to claim 2, characterized in that: The client uses the server's global model as the teacher model and calculates the distillation loss through KL divergence , guiding the student model to train includes: The client downloads the latest global model from the server As the teacher model, get its parameters Used for knowledge distillation; Based on the client's current local model Parameters , initialize the student model architecture to maintain input and output consistency with the teacher model; Enter local data To Teacher Model , generate the softened probability distribution through forward propagation ,in, is the teacher model output, is the temperature coefficient; The same local data Enter the local student model , generating a softened probability distribution , and record the output features ; The distillation loss of the teacher and student models is calculated based on the KL divergence. The calculation formula is as follows: ， in, Used to adjust the smoothness of probability distribution to avoid overfitting hard labels; Distillation losses With local classification loss The weighted sum is used to get the total loss, which is calculated as follows: in, is the distillation weight coefficient, which controls the strength of knowledge transfer; Backpropagate the total loss through the gradient descent algorithm , update the local parameters of the student model , while freezing the global shared parameters To protect federated knowledge consistency.

5. The personalized federated learning violation detection method according to claim 1 is characterized in that: Each client extracts the mean feature of local data to form a local prototype vector, and uploads the local prototype vector of each client and its global parameters to the server, including: The client will local data set Divided into small batch groups, through the feature extraction layer Calculate the feature vector of each sample batch by batch ; Take the mean of the feature vectors of all samples in the local data set to generate the global prototype vector. The calculation formula is as follows: ， in, For the client The total number of samples, Characterize the overall feature distribution of local data; According to the task tag issued by the server , load the corresponding task-specific mapping parameters , this parameter is defined by the task soft label generated by the server; The global prototype vector Input task-specific MLP network to generate local dual prototype vectors , the calculation formula is as follows: ； For local double prototype vector Perform L2 norm normalization processing, and the calculation formula is as follows: ， Check the global prototype vector With the local double prototype vector The numerical stability of the ,and its characterization capability was verified by local small sample ,tests; The generated local double prototype vector Perform lightweight encryption and temporarily store it in the local secure cache area, ready for uploading to the server.

6. The personalized federated learning violation detection method according to claim 1, characterized in that: The server dynamically clusters the local prototype vectors and global parameters of each client into several client groups, and obtains cluster centers, clusterer parameters and personalized models including: The server receives the heterogeneous dual prototype features from the client and embeds them into a meta-clusterer Perform dynamic clustering and divide client groups , update cluster centers ; Based on the clustering results, the server performs weighted aggregation on the client model parameters within the same group to generate a personalized global model and its parameters. .

7. A personalized federated learning violation detection method according to claim 6, characterized in that: The server receives the heterogeneous dual prototype features of the client and embeds the meta-clusterer Perform dynamic clustering and divide client groups , update cluster centers include: Loading a pre-trained deep embedding clusterer , where z represents the low-dimensional embedding representation of the double prototype vector, represents the clusterer parameters, the deep embedding clusterer Includes: Encoder and cluster assignment layer , and initialize the cluster centers As a result of K-means of random vectors based on local dual prototype vectors; The decrypted local double prototype vector Input encoder , generating a low-dimensional embedding representation , the calculation formula is as follows: ； Assigning layers via clustering Calculate each low-dimensional embedding representation Cluster centers The soft allocation probability is calculated as follows: ， in, Represents the client Belong to a cluster group Soft assignment probability of ; According to the soft allocation probability Calculating target distribution , enhance the weight of high confidence assignment, and the calculation formula is as follows: ， in, Cluster groups Frequency estimation of , used to balance the category distribution; Calculate clustering loss based on KL divergence , and back-propagate to update the clusterer parameters and cluster centers , the calculation formula is as follows: ； Use gradient descent algorithm to update cluster centers , the calculation formula is as follows: ， in, is the learning rate, ensuring that the center vector gradually approaches the true feature distribution; According to the final soft allocation probability , divide the clients into Groups , the rules are: ， Among them, argmax means that Traverse all cluster categories, Maximum The value of .

8. The personalized federated learning violation detection method according to claim 6, characterized in that: According to the clustering results, the server performs weighted aggregation on the client model parameters in the same group to generate a personalized global model and its parameters. include: The server loads the client groups divided by dynamic clustering , verify the integrity and logical consistency of group division; For each group , receive local model parameters from the corresponding client and prototype embedding vector , filter invalid and timeout unresponsive client data; Based on the client's low-dimensional embedding representation and group cluster centers The similarity is calculated by normalizing the weight. The calculation formula is as follows: ， in, is the temperature coefficient, which controls the concentration of weight distribution; For groups Model parameters for all clients within Perform weighted averaging according to the weights to generate personalized global model parameters. The calculation formula is as follows: ； Testing the Personalization Model on the Server The generalization performance of the model is calculated and the model with performance below the threshold is eliminated; The optimized personalized model parameters With the updated cluster center Pack and encrypt the data and send it to the clients in the corresponding group.

9. The personalized federated learning violation detection method according to claim 1, characterized in that: The server sends the updated cluster centers, clusterer parameters, and personalized models to the corresponding clients, and continuously optimizes the feature space distribution and model parameters of each client, including: The server will update the personalized model parameters , cluster center and clusterer parameters Package by group, attach version identifier and metadata, and send to the corresponding client; After receiving the encrypted data packet, the client verifies the legitimacy of the digital signature and decrypts it to obtain personalized model parameters, cluster centers, and related configuration information; The personalized model parameters sent by the server Align with the local teacher model architecture and replace the local teacher model Parameters, retain the task decision layer Local optimization parameters of ; Based on the new cluster center , by fine-tuning the local teacher model The weight of , which minimizes the distance between the feature representation and the cluster center, is calculated as follows: ； Combined with discriminative loss , distillation loss and prototype loss , jointly optimize local models.

10. A personalized federated learning violation detection method according to claim 9, characterized in that: The combined discriminative loss , distillation loss and prototype loss , the joint optimization of local models includes: Based on local dual prototype vector Prototype center aggregated with the server , calculate the prototype alignment loss: ， in, is the mean square error or cosine similarity loss, is the cluster center; Integrate the three types of losses into the total loss by weight , the calculation formula is as follows: ； Computing the total loss via automatic differentiation Local model parameters The gradient of , and use the optimizer to update the parameters, while freezing the global shared parameters To maintain federal consistency.

Images