A personalized federated learning method for traffic violation detection

Abstract

This invention discloses a personalized federated learning method for detecting traffic violations, comprising: training a basic feature extraction model using a violation training sample library; deploying a global feature extraction layer on the server; distributing the basic feature extraction layer to each client and generating a dynamic task description; training the basic feature extraction layer of each client using local data and performing knowledge distillation learning through a teacher model distributed by the server; extracting the mean features of local data to form a local prototype vector for each client, and uploading the local prototype vector and its global parameters of each client to the server; dynamically clustering the local prototype vectors and their global parameters of each client into several client groups to obtain cluster centers, clusterer parameters, and personalized models; and continuously optimizing the feature space distribution and model parameters of each client by distributing the updated cluster centers, clusterer parameters, and personalized models to the corresponding clients.

Description

Technical Field

[0001] This invention relates to the field of violation detection technology, and in particular to a violation detection method based on personalized federated learning. Background Technology

[0002] Traditional construction safety supervision heavily relies on centralized data collection and analysis technologies, such as capturing worker behavior data in real time through construction site cameras, sensors, and wearable devices and uploading it to a central server for model training. However, this model faces multiple challenges: data from different contractors and project teams in construction scenarios often form strictly isolated "data silos" due to the involvement of trade secrets and worker privacy, making it difficult for a single organization to obtain enough violation samples to train a robust model; the highly dynamic nature of the construction site environment (such as changes in the layout of temporary facilities and cross-operations of multiple trades) makes it difficult for traditional centralized models to adapt to the differences in data distribution across different project scenarios, easily leading to false positives and false negatives; at the same time, the computing power requirements of complex deep learning models on edge devices are significantly contradictory to the low-power IoT terminals commonly found on construction sites, creating a conflict between real-time detection needs and computing resource limitations.

[0003] Federated learning's distributed collaborative computing framework offers a breakthrough path. Its "data remains stationary while the model moves" characteristic satisfies both personal information privacy protection and compliance requirements for biometric information of construction workers, and enables knowledge sharing through the interaction of model parameters among multiple participants. However, the application of standard federated learning in construction scenarios still has limitations—a unified global model is difficult to adapt to the personalized characteristics of different construction stages and different types of work (such as high-altitude operations versus underground construction), which may lead to a decrease in the sensitivity of identifying violations in specific scenarios (such as not wearing safety ropes or machinery operating beyond its boundaries).

[0004] The purpose of this invention is to design a personalized federated learning method for detecting traffic violations, addressing the problems existing in the prior art. Summary of the Invention

[0005] In view of this, the purpose of this invention is to propose a personalized federated learning method for detecting violations, which can solve the above-mentioned problems.

[0006] This invention provides a method for detecting traffic violations using personalized federated learning, comprising:

[0007] A basic feature extraction model is trained using a training sample library of traffic violations. A global feature extraction layer is deployed on the server side, and the basic feature extraction layer is distributed to each client. A dynamic task description is generated through a large language model.

[0008] Each client's basic feature extraction layer is trained using local data and learns through knowledge distillation using a teacher model distributed by the server.

[0009] Each client extracts the mean feature of its local data to form a local prototype vector, and uploads each client's local prototype vector and its global parameters to the server.

[0010] The server dynamically clusters the local prototype vectors and their global parameters of each client into several client groups, obtaining cluster centers, clusterer parameters, and personalized models.

[0011] The server will distribute the updated cluster centers, clusterer parameters, and personalized models to the corresponding clients, continuously optimizing the feature space distribution and model parameters for each client;

[0012] The optimized personalized model is deployed to the client to detect violations at the construction site in real time, and the personalized model is continuously updated with real-time data.

[0013] The beneficial effects of this invention are:

[0014] First, by decoupling local features and sharing parameters to unify global feature extraction, all clients can learn a shared, high-quality basic representation, improving generalization ability for new or limited clients. The model structure can be quickly transferred to new scenarios and fine-tuned for each client, ensuring model personalization and synergistic effects. KL divergence distillation is performed using the latest server-side global model (teacher) and local model (student). The distillation loss is jointly optimized with the local classification loss to address the problem of model overfitting to local models and losing global generalization due to sparse or offset local samples.

[0015] Secondly, by extracting prototype vectors using the local feature mean, the technical problems of high communication overhead and privacy exposure caused by uploading all original samples or features are solved. The mean prototype vector greatly compresses the transmission volume and can effectively protect sensitive user data. By generating local dual prototypes through task-specific MLP mapping, the inability of a single prototype to cover all complex business requirements and scenario feature distributions is avoided, ensuring that the prototype space of each end is highly matched with the actual business task.

[0016] Third, a deep embedding clusterer (DEC) is used to reduce the dimensionality of the local prototype vectors uploaded by the client through a stacked denoising autoencoder, encoding more robust and discriminative low-dimensional embedding features. The weights of high-confidence samples are increased by soft-assignment probability and target distribution (based on KL divergence). The server uses the distance between the low-dimensional embedding and the cluster center to measure the distance between each end and the group center. Based on this, the end-side model parameters are weighted and summed to output a personalized global model specific to the group.

[0017] Fourth, by dynamically distributing personalized models and cluster centers, we improve the personalization, adaptability, and security of the edge devices, preventing performance degradation caused by model / cluster center drift. Different edge devices have heterogeneous data distributions; if they cannot dynamically synchronize with the latest centers, they will drift towards the global model or outside the clusters, weakening collaboration. Therefore, by fine-tuning the feature extraction layer using the latest cluster centers from the server, the feature distribution of the local data automatically aligns with the group centers. The global feature layer is frozen during the local training phase, optimizing only the local decision head and personalized parameters, protecting shared knowledge from being contaminated by extreme deviations in edge data. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings required in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart of the method in this embodiment.

[0020] Figure 2 This is a flowchart of the client-side method implementation in this embodiment.

[0021] Figure 3 This is a flowchart of the server-side method implementation in this embodiment.

[0022] Figure 4 This is a flowchart illustrating the implementation of the deep aggregation method in this embodiment. Detailed Implementation

[0023] To facilitate understanding by those skilled in the art, the structure of the present invention will now be described in further detail with reference to the accompanying drawings. It should be understood that, unless otherwise specified, the order of the steps mentioned in this embodiment can be adjusted according to actual needs, and they can even be executed simultaneously or partially simultaneously.

[0024] like Figure 1 As shown, this embodiment of the invention provides a method for detecting traffic violations using personalized federated learning, including:

[0025] S1 trains the basic feature extraction layer using a training sample library of violations, deploys the global feature extraction layer on the server, distributes the basic feature extraction layer to each client, and generates dynamic task descriptions through a large language model.

[0026] S101 collects and cleans publicly available datasets of traffic violations, performs data augmentation, normalization, and annotation, and builds a standardized training sample library.

[0027] S102 is based on a pre-trained model and uses a public dataset to train a global basic feature extraction layer through transfer learning. Optimize its feature representation capabilities and initialize the model weights for the client. ;

[0028] In this step, the pre-trained model can be ResNet, BERT, etc.

[0029] S103 uses a large language model to describe task types, generates a multi-dimensional soft label matrix, and assigns corresponding task soft labels based on the client's business scenario.

[0030] In this step, a large language model is used to describe the task types (such as "violation of regulations for working at heights" and "not wearing a safety helmet"). Large language models include ChatGPT, GPT-3 / 4 / 4o, ERNIE3.0, GLM, LLaMA, and ChatGLM. These textual descriptions are transformed into low-dimensional task feature vectors / soft label matrices through word encoding and embedding transformation. Each task receives a set of these soft labels (such as task distribution vectors) for each client, which are then used for subsequent model task adaptation and discriminant head parameter settings. This addresses the problems of diverse task categories, non-standard descriptions, and the error-prone and difficult-to-expand nature of manually defining the task space in real-world large-scale construction scenarios.

[0031] S104 will extract the basic feature layer The global model architecture and task soft tags are distributed to each client to complete the initial client deployment of the local model.

[0032] The basic feature extraction layer of each S2 client is trained using local data and learns through knowledge distillation using the teacher model distributed by the server.

[0033] In S201, each client uses a feature decoupling network and a task adapter to perform distributed model training using local data, and calculates the local classification loss. To optimize local model parameters ;

[0034] S2011 clients load local datasets For local datasets Input data Standardization, noise reduction, and enhancement processes are performed, and task tags are associated with them. Generate training batches;

[0035] In this step, it is assumed that there are One client, each client Each has a unique private dataset, represented as ,in express Size.

[0036] The S2012 client is based on the initial feature extraction layer issued by the server. and task decision-making level Initialize the client-side local model ,in, This indicates local model parameters, ensuring that the model architecture is adapted to the task.

[0037] In this step, such as Figure 2 As shown, the local parameters of the client model are decoupled into a feature extraction layer. and task decision-making level To adapt to the distribution of client data, the model weights in the global part Knowledge is shared across all clients. Each client has its own model. As the weight parameters of its model, Defined as the dimension of the model parameter vector, for each client. of By using personalization techniques, the model is divided into local and global parts, forming a client-side local model. .

[0038] S2013 will use local data Feature extraction layer input to the client Generate high-dimensional feature representations And pass it to the client's task decision layer. Make predictions;

[0039] S2014 predicts probabilities based on the output of the task decision layer. Related task tags Calculate the local classification loss using cross-entropy. Quantify the prediction error, where R is the number of all identifiable categories;

[0040] In this step, C=9, comprising five major categories and nine subcategories. 1. Failure to wear safety protective equipment: not wearing a safety helmet, not wearing a safety belt when working at heights; 2. Clothing violations: wearing slippers, sandals, high heels, shorts, or skirts to the construction site; 3. Behavioral violations: smoking in no-smoking areas, horseplay on site, or standing on conveyor belts; 4. Lack of protection: not having guardrails or safety nets for working near edges; 5. Operational violations: overloading during hoisting, or standing or walking under suspended loads.

[0041] S2015 utilizes the gradient descent algorithm to backpropagate the local classification loss. Optimize the feature extraction layer of the local model and task decision-making level Update local model parameters. ;

[0042] S2016 will use the trained local model parameters and feature extraction layer weights Temporarily stored on the client.

[0043] In this step, if all client models only learn from their own data, the models can only adapt to a limited range and lack generalization ability. If all models use globally uniform parameters, it is difficult to perfectly adapt to every complex and varied distribution. Therefore, by sharing parameters without transmitting the original training data, privacy is guaranteed while ensuring the model training effect.

[0044] The S202 client uses the server's global model as the teacher model and calculates the distillation loss using KL divergence. They guide students in training their models.

[0045] The S2021 client downloads the latest global model from the server. As a teacher model, obtain its parameters. Used for knowledge distillation;

[0046] S2022 is based on the client's current local model. parameters Initialize the student model architecture to maintain input-output consistency with the teacher model;

[0047] S2023 Input Local Data To Teacher Model The softening probability distribution is generated through forward propagation. ,in, Output for the teacher model, Temperature coefficient;

[0048] S2024 will use the same local data Input local student model Generate softening probability distribution And record the output features ;

[0049] S2025 calculates the distillation loss of the teacher and student models based on KL divergence, using the following formula:

[0050] ,

[0051] in, Used to adjust the smoothness of the probability distribution and avoid overfitting hard labels;

[0052] In this step, based on the output of the task decision layer... With real labels The prediction error is quantified by calculating cross-entropy loss.

[0053] S2026 will distillation loss Local classification loss The total loss is obtained by weighted summation, and the calculation formula is as follows:

[0054]

[0055] in, The distillation weighting coefficient controls the intensity of knowledge transfer.

[0056] S2027 backpropagates the total loss using the gradient descent algorithm. Update the local parameters of the student model At the same time, freeze globally shared parameters. To protect the consistency of federal knowledge.

[0057] In this step, such as Figure 2 As shown, the local student model, guided by the soft objective of the global teacher model, optimizes the local classification loss by combining KL divergence distillation with local classification loss. On the one hand, it inherits global knowledge to enhance generalization, and on the other hand, it finely adapts to local distribution. This is an important mechanism to improve the effectiveness and security of personalized federated scenario models.

[0058] S3 extracts the mean feature of local data from each client to form a local prototype vector, and uploads the local prototype vector of each client and its global parameters to the server.

[0059] The S301 client will use the local dataset Divided into small batches, and processed through a feature extraction layer. Calculate the feature vector of each sample in batches ;

[0060] S302 takes the mean of the feature vectors of all samples in the local dataset to generate a global prototype vector. The calculation formula is as follows:

[0061] ,

[0062] in, For the client The total number of samples, Characterizes the overall feature distribution of local data;

[0063] In this step, the global prototype vector can represent the overall data distribution information, greatly compressing the communication volume. It is more efficient and secure than uploading all samples. Taking the mean can suppress the influence of extreme data and improve feature stability.

[0064] S303 based on the task tags issued by the server Load the corresponding task-specific mapping parameters This parameter is defined by the task soft tag generated by the server;

[0065] S304 will use the global prototype vector Input a task-specific MLP network to generate local dual prototype vectors. The calculation formula is as follows:

[0066] ;

[0067] In this step, such as Figure 2 As shown, the feature vectors of all samples in the local dataset are averaged using a prototype generator, and nonlinear transformations and feature optimizations are performed using a task-specific MLP network. The network contains fully connected layers and non-linear activation functions, with output dimensions and a global prototype vector. Matching is crucial. Different businesses / scenarios / tasks (such as different types of violations) require different feature distribution spaces for discrimination, and a single prototype cannot adequately distinguish between multiple tasks. MLP networks possess non-linear projection capabilities, which can be combined with task soft labels to enhance the high-order feature distribution under different tasks, thus contributing to the optimization of task-related prototype representations.

[0068] S305 for local dual prototype vectors The L2 norm normalization process is performed, and the calculation formula is as follows:

[0069] ,

[0070] In this step, the local dual prototype vectors are... L2 norm normalization is performed to ensure that prototype vectors from different clients are comparable within a unified feature space. Different devices and scenarios may have significantly different scales for locally extracted feature vectors. Without normalization, prototypes cannot be compared (effective similarity / distance calculations are impossible).

[0071] S306 Check the global prototype vector With local double prototype vectors The numerical stability of the model was verified, and its characterization ability was validated through local small sample testing.

[0072] In this step, numerical stability is ensured, for example, by ensuring non-null values ​​and the absence of outliers. In reality, many devices / clients have sparse local data, and if extreme values ​​are included when calculating the mean, it will significantly reduce the discriminative power of the aggregation prototype. This small sample test can prevent meaningless vectors (such as all zeros or infinity) from affecting subsequent aggregation and task decisions.

[0073] S307 generates local dual prototype vectors Perform lightweight encryption and temporarily store the data in a secure local cache before uploading it to the server.

[0074] In this step, lightweight encryption, such as differential privacy noise, is added to ensure the security of subsequent data uploads. Although the prototype is more secure than the uploaded original data, attackers may be able to guess some sensitive features of the original data from the prototype vector through inversion or inference attacks. Compared to heavy encryption or fully homomorphic encryption, "lightweight encryption" such as differential privacy / Gaussian noise puts almost no strain on communication / computation, allows for batch uploads, and does not significantly hinder the effectiveness of federated aggregation. Because it is a public prototype, even if it is cracked, it will not cause data leakage to any particular client; uploading to the server only requires local dual prototype vectors. Simply upload; the global prototype server has a backup.

[0075] The S4 server dynamically clusters the local prototype vectors and their global parameters of each client into several client groups, obtaining cluster centers, clusterer parameters, and personalized models.

[0076] The S401 server receives heterogeneous dual-prototype features from the client and uses a deep embedding meta-clusterer. Perform dynamic clustering to divide client groups Update cluster centers ;

[0077] S4011 loads a pre-trained deep embedding clusterer , where z represents the low-dimensional embedding representation of the two prototype vectors. Indicates the clusterer parameters, the deep embedding clusterer Includes: encoder and clustering allocation layer and initialize cluster centers. As a result of K-means based on local dual-prototype vectors for random vectors;

[0078] In this step, such as Figure 3 and Figure 4 As shown, in the encoder During construction, a stacked autoencoder (SAE) is used to initialize the deep embedding clusterer (DEC). The SAE network is initialized layer by layer, with each layer being a denoised autoencoder. After training, it can reconstruct the output of the previous layer after random destruction. The denoised autoencoder is a two-layer neural network, defined as follows:

[0079]

[0080] Dropout is a random mapping that randomly sets a portion of its input dimensions to 0. and These are the activation functions used for the encoding and decoding layers, respectively. The least squares loss is minimized. Training is then performed. After training one layer, its output h is used as input to train the next layer. Each end device has different scenarios and data distributions, and directly clustering high-dimensional prototypes may be sparse and inefficient. Deep encoders significantly improve feature representation capabilities, reduce dimensionality to eliminate noise, and enhance discriminative power.

[0081] S4012 will decrypt the local dual prototype vector Input encoder Generate low-dimensional embedding representations The calculation formula is as follows:

[0082]

[0083] ;

[0084] S4013 assigns layers through clustering. Compute each low-dimensional embedding representation Cluster centers The soft assignment probability is calculated using the following formula:

[0085] ,

[0086] in, Indicates client Belongs to cluster group The soft assignment probability;

[0087] S4014 is based on soft allocation probability Calculate the target distribution To enhance the weighting of high-confidence assignments, the calculation formula is as follows:

[0088]

[0089] ,

[0090] in, For clustering groups Frequency estimation for balancing class distributions;

[0091] S4015 calculates clustering loss based on KL divergence. And backpropagate to update clusterer parameters With cluster center The calculation formula is as follows:

[0092] ;

[0093] S4016 uses the gradient descent algorithm to update the cluster centers. The calculation formula is as follows:

[0094] ,

[0095] in, The learning rate ensures that the center vector gradually approximates the true feature distribution;

[0096] S4017 is based on the final soft allocation probability. Divide the clients into C groups The rules are as follows:

[0097] ,

[0098] Where argmax represents letting Iterate through all cluster categories, Maximum The value of ;

[0099] In this step, such as Figure 3 As shown, each client is assigned to the group with the highest probability. Furthermore, the updated cluster centers are stored. Clusterer parameters The grouping results are then encrypted and sent to the corresponding clients to guide subsequent personalized aggregation. Due to different data environments on different devices, grouping and aggregation clusters similar task nodes with the same distribution into clusters, making personalized global weights more accurate. Clustering can balance the influence of samples, prevent large-scale clusters from overwhelming small-scale clusters, improve the overall fairness of the model, and reduce the impact of outliers on the overall model by grouping them into marginal clusters.

[0100] The updated cluster centers are sent to the client to guide the local model in adjusting the feature extraction layer. Simultaneously, the teacher model is returned to the corresponding group client, as detailed below:

[0101] Based on the clustering results, S402 performs weighted aggregation of client model parameters within the same group to generate a personalized global model and its parameters. ;

[0102] S4021 server loads dynamically clustered client groups. To verify the completeness and logical consistency of the group division;

[0103] In this step, the integrity and logical consistency of the group division can be verified by checking the uniqueness of the client ID.

[0104] S4022 for each group Receive local model parameters from the corresponding client and prototype embedding vector Filter invalid and timed-out client data;

[0105] S4023 is based on the client's low-dimensional embedding representation. Clustering Center The similarity is used to calculate the normalized weight, and the calculation formula is as follows:

[0106] ,

[0107] in, The temperature coefficient controls the concentration of the weight distribution.

[0108] S4024 group Model parameters for all clients The weighted average is calculated based on the weights to generate personalized global model parameters. The calculation formula is as follows:

[0109] ;

[0110] This step is to ensure that the contribution of client-side models with similar feature distributions matches their similarity.

[0111] S4025 utilizes server-side testing of personalized models The generalization performance is evaluated, and metrics (such as accuracy and recall) are calculated and models with performance below the threshold are removed.

[0112] In this step, the server can use a validation set or cross-group prototype vectors to test the performance of the personalized model.

[0113] S4026 will optimize the personalized model parameters With the updated cluster centers The package is encrypted and then distributed to the corresponding group's clients.

[0114] The S5 server will distribute the updated cluster centers, clusterer parameters, and personalized models to the corresponding clients, continuously optimizing the feature space distribution and model parameters for each client;

[0115] The S501 server will update the personalized model parameters. Cluster Center and clusterer parameters Packaged by group, and with version identifiers and metadata attached, sent to the corresponding clients;

[0116] In this step, metadata includes aggregation rounds, client lists, etc.

[0117] After receiving the encrypted data packet, the S502 client verifies the legality of the digital signature, decrypts it, and obtains the personalized model parameters, cluster centers, and related configuration information.

[0118] S503 will send personalized model parameters from the server. Align with the local teacher model architecture and replace the local teacher model. The parameters are retained by the task decision layer. Local optimization parameters;

[0119] S504 is based on new cluster centers. By fine-tuning the local teacher model The weights are calculated by minimizing the distance between the feature representation and the cluster center, using the following formula:

[0120] ;

[0121] In this step, calculating the minimum distance between the feature representation and the cluster center is used to ensure consistency between the feature distribution and the group.

[0122] Furthermore, such as Figure 2 As shown, after the server distributes the data, three types of losses can be used to optimize the local model in order to align with the latest optimization target on the server. Specifically:

[0123] S505 combined with loss discrimination Distillation loss and prototype loss Jointly optimize the local model;

[0124] In this step, the local model and its local dual prototype vectors are optimized using a loss function. To ensure that the feature space is adapted to the task requirements, the specific steps are as follows:

[0125] S5051 is based on local dual prototype vectors Prototype Center Aggregated with Server Calculate the prototype alignment loss:

[0126] ,

[0127] in, This refers to the mean squared error or cosine similarity loss. As cluster center;

[0128] S5052 integrates the three types of losses according to their weights into a total loss. The calculation formula is as follows:

[0129] ;

[0130] This step is used to ensure a balance between task classification accuracy, knowledge transfer, and feature space alignment.

[0131] The S5053 calculates the total loss using automatic differentiation. For local model parameters The gradient is calculated, and the optimizer updates the parameters while freezing the globally shared parameters. To maintain federal consistency.

[0132] In this step, real-time tracking is possible. , and If a certain loss term dominates the training, the prototype loss is adaptively adjusted according to the changing trend. parameters Or distillation loss parameters To prevent model bias.

[0133] S6 deploys the optimized personalized model to the client to detect violations at the construction site in real time, and continuously updates the personalized model with real-time data.

[0134] S601 will optimize the personalized model Convert to a lightweight format and perform model pruning or quantization based on the computing power and memory of the target edge device to ensure real-time inference efficiency;

[0135] The S602 deploys a model in edge devices at the construction site, integrates it into the existing violation detection system, and configures input / output interfaces;

[0136] In this step, edge devices at the construction site include cameras, sensor nodes, etc. Input / output interfaces such as video stream parsing and alarm triggering are configured.

[0137] S603 collects image data of the construction scene in real time through edge devices, preprocesses it, and then inputs it into the model to perform inference.

[0138] In this step, preprocessing includes normalization and keyframe extraction.

[0139] S604 uses the deployment model to perform inference on the input data, outputs the violation detection results, and triggers tiered alarms;

[0140] In this step, detection results such as not wearing a safety rope or operating outside designated areas will be recorded, and alarms such as audible and visual alerts and notifications from the management terminal will be issued.

[0141] The S605 caches abnormal data fragments detected locally on the edge device and generates incremental training samples by combining them with manual verification or semi-automatic annotation tools. This is used for subsequent model updates;

[0142] S606 monitors the computing load, memory usage and power consumption of edge devices in real time, and dynamically adjusts the model inference frequency or resolution to avoid system paralysis caused by device overload.

[0143] S607 generates deployment reports regularly and optimizes the model version and update strategy through the server-edge feedback link, forming a closed-loop optimization.

[0144] In this step, the deployment report may include information such as response latency, detection accuracy, and false alarm rate.

[0145] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0146] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0147] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0148] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0149] It should be noted that any reference signs placed between parentheses in the claims should not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention can be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The words first, second, and third, etc., do not indicate any order. These words can be interpreted as names.

[0150] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the invention.

[0151] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

[0152] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0153] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms should not be construed as necessarily referring to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

Claims

1. A method for detecting traffic violations using personalized federated learning, characterized in that, include: A basic feature extraction model is trained using a training sample library of traffic violations. A global feature extraction layer is deployed on the server side, and the basic feature extraction layer is distributed to each client. A dynamic task description is generated through a large language model. Each client's basic feature extraction layer is trained using local data and learns through knowledge distillation using a teacher model distributed by the server. Each client extracts the mean feature of its local data to form a local prototype vector. Each client's local prototype vector and its global parameters are then uploaded to the server. Specifically: The client will use the local dataset Divided into small batches, and processed through a feature extraction layer. Calculate the feature vector of each sample in batches ; The global prototype vector is generated by averaging the feature vectors of all samples in the local dataset, using the following formula: ， in, For the client The total number of samples, Characterizes the overall feature distribution of local data. Indicates the weights of the feature extraction layer; The client uses the task soft tags issued by the server. Load the corresponding task-specific mapping parameters This parameter is defined by the task soft tag generated by the server; The client will use the global prototype vector. The input is a task-specific MLP network, which contains fully connected layers and non-linear activation functions. The output dimensions are a global prototype vector. In matching, the MLP network has non-linear projection capabilities, and combined with task soft labels, it enhances the distribution of high-order features under different tasks, generating local dual prototype vectors. The calculation formula is as follows: ； For local dual prototype vectors The L2 norm normalization process is performed, and the calculation formula is as follows: ， Check global prototype vector With local double prototype vectors The numerical stability of the model was verified, and its characterization ability was validated through local small sample testing. For the generated local double prototype vector Perform lightweight encryption and temporarily store the data in a secure local cache, ready to be uploaded to the server. The server dynamically clusters the local prototype vectors and their global parameters of each client into several client groups, obtaining cluster centers, clusterer parameters, and personalized models. The server will distribute the updated cluster centers, clusterer parameters, and personalized models to the corresponding clients, continuously optimizing the feature space distribution and model parameters for each client; The optimized personalized model is deployed to the client to detect violations at the construction site in real time, and the personalized model is continuously updated through real-time data.

2. The method for detecting traffic violations using personalized federated learning according to claim 1, characterized in that, The basic feature extraction layer of each client is trained using local data and undergoes knowledge distillation learning through the teacher model distributed by the server, including: Each client performs distributed model training using a feature decoupling network and a task adapter, combined with local data, and calculates the local classification loss. To optimize local model parameters ; The client uses the server's global model as the teacher model and calculates the distillation loss using KL divergence. They guide students in training their models.

3. The method for detecting traffic violations using personalized federated learning according to claim 2, characterized in that, Each client performs distributed model training by combining local data with a feature decoupling network and a task adapter, and calculates the local classification loss. To optimize local model parameters include: Each client loads the local dataset For local datasets Input data Standardization, noise reduction, and enhancement processes are performed, and task tags are associated with them. Generate training batches; The client uses the initial feature extraction layer issued by the server. and task decision-making level Initialize the client-side local model ,in, This indicates local model parameters, ensuring that the model architecture is adapted to the task. Local data Feature extraction layer input to the client Generate high-dimensional feature representations And pass it to the client's task decision layer. Make predictions; Predict probabilities based on the output of the task decision layer. Related task tags Calculate the local classification loss using cross-entropy. Quantify the prediction error, where C is the number of all identifiable categories; Using the gradient descent algorithm, backpropagate the local classification loss. Optimize the feature extraction layer of the local model and task decision-making level Update local model parameters. ; The trained local model parameters and feature extraction layer weights Temporarily stored on the client.

4. The method for detecting traffic violations using personalized federated learning according to claim 2, characterized in that, The client uses the server's global model as the teacher model and calculates the distillation loss using KL divergence. Guiding students in training their models includes: The client downloads the latest global model from the server. As a teacher model, obtain its parameters. Used for knowledge distillation; Based on the client's current local model parameters Initialize the student model architecture to maintain input-output consistency with the teacher model; Input local data To Teacher Model The softening probability distribution is generated through forward propagation. ,in, Output for the teacher model, Temperature coefficient; The same local data Input local student model Generate softening probability distribution And record the output features ; The distillation loss of the teacher and student models is calculated based on KL divergence, using the following formula: ， in, Used to adjust the smoothness of the probability distribution and avoid overfitting hard labels; Distillation loss Local classification loss The total loss is obtained by weighted summation, and the calculation formula is as follows: in, The distillation weighting coefficient controls the intensity of knowledge transfer. The total loss is backpropagated using the gradient descent algorithm. Update the local parameters of the student model At the same time, freeze globally shared parameters. To protect the consistency of federal knowledge.

5. The method for detecting traffic violations using personalized federated learning according to claim 3, characterized in that, The server dynamically clusters the local prototype vectors and their global parameters of each client into several client groups, obtaining cluster centers, clusterer parameters, and personalized models, including: The server receives heterogeneous dual-prototype features from the client and uses a deep embedding meta-clusterer. Perform dynamic clustering to divide client groups Update cluster centers ; Based on the clustering results, the server performs weighted aggregation of client model parameters within the same group to generate a personalized global model and its parameters. .

6. The method for detecting traffic violations using personalized federated learning according to claim 5, characterized in that, The server receives heterogeneous dual-prototype features from the client and uses a deep embedding meta-clusterer. Perform dynamic clustering to divide client groups Update cluster centers include: Loading pre-trained deep embedding clusterers , where z represents the low-dimensional embedding representation of the two prototype vectors. Indicates the clusterer parameters, the deep embedding clusterer Includes: encoder and clustering allocation layer and initialize cluster centers. As a result of K-means based on local dual-prototype vectors for random vectors; The decrypted local biprototype vector Input encoder Generate low-dimensional embedding representations The calculation formula is as follows: ； Assignment layer through clustering Compute each low-dimensional embedding representation Cluster centers The soft assignment probability is calculated using the following formula: ， in, Indicates client Belongs to cluster group The soft assignment probability; Based on soft allocation probability Calculate the target distribution To enhance the weighting of high-confidence assignments, the calculation formula is as follows: ， in, For clustering groups Frequency estimation for balancing class distributions; Clustering loss calculated based on KL divergence. And backpropagate to update clusterer parameters With cluster center The calculation formula is as follows: ； Update cluster centers using gradient descent algorithm The calculation formula is as follows: ， in, The learning rate ensures that the center vector gradually approximates the true feature distribution; Based on the final soft allocation probability Divide the client into Groups The rules are as follows: ， Where argmax represents letting Iterate through all cluster categories, Maximum The value of .

7. The method for detecting traffic violations using personalized federated learning according to claim 6, characterized in that, Based on the clustering results, the server performs weighted aggregation of client model parameters within the same group to generate a personalized global model and its parameters. include: The server loads dynamically clustered client groups. To verify the completeness and logical consistency of the group division; For each group Receive local model parameters from the corresponding client and prototype embedding vector Filter invalid and timed-out client data; Based on the client's low-dimensional embedding representation Clustering Center The similarity is used to calculate the normalized weight, and the calculation formula is as follows: ， in, The temperature coefficient controls the concentration of the weight distribution. For groups Model parameters for all clients The weighted average is calculated based on the weights to generate personalized global model parameters. The calculation formula is as follows: ； Using server-side testing of personalized models The generalization performance is evaluated, metrics are calculated, and models with performance below a threshold are eliminated. Optimized personalized model parameters With the updated cluster centers The package is encrypted and then distributed to the corresponding group's clients.

8. The method for detecting traffic violations using personalized federated learning according to claim 7, characterized in that, The server distributes the updated cluster centers, clusterer parameters, and personalized models to the corresponding clients, continuously optimizing the feature space distribution and model parameters for each client, including: The server will update the personalized model parameters. Cluster Center and clusterer parameters Packaged by group, and with version identifiers and metadata attached, sent to the corresponding clients; After receiving the encrypted data packet, the client verifies the validity of the digital signature, decrypts it, and obtains the personalized model parameters, cluster centers, and related configuration information. Personalized model parameters sent by the server Align with the local teacher model architecture and replace the local teacher model. Parameters, retain the task decision layer Local optimization parameters; Based on the new cluster centers By fine-tuning the local teacher model The weights are calculated by minimizing the distance between the feature representation and the cluster center, using the following formula: ； Combined with loss assessment Distillation loss and prototype loss Jointly optimize the local model.

9. The method for detecting traffic violations using personalized federated learning according to claim 8, characterized in that, The combined discriminant loss Distillation loss and prototype loss Joint optimization of the local model includes: Based on local dual prototype vectors Prototype Center Aggregated with Server Calculate the prototype alignment loss; The three types of losses are combined into a total loss according to their weights. The calculation formula is as follows: ； The total loss is calculated using automatic differentiation. For local model parameters The gradient is calculated, and the optimizer updates the parameters while freezing the globally shared parameters. To maintain federal consistency.

Images