A road defect detection method and system based on hierarchical federated learning

Abstract

The application discloses a kind of road defect detection method and system based on layered federal learning, it is related to road defect detection technical field.Client receives global model parameter and the regional model parameter of the region to which it belongs issued by server;The road image of target road is input into global model, and global feature representation is obtained, and global prediction result is generated based on global feature representation;Global feature representation is input into regional model, and regional feature is obtained, and regional prediction result is generated based on regional feature;Regional feature is input into local model, and local feature is obtained by mapping regional feature to local model, and local prediction result is generated according to local feature;Client weights global prediction result, regional prediction result and local prediction result, and obtains final prediction result, and determines road defect detection result according to final prediction result.The method improves the accuracy of road defect detection.

Description

Technical Field

[0001] This application relates to the field of road defect detection technology, and in particular to a road defect detection method and system based on hierarchical federated learning. Background Technology

[0002] With the continuous expansion of urban road scale and the increasing demand for intelligent inspection, image-based road defect detection technology has gradually become one of the important technical means for road operation and maintenance management.

[0003] In existing technologies, road surface images are typically acquired using road inspection vehicles, fixed cameras, or mobile terminals, and deep learning models are used to automatically identify and segment defects such as cracks and potholes. These methods generally rely on a centralized training framework, where various types of road image data are collected and stored on a central server. Convolutional neural networks or semantic segmentation models are then trained to obtain predictive models for road defect detection. While centralized methods can achieve good detection results when the data source is singular and the scenario is relatively consistent, their dependence on data centralization presents significant limitations for large-scale deployments.

[0004] To mitigate the privacy and compliance risks associated with centralized datasets, existing technologies have introduced federated learning frameworks into road detection or traffic image analysis tasks. This enables multi-party collaboration by training the model locally and uploading only parameter updates. However, many existing federated learning methods employ simple global parameter averaging strategies, treating all clients equally. These methods typically construct only a single-level shared model structure, lacking the ability to effectively model common features among clients. Particularly when facing highly heterogeneous data distributions, they are prone to training oscillations, slow convergence, and even performance degradation, resulting in low accuracy in road defect detection. Summary of the Invention

[0005] Therefore, it is necessary to provide a road defect detection method and system based on hierarchical federated learning to address the aforementioned technical problems.

[0006] The following technical solution is adopted in this specification: This specification provides a road defect detection method based on hierarchical federated learning. This method is applied to a road defect detection system based on hierarchical federated learning, which includes multiple clients and a server. The method includes: For any client, the client receives global model parameters and regional model parameters of its own region from the server; the global model parameters are shared by all clients and are used to learn the common structure that is invariant across regions; the regional model parameters are shared by clients in the same region and are used to compensate for consistency deviations within the region without destroying the global expressive power. The client inputs the road image of the target road into the global model corresponding to the global model parameters. The global model performs cross-regional shared feature extraction on the road image to obtain a global feature representation. Based on the global feature representation, a global prediction result is generated. The client inputs the global feature representation into the regional model corresponding to the regional model parameters, performs regional-specific feature modulation on the global feature representation through the regional model to obtain regional features, and generates regional prediction results based on the regional features. The client inputs regional features into the local model, maps the regional features through the local model to obtain local features, and generates local prediction results based on the local features. The local model is unique to the client and is used to enhance the client's specific adaptability to its own data distribution. The client weights the global prediction results, regional prediction results, and local prediction results to obtain the final prediction result, and determines the road defect detection result based on the final prediction result.

[0007] Optionally, the process of dividing the region includes: The client extracts feature vectors representing the statistical characteristics of the local road images and uploads these feature vectors to the server. Based on the similarity between feature vectors, the server performs clustering processing on feature vectors from different clients, dividing all clients into multiple region clusters, with each cluster corresponding to a region; The server constructs a region distance matrix based on the feature mean vectors of each region; the elements of the region distance matrix represent the Euclidean distance between the feature mean vectors of two corresponding regions. The server sends the corresponding region identifier to each client.

[0008] Optionally, the process of determining the global model parameters, regional model parameters, and local model parameters includes: For any client, the server will send the initialized global model parameters, the regional model parameters of the client's region, the client's local model parameters, and the fusion weights of each model to the client. The client uses the global model, regional model, local model, and the fusion weights of each model to update the parameters of the global model, regional model, and local model. The client updates and saves the updated local model parameters as parameters of the local model, and uploads the updated global model parameters and regional model parameters as aggregate parameters to the server; The server performs global aggregation based on the global model parameters sent by all clients to obtain aggregated global model parameters. At the regional level, it performs regional aggregation on the regional model parameters sent by clients within the same region, based on the regional distance matrix, to obtain aggregated regional model parameters. Based on the model training status and performance feedback, the server adjusts the fusion weights of each model and redistributes the updated global model parameters, regional model parameters, and fusion weights to the client, entering the next round of federated training until the preset termination condition is met, thus obtaining the road defect detection model. The parameters of the road defect detection model include the global model parameters, regional model parameters, local model parameters, and fusion weights corresponding to the termination condition.

[0009] Optionally, the global model parameters, regional model parameters, and local model parameters are updated using the global model, regional model, local model, and the fusion weights of each model, including: For each training sample, the global model, regional model, local model, and the fusion weights of each model are used to determine the global prediction result, regional prediction result, local prediction result, and final prediction result of the sample road image. The main loss for the segmentation task is determined based on the global prediction results, regional prediction results, local prediction results, and final prediction results of the sample. Obtain a snapshot of the client's regional teacher model; the snapshot is generated by the regional model parameters of the client's region in the previous round or a historical round. Based on the sample region features extracted from the client's regional model and the sample region features extracted from the snapshot of the regional teacher model, as well as the sample region features extracted from the snapshot of the regional teacher model in other regions, the regional distillation constraints are determined, and the total loss is determined based on the main loss of the segmentation task and the regional distillation constraints. Backpropagation is performed on the global model parameters, regional model parameters, and local model parameters using the total loss; Once the local training reaches the preset number of iterations, the global model parameters, regional model parameters, and local model parameters corresponding to the preset number of iterations will be determined as the final global model parameters, regional model parameters, and local model parameters for this round.

[0010] Optionally, based on the sample region features extracted from the client's regional model, the sample region features extracted from the regional teacher model snapshot, and the sample region features extracted from the regional teacher model snapshots of other regions, regional distillation constraints are determined, including: Based on the sample region features extracted from the client's regional model and the sample region features extracted from the snapshot of the regional teacher model, the distillation loss within the region is determined; Based on the regional distance matrix and the sample regional features extracted from the regional model, as well as the sample regional features extracted from the regional teacher model snapshots of other regions, cross-regional distillation terms are determined. Regional distillation constraints are determined based on intra-regional distillation losses and inter-regional distillation terms.

[0011] Optionally, the distillation loss function within the region is: ; in, Distillation loss within the region, Sample region features extracted from the region model of the client's region. The sample region features are extracted from the snapshot of the regional teacher model for the client's region. Indicates Euclidean distance; The cross-regional distillation term is: ; ; in, For cross-regional distillation, For the region With the region Distillation weights between For the region The sample region features extracted from the snapshot of the regional-level teacher model. For regions in the region distance matrix With the region The Euclidean distance between them This is the attenuation coefficient due to distillation distance; The zone distillation constraint is: ; in, For regional distillation constraints, This is the inter-regional distillation adjustment coefficient; The total loss function is: ; in, For the total loss, To divide the main loss of the task, This is the distillation loss weighting coefficient.

[0012] Optionally, the global aggregation formula for the global model parameters is: ; in, These are the aggregated global model parameters. For the total number of clients, For the first The number of training samples per client. The total number of training samples across all clients. For the first Global model parameters for each client; The formula for calculating region aggregation is: ; ; ; in, These are the parameters of the aggregated region model. To achieve cross-regional sharing ratios, For the region With the region Similarity weights between them For the region Regional adaptation parameters, For the region The number of Chinese clients For the first The number of training samples per client. For the region The total number of training samples across all clients. For the first Regional model parameters for each client, For the region With the region The Euclidean distance between them This is the polymerization distance attenuation coefficient.

[0013] Optionally, the model training state includes the current training round, and the performance feedback includes the global average intersection-over-union ratio (IoU) of the defect detection task corresponding to the previous round of the model; based on the model training state and performance feedback, the fusion weights of each model are adjusted, including: Obtain the regional distribution difference index; the regional distribution difference index is derived from the regional distance matrix. The global average intersection-over-union ratio, regional distribution difference index, and the current round are input into a pre-trained reinforcement learning algorithm to obtain the action set of the reinforcement learning algorithm. Action set for: ; in, For the fusion weights of the global model, For the fusion weights of the regional model, For the fusion weights of the local model; .

[0014] Optionally, the reinforcement learning algorithm employs the Q-learning algorithm, whose reward function is: ; ; in, For the first The reward function value for each round, Indicates performance gain. This is the regional difference penalty coefficient. This is an indicator of regional distribution differences. For the first The global average intersection-union ratio of the rounds, For the first The global average intersection-union ratio of the rounds.

[0015] This specification provides a road defect detection system based on hierarchical federated learning, which includes multiple clients and one server; For any client, the client receives global model parameters and regional model parameters of its own region from the server; the global model parameters are shared by all clients and are used to learn the common structure that is invariant across regions; the regional model parameters are shared by clients in the same region and are used to compensate for consistency deviations within the region without destroying the global expressive power. The client inputs the road image of the target road into the global model corresponding to the global model parameters. The global model performs cross-regional shared feature extraction on the road image to obtain a global feature representation. Based on the global feature representation, a global prediction result is generated. The client inputs the global feature representation into the regional model corresponding to the regional model parameters, performs regional-specific feature modulation on the global feature representation through the regional model to obtain regional features, and generates regional prediction results based on the regional features. The client inputs regional features into the local model, maps the regional features through the local model to obtain local features, and generates local prediction results based on the local features. The local model is unique to the client and is used to enhance the client's specific adaptability to its own data distribution. The client weights the global prediction results, regional prediction results, and local prediction results to obtain the final prediction result, and determines the road defect detection result based on the final prediction result.

[0016] This specification provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described road defect detection method based on hierarchical federated learning.

[0017] This specification provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-described road defect detection method based on hierarchical federated learning.

[0018] The above-mentioned technical solutions adopted in this specification can achieve the following beneficial effects: In the road defect detection method based on hierarchical federated learning provided in this specification, a global feature representation shared across regions is extracted from road images. This design captures common visual patterns of road defects in different regions, avoiding the loss of common information due to averaging of a single global parameter in traditional federated learning. Clients in the same region share regional model parameters. The regional model can adaptively compensate for regional consistency deviations such as specific geographical environments, climate conditions, or road surface materials while maintaining global expressive power, effectively reducing training oscillations caused by heterogeneous data distribution and accelerating convergence. Each client has unique local model parameters. The local model further maps regional features to generate local features and prediction results. This design can be personalized to the data characteristics of a single client, significantly enhancing the model's specific adaptability to local data distribution and reducing performance degradation caused by differences between clients. Finally, the prediction results of the global model, regional model, and local model are weighted and fused to generate the final prediction result. The multi-level decision-making mechanism provided by this invention takes into account common features, regional characteristics and local details, and can balance generalization and specificity in complex road scenarios. It effectively suppresses false detections or false negatives that may be caused by single-level prediction, and significantly improves the overall accuracy of road defect detection compared with traditional centralized or single-layer federated learning methods. Attached Figure Description

[0019] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0020] Figure 1 This document provides a schematic diagram of a road defect detection method based on hierarchical federated learning. Figure 2 This specification provides a schematic diagram of the overall architecture of a road defect detection system based on hierarchical federated learning. Figure 3 This is a schematic diagram of a three-layer hierarchical model structure provided in this specification; Figure 4 This document provides a schematic diagram of the process for regional division and regional distance matrix construction under the condition that data cannot be centralized. Figure 5This specification provides a local training process for a single client in a round of federated training; Figure 6 This document provides a schematic diagram of a server aggregation process. Figure 7 This document provides a Q-learning dynamic parameter optimization process; Figure 8 This specification provides a flowchart of the specific implementation of a zone distillation mechanism; Figure 9 This is a schematic diagram of a three-layer fusion provided in this specification. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments in this specification without creative effort are within the scope of protection of this application.

[0022] In existing technologies, the scale of data collected from different regions is often uneven, with regions with larger data volumes dominating centralized training, further exacerbating the performance degradation of the model in regions with fewer samples. Therefore, existing centralized road defect detection technologies struggle to simultaneously ensure detection accuracy and model stability in different regions when deployed across multiple areas.

[0023] Furthermore, existing federated learning methods typically use fixed fusion weights, which are difficult to dynamically adjust based on training stages and regional differences, thus limiting further improvements in model performance.

[0024] Furthermore, road imagery data often contains potentially sensitive information such as license plates, faces, and building layouts. Due to data security and privacy protection requirements, raw imagery data is typically not allowed to be centrally stored or shared across regions. This makes it difficult to implement traditional improvement methods that rely on centralized data training or direct data exchange between regions. Against this backdrop, how to achieve collaborative training across geographical regions without uploading raw imagery data, effectively characterize regional differences, stabilize the federated training process, and balance global sharing with terminal personalization, has become a pressing technical problem that existing road defect detection technologies need to solve. Therefore, it is necessary to propose a new road defect detection method and system to overcome the shortcomings of existing technologies in cross-regional collaborative learning, privacy protection, and model stability.

[0025] This invention aims to address the core technical bottlenecks faced by road defect detection deployments across multiple regions, including significant distributional differences, lack of data centralization, and the difficulty in balancing generalization and personalization in model implementation. In real-world scenarios, road imagery is collected across diverse geographical areas, each exhibiting significant differences in road surface materials, lighting, climate, camera equipment, and damage patterns. The term "region" in this invention primarily refers to road management areas or urban districts defined geographically, rather than solely by road surface material or scene category. Relying solely on local training limits the model's ability to learn features from a single region, hindering its ability to express commonalities across regions. This results in insufficient generalization capabilities in new regions or areas with low data volumes, particularly for regions with limited sample sizes or relatively homogeneous defect types, where independent local training often fails to yield stable and representative defect representations. While traditional centralized training can aggregate multi-source information, it is limited by privacy concerns, data security, and the inability to centralize data across institutions, making it difficult to implement in road monitoring systems. Therefore, to achieve cross-regional knowledge sharing without compromising data compliance, a joint training mechanism is necessary, enabling each region to collaboratively optimize the model while maintaining local data integrity. Joint training can fuse feature representations learned from various geographic regions at the parameter level without exchanging original images, thereby improving detection performance in low-data-volume areas and newly accessed areas. However, most existing federated learning frameworks only provide a globally shared structure, which cannot effectively characterize style differences between geographic regions; fixed aggregation weight mechanisms cannot adapt to the dynamic changes in the road environment, resulting in the model being unable to adjust the contribution ratio between the global, regional, and local models based on regional performance feedback in different training rounds; at the same time, local personalized models often lack structural knowledge supplementation from the regional level, making it difficult to balance local specialization capabilities and cross-regional consistency. Based on this, this invention provides a road defect detection method and system based on hierarchical federated learning. This method constructs a novel hierarchical collaborative learning system that can simultaneously support global sharing, regional difference learning, and local personalized modeling. It improves cross-regional generalization ability through a joint training mechanism, achieves distributed optimization with data remaining within its domain through a privacy-friendly structure, and overcomes the shortcomings of existing technologies in multi-regional road defect detection through mechanisms such as dynamic weight adjustment and regional layer knowledge injection. It solves problems such as large differences in image distribution across multiple geographic regions, inability to centralize data, and the difficulty of balancing global sharing, regional adaptation, and terminal personalization with fixed structures and fixed aggregation weights. This enables a safe, robust, and widely applicable intelligent identification capability for road defects across regions.

[0026] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0027] Figure 1This is a flowchart illustrating a road defect detection method based on hierarchical federated learning, as described in this specification. The method is applied to a road defect detection system based on hierarchical federated learning, which includes multiple clients and one server. The method specifically includes the following steps: S101: For any client, the client receives the global model parameters and the regional model parameters of its own region from the server. The global model parameters are shared by all clients and are used to learn the common structure that is invariant across regions. The regional model parameters are shared by clients in the same region and are used to compensate for consistency deviations within the region without destroying the global expressive power.

[0028] This embodiment constructs a three-layer hierarchical model structure consisting of a global layer, a regional layer, and a local layer. The three layers are functionally divided according to the principle of "decreasing sharing and increasing specialization" to ensure that cross-regional tasks can achieve a hierarchical expression that balances generalization ability and regional adaptability under privacy constraints. Within the three-layer structure, it is specified that… These represent the model parameters for the global layer, regional layer, and local layer, respectively. The feature representations of the corresponding layer outputs; These represent the predicted outputs obtained from feature mapping; This represents the fusion weights of the three-layer prediction output. The values ​​are dynamically determined by the Q-learning control module. The fusion weights do not participate in the model parameter training and are only used as the output fusion coefficients in the prediction stage.

[0029] In the absence of prior region labels in multi-source road image data, this invention introduces an automatic region segmentation mechanism based on client feature distribution to achieve regional difference modeling while satisfying privacy constraints. In one embodiment, the region segmentation process includes: the client extracting feature vectors representing regional statistical characteristics from local road images and uploading these feature vectors to the server; the server performing clustering processing on feature vectors from different clients based on the similarity between them, dividing all clients into multiple region clusters, each cluster corresponding to one region; the server constructing a region distance matrix based on the feature mean vectors of each region; the elements of the region distance matrix representing the Euclidean distance between the feature mean vectors of two corresponding regions; and the server issuing its corresponding region identifier to each client.

[0030] Specifically, without transmitting the original road image data, the client extracts feature vectors representing the statistical characteristics of the area from the local road image and uploads the feature vectors to the server.

[0031] Specifically, through multi-layer convolution and downsampling operations, low-level features such as edges and textures in the original image are gradually extracted and abstracted into semantic features with discriminative capabilities, and represented as feature tensors, i.e. feature vectors.

[0032] The server performs statistical aggregation on feature vectors from different clients and performs clustering based on the distributional similarity between feature vectors. K-means clustering divides the clients into multiple region clusters, each corresponding to a geographically or statistically significant region. After clustering, the server sends its corresponding region identifier to each client for subsequent training and optimization of the region model.

[0033] To characterize the distribution differences between different regions, this invention further constructs a Region Distance Matrix (RDM) to describe the difficulty of alignment between regions and the potential benefits of knowledge sharing. Let the regions... The feature mean vector is ,area The feature mean vector is The distribution difference between any two regions can be measured by Euclidean distance:

[0034] (1); in, For regions in the region distance matrix With the region The Euclidean distance between them This represents the Euclidean distance. In RDM, a larger element value indicates a more significant difference in data style between the two regions, and direct parameter sharing between regional models may lead to performance degradation; conversely, regions with smaller distances are more suitable for sharing structured knowledge or performing regional distillation.

[0035] This invention utilizes this matrix to guide the weight adjustment between regions during federated aggregation and distillation processes, enabling the system to dynamically determine the knowledge transfer path and aggregation intensity based on regional differences. Through the above process, this method achieves automatic geographical region division and difference modeling without requiring the release of original imagery, forming the regional infrastructure of a three-layer collaborative learning system.

[0036] It should be noted that each client corresponds to a road data acquisition node within one or more geographical areas. The client can be a vehicle-mounted terminal, a roadside device, or a mobile data acquisition device. The client acquires and stores road images locally within its designated area and performs defect detection tasks based on the acquired road images.

[0037] Data collected by different clients comes from different geographical regions, resulting in differences in data distribution among clients. A single client may contain multiple clients, and a single client may cover multiple roads. However, in any given training round, each client only uses its locally collected road images for model training and updates.

[0038] S102, the client inputs the road image of the target road into the global model corresponding to the global model parameters, performs cross-regional shared feature extraction on the road image through the global model to obtain a global feature representation, and generates a global prediction result based on the global feature representation.

[0039] At the global layer, a unified basic encoding network is used as a common representation space for feature abstraction of road images from different geographic regions. The global model typically includes multi-layer convolutional modules, downsampling modules, and feature bottleneck modules. Specifically: multi-layer convolutional modules are used to extract local spatial features from the input road images; downsampling modules are used to progressively reduce the feature map resolution and expand the receptive field; and feature bottleneck modules are used to compress and reconstruct high-dimensional features to enhance feature expressiveness. The aim is to extract high-level semantic information such as edge texture, contour gradients, and crack structures, which have broad applicability. The global model parameters are shared across all clients, and their updates are driven by model gradients contributed by all regions. Therefore, this layer is responsible for learning cross-regional invariant common structures, forming the expressive basis of the entire layered model. Assume the client... k The input road image is The global model consists of a global model parameter set. This indicates that it only participates in forward and backward computations locally on the client side and is not stored locally for long periods. The global model abstracts features from the input road image using the forward method to obtain a global feature representation:

[0040] (2); in, For the first k Global feature representations extracted from the global model of each client. The feature extraction function of the global model consists of a multi-layer convolution module, a downsampling module, and a feature bottleneck module, used to map the input road image into a high-level semantic feature representation. Equation (2) represents the global abstract features captured by the global model in the semantic space. This global feature representation reflects a consistent high-level semantic structure in road images of different regions and can further generate corresponding global prediction results:

[0041] (3); in, For the first k The global prediction results output by the global model of each client. This is the prediction function of the global model, used to map the global feature representation to the corresponding defect segmentation prediction result. The prediction function consists of a decoder network or prediction head module, including upsampling layers and convolutional layers.

[0042] S103, the client inputs the global feature representation into the regional model corresponding to the regional model parameters, performs regional-specific feature modulation on the global feature representation through the regional model to obtain regional features, and generates regional prediction results based on the regional features.

[0043] At the regional layer, this invention introduces a regional adaptation structure designed for geographical regional differences to capture consistent features such as texture, lighting, surface material, and damage patterns within the region. The regional layer provides global features. Based on this, region features are specialized using methods such as lightweight modulated convolution, feature bias adjustment, or region embedding to obtain region features:

[0044] (4); in, For the first k Regional features extracted from the regional model of each client. For the first k The region to which each client belongs Regional model parameters, This is a region feature extraction function for a region model, used to introduce region-specific feature modulation on the basis of global feature representation. It consists of a region adaptation module, including modulation convolution, feature bias adjustment or region embedding mechanism, used to extract feature representation with regional characteristics.

[0045] Regional features are used to characterize the statistical consistency of road images within the same region in terms of pavement material, lighting conditions, and damage patterns, and further generate regional prediction output: (5); in, For the first k The regional prediction results output by the regional model of each client. This is the region prediction function for the region model, used to map the region feature representation to the corresponding region defect prediction result. This function consists of a region decoder or a prediction head module.

[0046] Parameters of the region adaptation layer The parameters are shared among clients within the same region, while different regions have independent sets. This structure allows the model to compensate for consistency biases within regions without compromising global expressiveness, making the objective function more stable during cross-region training. In the three-layer structure of this invention, the region layer plays the role of "regional intermediate state modeling," forming a structured knowledge transition layer at the feature representation level between global sharing and local personalization.

[0047] S104, the client inputs regional features into the local model, maps the regional features through the local model to obtain local features, and generates local prediction results based on the local features; the local model is unique to the client and is used to enhance the client's specific adaptability to its own data distribution.

[0048] At the local layer, this invention configures an independent personalized module for each client to capture individualized factors such as differences in road inspection equipment, installation angles, specific road segment characteristics, and individual environmental noise. The local layer typically employs a shallow network with a small number of parameters (local model), such as lightweight convolutional blocks, linear projection layers, or channel attention modules. Its parameters are updated only locally at the terminal and do not participate in regional or global aggregation. Local characteristics include:

[0049] (6); in, For the first k Local features extracted from the local model of each client. For the first k Local model parameters for each client. This is a local feature extraction function for the local model, used to further extract personalized features for the client based on the regional feature representation. It consists of lightweight network modules, including shallow convolutional structures, linear mapping layers, or channel attention modules.

[0050] Further personalized local prediction results for the client were obtained: (7); in, For the first k The local prediction results output by the local model of each client. This is the local prediction function for the local model, used to map local feature representations to corresponding personalized defect prediction results. This function consists of a local decoder or prediction head module.

[0051] The main enhancement is the model's ability to adapt to specific data distributions on the client side, thereby avoiding performance degradation caused by excessive sharing. The local layer and the regional layer together constitute a differentiated representation structure, giving the model the dual ability of local optima and cross-regional generalization.

[0052] S105, the client weights the global prediction results, regional prediction results and local prediction results to obtain the final prediction result, and determines the road defect detection result based on the final prediction result.

[0053] To effectively fuse the feature representations generated by the three-layer structure, this invention constructs the final prediction output based on a dynamic fusion mechanism. Specifically, the model generates global prediction results separately. Regional prediction results and local prediction results The final prediction result is obtained through weighted fusion. :

[0054] (8); in, , and The fusion weights are the fusion weights for the global model, the regional model, and the local model, respectively, and the fusion weights satisfy the following conditions: Furthermore, it can adaptively adjust in conjunction with subsequent dynamic learning strategies. This fusion approach can dynamically balance cross-regional consistency and local specialization capabilities during training, enabling the model to achieve more stable optimization performance in heterogeneous environments. Through the above three-layer hierarchical structure, this invention provides a model system that simultaneously possesses sharing, regionality, and individualization capabilities, laying a structural foundation for realizing cross-regional road defect detection.

[0055] In one embodiment, the process of determining the global model parameters, regional model parameters, and local model parameters includes the following steps: S201: For any client, the server will send the initialized global model parameters, the regional model parameters of the client's region, the local model parameters corresponding to the client, and the fusion weights of each model to the client.

[0056] S202, the client uses the global model, regional model, local model, and the fusion weights of each model to update the parameters of the global model, regional model, and local model.

[0057] In one embodiment, updating the parameters of the global model, regional model, and local model is accomplished using the global model, regional model, local model, and the fusion weights of each model, including the following steps: S301, on each training sample, using the global model, regional model, local model, and the fusion weights of each model, determine the global prediction result, regional prediction result, local prediction result, and final prediction result of the sample road image.

[0058] S302. Based on the global prediction results of the samples, the regional prediction results of the samples, the local prediction results of the samples, and the final prediction results of the samples, determine the main loss of the segmentation task.

[0059] S303, Obtain a snapshot of the client's regional teacher model; the snapshot of the regional teacher model is generated by the regional model parameters of the client's region in the previous round or a historical round.

[0060] S304. Based on the sample region features extracted by the client's regional model, the sample region features extracted by the regional teacher model snapshot, and the sample region features extracted by the regional teacher model snapshot of other regions, determine the regional distillation constraints, and determine the total loss based on the main loss of the segmentation task and the regional distillation constraints. S305 performs backpropagation on the global model parameters, regional model parameters, and local model parameters using the total loss.

[0061] S306, when the local training reaches the preset number of times in this round, the global model parameters, regional model parameters and local model parameters corresponding to the preset number of times are determined as the final global model parameters, regional model parameters and local model parameters for this round.

[0062] To enhance the stability of the regional layer model during multi-round joint training and to achieve continuous transfer of feature patterns within the region without sharing the original road image data, this embodiment introduces a regional distillation mechanism. This mechanism imposes constraints on the regional model training process by constructing a regional-level teacher model snapshot on the server side, thereby mitigating the impact of data distribution fluctuations within the region on model stability.

[0063] In constructing the regional-level teacher model, the server maintains snapshots of historical model parameters for each region. These snapshots are generated from the parameters of the regional model in the previous or previous rounds, and their parameters remain fixed during subsequent training, not participating in gradient updates. The regional-level teacher model snapshots serve as historical semantic references for the regional layer models, constraining the update direction of the current regional model. Regarding regional feature distillation, this embodiment does not distill the final predicted probability. Instead, it uses "regional feature space alignment" as its core idea, constraining the consistency between the representation of the current regional model in the bottleneck feature space or the adaptation layer output space and the corresponding representation of the teacher snapshot model.

[0064] In one embodiment, determining regional distillation constraints based on sample region features extracted from the client's regional model, sample region features extracted from the regional teacher model snapshot, and sample region features extracted from the regional teacher model snapshots of other regions includes: determining intra-regional distillation loss based on the sample region features extracted from the client's regional model and sample region features extracted from the regional teacher model snapshots; determining cross-regional distillation terms based on the regional distance matrix, sample region features extracted from the regional model, and sample region features extracted from the regional teacher model snapshots of other regions; and determining regional distillation constraints based on intra-regional distillation loss and cross-regional distillation terms.

[0065] Optionally, the distillation loss function within the region is: (9); in, Distillation loss within the region, Sample region features extracted from the region model of the client's region. The sample region features are extracted from the snapshot of the regional teacher model for the client's region. The distillation loss within the aforementioned region enables the regional model to quickly adapt to local data while preserving the structural characteristics within the region, thereby avoiding parameter oscillations caused by differences in local distribution among different clients. This invention controls the distillation adjustment coefficient between regions, enabling the regional model to balance stability and adaptability during the optimization process.

[0066] Regarding inter-regional knowledge transfer, this invention utilizes a regional distance matrix constructed by the server to adjust the distillation intensity between different regions, aiming to support complementary knowledge sharing between nearby regions while avoiding negative transfer caused by excessive cross-regional differences. For regions... With the region If the Euclidean distance between the two If the value is small, the server can incorporate some feature references from neighboring regions into the regional teacher model snapshot, achieving flexible alignment of feature spaces in nearby regions through low-weight cross-regional distillation terms. The cross-regional distillation term is:

[0067] (10); (11); in, For cross-regional distillation, For the region With the region Distillation weights between For the region The sample region features extracted from the snapshot of the regional-level teacher model. For regions in the region distance matrix With the region The Euclidean distance between them This is the distillation distance attenuation coefficient, used to control the degree of influence of inter-regional distance on the weight of cross-regional distillation. A larger value indicates higher regional similarity, making it suitable for weak sharing or feature-guided approaches.

[0068] This invention combines intra-regional distillation loss with inter-regional migration loss to form a total regional distillation constraint, which is as follows: (12); in, For regional distillation constraints, This is the inter-regional distillation adjustment coefficient. This structure enables regional models to absorb supplementary knowledge from similar regions while maintaining the stability of their own characteristics, thereby achieving more stable performance in sparsely distributed or insufficiently sampled regions.

[0069] Through the above mechanism, the regional distillation process in this embodiment achieves the preservation of historical knowledge within a region, controllable knowledge transfer between regions, and enhanced model consistency in multiple iterations, significantly improving the robustness, generalization ability, and training stability of the regional layer model under multi-regional heterogeneous data conditions.

[0070] The total loss function is: (13); in, For the total loss, To divide the main loss of the task, This is the distillation loss weighting coefficient, used to adjust the contribution of the regional distillation constraint to the total loss function. Optionally, the main loss for the segmentation task can be cross-entropy or Dice loss.

[0071] The main loss function for the segmentation task is: (14); in, Indicates the global prediction result of the sample. Compared with the actual defect labels of the samples Cross-entropy between Indicates the prediction results for the sample region. Compared with the actual defect labels of the samples Cross-entropy between Indicates the local prediction results of the sample. Compared with the actual defect labels of the samples Cross-entropy between Indicates the final prediction result of the sample Compared with the actual defect labels of the samples Cross-entropy between them.

[0072] After the current round of local training iterations reaches the preset number of epochs, the client extracts a subset of parameters from the updated model that can participate in federated aggregation, including global model parameters. Regional model parameters and local sample size Information such as local model parameters. It is kept locally and is not uploaded.

[0073] S203, the client updates and saves the updated local model parameters as parameters of the local model, and uploads the updated global model parameters and regional model parameters as aggregate parameters to the server.

[0074] The client sends the packaged update results back to the server via the communication module.

[0075] S204, the server performs global aggregation based on the global model parameters sent by all clients to obtain aggregated global model parameters, and at the regional level, it performs regional aggregation on the regional model parameters sent by clients within the same region based on the regional distance matrix to obtain aggregated regional model parameters.

[0076] To improve the stability of federated model aggregation under the condition of highly heterogeneous road images in multiple regions, this invention introduces a regional difference perception mechanism on the basis of the traditional FedAvg aggregation framework. By performing hierarchical aggregation of global layer parameters and regional layer parameters, and combining the regional distance matrix, weak sharing control between regions is achieved.

[0077] In one embodiment, regarding global aggregation, the present invention extends the traditional FedAvg formula, limiting the aggregation target to the shared parameter set of the global model. Let the number of all clients be denoted as _____. The global model parameters uploaded by each client are Its local training sample size is The total number of all participating client samples is Then the global aggregation formula for the global model parameters is:

[0078] (15); in, These are the aggregated global model parameters. For the total number of clients, For the first The number of training samples per client. The total number of training samples across all clients. For the first The global model parameters for each client. This aggregation process only applies to global layer parameters; region layer and local layer parameters are not involved in this step.

[0079] In one embodiment, regarding region aggregation, this invention groups clients according to their geographical regions, constructing region buckets. Furthermore, to achieve cross-regional weight control that is aware of regional differences, this invention introduces a region-level dynamic weighting mechanism in conjunction with a regional distance matrix; that is, this invention utilizes an exponential decay function to construct regional similarity weights. During region aggregation, regions... It not only relies on parameters within the region, but can also reference parameters from similar regions in weak sharing mode.

[0080] For the region Let the set of clients it contains be... The client uploads the region adaptation layer parameters as follows: Then the formula for calculating region aggregation is: (16); (17); (18); in, These are the parameters of the aggregated region model. The cross-regional sharing ratio is used to control the extent to which cross-regional information is introduced into the regional model during the aggregation process; For the region With the region Similarity weights between them For the region Regional adaptation parameters, For the region The number of Chinese clients For the first The number of training samples per client. For the region The total number of training samples across all clients. , For the first Regional model parameters for each client, For the region With the region The Euclidean distance between them This is the aggregation distance decay coefficient, used to control the influence of inter-region distance on the cross-region weak sharing weight. When the difference between two regions is small, the similarity weight is higher; as the distance increases, the regional similarity weight tends to zero, thus automatically avoiding regional performance degradation caused by excessive sharing. Indicates the region For the region The similarity weights are used. This step can enhance the generalization ability in regions with scarce data and reduce training instability.

[0081] This aggregation method enables the region layer to learn the specific style and damage characteristics of the region, while avoiding the negative transfer risk caused by cross-region parameter mixing. This invention, through independent aggregation of region adaptation layers, allows the model to characterize regional differences in a structured manner.

[0082] After completing the global aggregation, intra-regional aggregation, and weak inter-regional sharing updates, the server will use the updated model parameters as input for the next round of federated training to achieve stable collaborative optimization under the condition of regional difference awareness.

[0083] S205, the server adjusts the fusion weights of each model based on the model training status and performance feedback, and redistributes the updated global model parameters, regional model parameters, and fusion weights to the client to enter the next round of federated training loop until the preset termination condition is met, thus obtaining the road defect detection model; the parameters of the road defect detection model include the global model parameters, regional model parameters, local model parameters, and fusion weights corresponding to the termination condition.

[0084] In one embodiment, to achieve adaptive adjustment of the prediction fusion weights of the global layer, regional layer and local layer, the present invention introduces a dynamic parameter optimization mechanism based on Q-learning on the server side, and models the combination selection process of the three-layer fusion weights as a reinforcement learning decision problem, thereby learning the optimal fusion strategy in multiple rounds of federated training.

[0085] In terms of state-space design, this invention constructs state representations using statistical indicators that reflect the overall performance of the model and regional heterogeneity. Let the previous round's global average intersection-union ratio (Mean IoU, mIoU) be... The regional distribution difference index derived from the regional distance matrix is: The current round is Then the state can be represented as: The state can characterize the training progress, the degree of regional heterogeneity, and the overall performance of the model, providing a basis for action selection.

[0086] Regarding the definition of the action space, this invention uses a discrete combination of fused weights as the action set for Q-learning. It consists of three items:

[0087] (19); in, For the fusion weights of the global model, For the fusion weights of the regional model, For the fusion weights of the local model; Each fusion weight takes a preset discrete value. Before the start of each training round, the server selects the current action and sends the corresponding fusion weight to each client to guide the three-layer prediction fusion in this round.

[0088] Regarding reward construction, this invention constructs a reward signal based on the change in global mIoU and the change in regional difference indicators after the current training round. Performance gain. for:

[0089] (20); in, For the first The global average intersection-union ratio of the rounds, For the first The global average intersection-union ratio of the rounds.

[0090] Meanwhile, considering that excessive regional differences may lead to training instability, this invention incorporates a regional difference penalty coefficient into the reward item. The reward function is constructed as follows: (twenty one); in, For the first The reward function value for each round, This is an indicator of regional distribution differences; This is the regional difference penalty coefficient, used to adjust the influence of regional distribution difference indicators on the reward function, and to suppress the instability caused by excessive regional differences. The larger the value, the stronger the penalty for regional differences, thus guiding the model to pay more attention to the consistency of distribution across regions during the optimization process. The smaller the value, the more the model focuses on improving overall detection performance. As model performance improves, the reward value tends to increase; however, when regional differences cause training oscillations, the reward decreases accordingly.

[0091] Regarding Q-value updates, this invention employs a tabular Q-learning method, which backtracks and updates the Q-value of the current action using the maximum Q-value between the current reward and the next round's state. Let the learning rate be... The discount factor is Then the Q-value update formula is:

[0092] (twenty two); in, Indicates the state Take action below The current Q value is used to evaluate the action. In this state The long-term benefits; Indicates the first The state corresponding to each training round is composed of model performance metrics and regional distribution difference metrics. Indicates the first The action selected in the round, i.e., the fusion weight combination of the three-layer model, is defined as... ; Indicates the learning rate; Indicates the first The immediate reward function value corresponding to each round of training is used to reflect the performance improvement and regional differences brought about by the current action; This represents the discount factor, used to measure the importance of future rewards in current decision-making; Indicates the next state In, all possible actions The corresponding maximum Q value represents the optimal cumulative reward that can be obtained in the future.

[0093] Regarding the action selection strategy, this invention adopts... ε - The greedy strategy strikes a balance between exploration and exploitation. Through the Q-learning dynamic parameter optimization mechanism described above, the server can adaptively adjust the three-layer prediction fusion weights during multiple rounds of federated training, enabling the model to achieve more stable optimization results under different regional distributions and training stages.

[0094] Therefore, in one embodiment, the model training state includes the current training round, and the performance feedback includes the global average intersection-union ratio (IU / U) of the defect detection task corresponding to the previous round model; based on the model training state and performance feedback, the fusion weights of each model are adjusted, including: obtaining a regional distribution difference index; the regional distribution difference index is derived from the regional distance matrix; and inputting the global average IU / U, the regional distribution difference index, and the current round into a pre-trained reinforcement learning algorithm to obtain the action set of the reinforcement learning algorithm. That is, the values ​​of the three fusion weights.

[0095] The regional distribution difference index can be defined as the average of the pairwise distances between regions: (twenty three); in, For the first Regional distribution difference index corresponding to each round of training This represents the number of regions currently participating in the training. For the first Regions in the wheel's region distance matrix With the region The distance between them.

[0096] In one embodiment, the present invention provides an overall process for federated training and model deployment. This process is collaboratively completed by a server and multiple clients, adhering to the basic interaction paradigm of federated learning. Furthermore, it incorporates a three-layer hierarchical model structure, a regional difference-aware aggregation mechanism, and a dynamic fusion weight control strategy to achieve collaborative training of multi-regional road defect detection models. Specifically, it includes:

[0097] During the training initialization phase, the server initializes respectively. The server sets the parameters and initializes the Q-learning control module for dynamically scheduling prediction fusion weights. Then, the server distributes the global model parameters, local model parameters, and regional model parameters corresponding to the client's region to each client.

[0098] The client performs a local training process based on local road image data, using global, regional, and local layer models to complete forward computation and parameter updates. The update results of global and regional model parameters are uploaded to the server as aggregateable parameters, while local model parameters are only updated and saved locally on the client and are not shared across clients.

[0099] After receiving parameter updates uploaded by the client, the server performs global aggregation on the global model parameters. Simultaneously, at the regional level, it performs regional aggregation on the regional model parameters within the same region. Furthermore, it uses the regional distance matrix to perform difference-aware weak-sharing correction on the regional model parameters to suppress negative cross-regional transfer. During parameter aggregation, the server dynamically adjusts the fusion weights of global prediction, regional prediction, and local prediction in the final output based on model training status and performance feedback through the Q-learning control module.

[0100] After completing parameter aggregation and weight updates, the server will redistribute the updated global layer parameters, regional layer parameters, and fusion weight information to each client, and enter the next round of federated training until the preset termination conditions are met, resulting in a road defect detection model with cross-regional generalization and terminal personalization capabilities.

[0101] Compared with existing road defect detection technologies, this invention achieves significant technical results in cross-regional distributed training scenarios by introducing mechanisms such as multi-regional hierarchical modeling, regional distillation, and dynamic aggregation. Its beneficial effects mainly include the following aspects.

[0102] First, this invention effectively addresses the significant differences in texture style, lighting conditions, and damage patterns of road images across geographic regions through region partitioning and region-layer adaptation structures. This enables the model to construct structured differential representations across multiple regions, thereby significantly improving cross-regional generalization ability. Compared to traditional unified models, this invention maintains more stable semantic segmentation performance and exhibits greater adaptability under multi-regional data conditions.

[0103] A dynamic weight adjustment mechanism based on Q-learning replaces static fusion parameters, enabling the global layer, regional layer, and local layer to automatically adjust their contribution ratios based on performance feedback during training. This avoids the training oscillations and convergence instability problems commonly found in traditional federated learning in highly heterogeneous scenarios. This mechanism allows the model to emphasize shared characteristics, regional adaptability, or local personalization at different training stages, improving overall convergence efficiency.

[0104] The region distillation mechanism utilizes regional teacher snapshots to impose consistency constraints on the feature space of the region layer, effectively improving the stability of the region model in multi-round communication and enhancing the robustness of features within the region. Combined with weak cross-regional knowledge transfer achieved through the region distance matrix, regions with low data volume or significant environmental fluctuations gain additional knowledge support during training, further improving the quality of region layer modeling.

[0105] The three-layer hierarchical model structure can achieve cross-regional sharing, regional preference modeling, and terminal personalized optimization while maintaining data privacy. It can complete end-to-end training without centralizing the original road images on the server, which better meets the requirements for data security and compliance in actual urban road inspection scenarios.

[0106] Finally, the hierarchical aggregation and distributed training framework has good scalability, and can expand the number of clients and regions as needed in city-level, regional-level or cross-city-level road monitoring systems. It can adapt to multi-source devices such as vehicle terminals, fixed cameras, and edge nodes, and therefore has the feasibility of deployment in large-scale road scenarios.

[0107] When applying the road defect detection method based on hierarchical federated learning provided in this manual, it is not necessary to... Figure 1 The steps shown are executed in sequence. The specific execution order of each step can be determined as needed, and this manual does not impose any restrictions on it.

[0108] The above describes one or more embodiments of a road defect detection method based on hierarchical federated learning provided in this specification. Based on the same idea, this specification also provides a corresponding road defect detection system based on hierarchical federated learning. This system is used to implement the road defect detection method based on hierarchical federated learning provided in this specification. The system includes multiple clients and one server, and the implementation process of the system includes: For any client, the client receives global model parameters and regional model parameters of its own region from the server. The global model parameters are shared by all clients and are used to learn common structures that are invariant across regions. The regional model parameters are shared by clients in the same region and are used to compensate for consistency deviations within the region without compromising the global expressive power.

[0109] The client inputs the road image of the target road into the global model corresponding to the global model parameters. The global model performs cross-regional feature extraction on the road image to obtain a global feature representation, and generates a global prediction result based on the global feature representation.

[0110] The client inputs the global feature representation into the regional model corresponding to the regional model parameters, performs regional-specific feature modulation on the global feature representation through the regional model to obtain regional features, and generates regional prediction results based on the regional features. The client inputs regional features into the local model, maps the regional features through the local model to obtain local features, and generates local prediction results based on the local features. The local model is unique to the client and is used to enhance the client's specific adaptability to its own data distribution.

[0111] The client weights the global prediction results, regional prediction results, and local prediction results to obtain the final prediction result, and determines the road defect detection result based on the final prediction result.

[0112] In one specific embodiment, such as Figure 2 As shown, Figure 2 This is a schematic diagram of the overall architecture of a road defect detection system based on hierarchical federated learning. The system includes a federated learning server 100 and multiple clients 200. The federated learning server 100 includes a communication interface 103, a region management module 300, a dynamic aggregation module 400, a region distillation module 500, and a model fusion module 600. The modules work together to achieve model optimization under the hierarchical federated learning framework.

[0113] The federated learning server 100 establishes connections with multiple clients 200 through communication interface 103 to collaboratively complete the distributed training and updating of the road defect detection model. In this embodiment, the system connects to a total of 8 client terminals, which are deployed in different road environments.

[0114] Based on the statistical characteristics of the road environment in which the clients are located, the server divides the clients into three regions: region0, region1, and region2. Region0 contains clients C1, C2, and C3; region1 contains clients C4 and C5; and region2 contains clients C6, C7, and C8. Each client is configured with a local communication module 203 for interacting with the federated learning server regarding model parameters and control information.

[0115] The federated learning server maintains the client set and regional statistics for each region through the region management module 300, and provides the region division results to the dynamic aggregation module 400 and the region distillation module 500 for subsequent region difference-aware aggregation and region knowledge constraints. Through this system deployment, the present invention can achieve hierarchical training and collaborative optimization of the model in a multi-regional road environment.

[0116] In one embodiment, such as Figure 3 As shown, Figure 3 This is a schematic diagram of a three-layer hierarchical model structure. In this embodiment, a three-layer hierarchical model structure consisting of a global layer, a regional layer, and a local layer is constructed for a single client 200. This structure is used to achieve collaborative modeling of cross-regional sharing, regional specialization, and terminal personalization in a multi-regional road environment.

[0117] In this embodiment, client C4 within region1 is used as an example to illustrate the forward computation process of the three-layer model. Client C4 receives global model parameters from the federated learning server. Regional model parameters of region1 It also maintains local model parameters locally, which are used only for its own training. .

[0118] At the global level, the client inputs a road image. As input to the model, the global base network (global model) employs a lightweight encoder-decoder structure to extract cross-region shared features from the input image, thereby obtaining a global feature representation. : (twenty four).

[0119] And generate the corresponding global prediction output based on the global feature representation. : (25).

[0120] Among them, global layer parameters Shared across all clients, it is used to learn consistent road structure and defect semantic features across regions.

[0121] The region adaptation layer is located after the global base network and is used to adapt to... Region-specific feature modulation is performed. Specifically, each region can have independent region model parameters, and channel-wise scaling and bias transformation are applied to the input features to obtain the region features. In simplified form:

[0122] (26).

[0123] in It can be implemented by combining several convolutional layers, channel attention layers, or affine transformation linear layers. Clients within the same region share the same set of region model parameters, while different regions use different region model parameters, thereby introducing the ability to model regional differences in the feature space.

[0124] The local personalization layer, located after the region adaptation layer, is used to adjust for fine-grained features of individual clients. Its parameters are denoted as... It is trained and updated locally on the client-side C4 and does not participate in federated aggregation. This layer can consist of one or more convolutional layers or small attention modules, which integrate region features. Mapping to local features Then, a local prediction is generated through a decoding network. Accordingly, the global network output Region-adaptive output Based on and The decoding head is generated. The final output is linearly combined according to the fusion weights by the model fusion module 600 to obtain the final prediction result. .

[0125] This three-layer structure provides a clear division of parameters for sharing and specialization at different scales, enabling the model to maintain good generalization ability and local adaptability when deployed across regions.

[0126] In one embodiment, such as Figure 4 As shown, Figure 4 This diagram illustrates a process for regional partitioning and distance matrix construction under conditions where data cannot be centrally distributed, supporting subsequent regional layer modeling and difference-aware aggregation. The overall process can be represented by steps S401–S405.

[0127] S401, Feature Extraction.

[0128] In step S401, the system first extracts features from the original road dataset. Each client 200 performs feature encoding and statistical processing on the acquired road image data locally, extracts intermediate layer features, and obtains a fixed-length feature mean vector through methods such as global averaging. μTo reduce communication overhead and privacy risks, the client only uploads the local feature mean vector. μ Instead of uploading the original image data.

[0129] S402, Cluster Analysis and Region Division.

[0130] In step S402, the federated learning server 100 collects the feature mean vectors from each client and performs clustering processing on the federated learning server 100 to complete the automatic region division. In this embodiment, the federated learning server 100 uses K-means clustering to divide the clients into multiple region sets and assigns a corresponding region identifier to each client, forming region sets such as region0, region1, and region2.

[0131] For example, in a specific instance, the federated learning server 100 assigns clients C1, C2, and C3 to region 0, clients C4 and C5 to region 1, and clients C6, C7, and C8 to region 2. The road data collection environments in region 0 and region 1 are quite similar, while the road environments in region 2 are significantly different.

[0132] S403, Cluster center calculation.

[0133] In step S403, the federated learning server 100 statistically summarizes the client features within the same region and calculates the cluster center vector for each region. μᵢ This is used to represent the overall road image distribution characteristics of the region. The region center vector is calculated by weighting the mean vectors of features from each client within the region.

[0134] S404, Construction of the regional distance matrix.

[0135] In step S404, the federated learning server 100 constructs a region distance matrix based on the Euclidean distance between the center vectors of each region.

[0136] In the above example, since the road scenes of region0 and region1 are similar, their corresponding center vector distances RDM[0,1] are small; the center vectors of region1 and region2 are significantly different, and their corresponding distances RDM[1,2] are significantly greater than RDM[0,1].

[0137] S405, write operation.

[0138] In step S405, the region division results and the region distance matrix are written into the region management module 300 for subsequent region difference perception aggregation and region distillation processes. Simultaneously, the region distance matrix, as a quantitative indicator describing the degree of regional heterogeneity, provides state input to the Q-learning dynamic weight optimization module, enabling explicit modeling of environmental differences across multiple regions.

[0139] In one embodiment, such as Figure 5 As shown, Figure 5 The local training process for a single client 200 in a round of federated training mainly includes steps S501 to S504.

[0140] S501 receives parameters from the server and loads the local dataset.

[0141] In step S501, the client 200 receives the model parameters and control information issued in the current round from the federated learning server 100 through the communication module 203, including global model parameters, regional model parameters of the region, and three-layer fusion weights. This includes parameters for the corresponding regional teacher snapshot model. Simultaneously, the client reads a pre-divided local region dataset and constructs a data loader for images and labeled masks.

[0142] S502: Construct a three-layer hierarchical model, perform local training, and calculate the total loss.

[0143] In step S502, the client constructs a three-layer hierarchical model structure based on the received parameters and performs a local training process. Specifically, for each mini-batch sample, the global feature representation is first calculated through the global base network. Regional features are obtained through the regional adaptation layer. Local features are then obtained through a local personalization layer. Generate global predictions respectively Regional forecasting and local forecasts The client uses the main loss function defined for the segmentation task. (such as cross-entropy or Dice loss) and region distillation constraints obtained from teacher snapshots. Construct the total loss function, as shown in formula (13).

[0144] in The L2 distance in the feature space can be used. The client then performs backpropagation on the trainable parameters and updates the corresponding parameters in the global layer, the region layer, and the local layer.

[0145] S503, update the aggregateable parameters.

[0146] In step S503, after the current round of local training iterations reaches a preset number of epochs, the client extracts a subset of parameters that can participate in federated aggregation from the updated model, including global layer weights. Regional layer weights and local sample size Information such as local personalization layer weights. It is kept locally and is not uploaded.

[0147] S504, client update package.

[0148] In step S504, the client sends the packaged update results back to the federated learning server 100 via communication module 203. After receiving the upload results from multiple clients, the federated learning server 100 updates the global and regional layer parameters uniformly through an aggregation process. Through steps S501 to S504 given in this embodiment, the client completes collaborative training with the server without disclosing the original image data.

[0149] In one embodiment, such as Figure 6 As shown, Figure 6 This is a schematic diagram of the server aggregation process. This embodiment provides a hierarchical aggregation method executed by the federated learning server 100 in each round of communication. It mainly consists of steps S601 to S5606, which are used to update the parameters of the global layer and the regional layer based on the results uploaded by the client, and generate the model version to be issued in the next round.

[0150] S601 receives client updates and categorizes clients by region.

[0151] In step S601, the federated learning server 100 receives the training results uploaded by all clients 200 in this round through the communication interface 103, including the global model parameters updated by the corresponding clients. Regional model parameters Local sample size The region tag (region) k The server establishes a result caching structure based on regions on a local basis, and prepares to group and aggregate the results uploaded by each client.

[0152] S602, Global Aggregation.

[0153] In step S602, the federated learning server 100 calls the dynamic aggregation module 400 to first perform weighted aggregation on the global model parameters updated by all clients. Based on the client sample size, this embodiment uses an extended form of FedAvg to calculate the new round of global layer parameters: .

[0154] The dynamic aggregation module 400 will generate the results Write the model to the global model storage area of ​​the federated learning server 100 to provide a foundation for subsequent region layer aggregation.

[0155] S603, Regional Aggregation.

[0156] In step S603, the federated learning server 100 aggregates the regional model parameters within the region. For each region i, the regional layer parameters uploaded by the clients under that region are weighted by the sample size to obtain the regional aggregation result, as shown in formula (17). This step ensures that the feature modulation parameters within the region can reflect the local data distribution of that region.

[0157] S604, Difference Perception Correction.

[0158] In step S604, the federated learning server 100 performs weak sharing correction between regions based on the region distance matrix RDM. For regions i The final regional layer parameters are aggregated within the region and weighted across regions, as shown in formula (16).

[0159] For example, the RDM distance between region0 and region1 is small, while the distance between region2 and region1 is large.

[0160] S605, generate a new round of parameters.

[0161] In step S605, the federated learning server 100 adjusts the weights of the model fusion module 600 based on the aggregation results of each region. The fusion weights are updated synchronously. These weights originate from the Q-learning control module and will be distributed to all clients in the next training round.

[0162] S606, save parameters and proceed to the next update.

[0163] In step S606, the server uniformly saves the parameters obtained from the current aggregation as the "server model version for round t" and generates a teacher snapshot for regional distillation using snapshot_manager. Subsequently, the server enters the next round of parameter broadcasting, i.e., re-executes steps S601-S605.

[0164] Through steps S601 to S606 described in Example 5, the server side implements two-level weighted aggregation of the global layer and the regional layer, and improves the cross-regional adaptability of the model by combining regional difference modeling.

[0165] In one embodiment, such as Figure 7 As shown, Figure 7To optimize the dynamic parameters of Q-learning, this embodiment addresses the problem of difficulty in selecting fusion weights in highly heterogeneous federated learning by introducing a Q-learning module on the server side to achieve automatic policy search. The process consists of steps S701 to S706.

[0166] S701, construct the current state.

[0167] In step S701, the federated learning server 100 calls the Q-learning control submodule (deployed on the federated learning server 100) to construct the state vector for the current round. The state vector consists of the global mIoU from the previous round, regional difference metrics (such as the RDM mean), etc. The state vector is smoothed through an experience pool to guide action selection.

[0168] S702, based on ε - Greedy strategy for selecting actions.

[0169] In step S702, the federated learning server 100 selects from the action set according to a policy (e.g., ε-greedy). Select this round's action The action represents a candidate combination of weights for the three-layer fusion, derived from a discretized grid. The selected weight combination will be used in the next round of model fusion and client training.

[0170] S703, calculate rewards.

[0171] In step S703, the federated learning server 100 receives the reward for this round after completing the aggregation and evaluation. The reward is typically defined as the amount of improvement in overall performance.

[0172] S704, update Q value.

[0173] In step S704, the federated learning server 100 updates the Q value according to formula (22).

[0174] S705 stores the Q-value table.

[0175] In step S705, the updated Q-value table is written locally for use in the next round of action selection.

[0176] S706, Action Application.

[0177] In step S706, the federated learning server 100 applies the fusion weight combination corresponding to the optimal action to the fusion formula of the three-layer output and sends it to all clients at the start of the next round of federated iteration.

[0178] Through the above process, the Q-learning module continuously explores better strategies during training, enabling the weights of the three-layer structure to be adaptively adjusted, thereby improving the overall training stability and convergence speed.

[0179] In one embodiment, such as Figure 8 As shown, Figure 8 This is a flowchart illustrating the specific implementation of the regional distillation mechanism. This mechanism transmits the stable feature spatial structure within a region to the regional adaptation layer through regional teacher snapshots, and includes steps S801–S805.

[0180] S801, obtain the regional teacher model snapshot and regional model parameters.

[0181] In step S801, the federated learning server 100 calls snapshot_manager to save the "t-th round region adapter parameters" for each region, forming a region-level teacher model snapshot: This snapshot serves as a teacher for regional distillation.

[0182] S802, extract features.

[0183] In step S802, after receiving the snapshot of the regional-level teacher model and the regional model parameters, the client 200 freezes and loads them into its local device. The teacher model is only used for forward inference and does not participate in training; regional features are extracted through the regional-level teacher model and the regional model, respectively.

[0184] S803, Feature Alignment.

[0185] In step S803, during the training process, the client outputs the features of the region adaptation layer. Features generated by the teacher model Align them.

[0186] S804, Calculation of total loss for the split task.

[0187] S805, Update regional model parameters.

[0188] In step S805, the regional layer weights are updated under distillation constraints through backpropagation, making them closer to the stable feature distribution of the region.

[0189] This regional distillation mechanism ensures that the feature space within a region remains consistent across multiple rounds of communication, improves the robustness of the regional layer, and provides additional structural priors for data-scarce regions.

[0190] In one embodiment, such as Figure 9 As shown, Figure 9This is a schematic diagram of the three-layer fusion; this embodiment illustrates the output mechanism of the three-layer fusion module 600. The three-layer fusion module is used to combine the global output, regional output, and local output according to weights to form the final prediction.

[0191] S901, global base network generates global output.

[0192] S902, the region adaptation layer generates region output.

[0193] S903, local personalization layer generates local output.

[0194] S904, the fusion module 600 calls the following linear fusion formula to generate the final prediction result.

[0195] The S905 outputs the final prediction results to the client application for road defect detection tasks.

[0196] The three-layer output fusion enables the model to simultaneously take into account general knowledge, regional features, and personalized needs, thereby achieving better overall performance in highly heterogeneous scenarios.

[0197] This invention achieves a road defect detection framework with significantly improved privacy protection, cross-regional generalization, and robustness through a three-layer hierarchical structure, region adaptation, reinforcement learning dynamic aggregation, and snapshot strategy, which has significant technical advantages compared to traditional federated methods.

[0198] Specific limitations regarding the road defect detection system based on hierarchical federated learning can be found in the limitations of the road defect detection method based on hierarchical federated learning mentioned above, and will not be repeated here. Each module in the aforementioned road defect detection system based on hierarchical federated learning can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0199] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0200] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A road defect detection method based on hierarchical federated learning, characterized in that, The method is applied to a road defect detection system based on hierarchical federated learning, which includes multiple clients and one server, including: For any client, the client receives global model parameters and regional model parameters of its own region from the server; the global model parameters are shared by all clients and are used to learn the common structure that is invariant across regions; the regional model parameters are shared by clients in the same region and are used to compensate for consistency deviations within the region without destroying the global expressive power. The client inputs the road image of the target road into the global model corresponding to the global model parameters. The global model performs cross-regional shared feature extraction on the road image to obtain a global feature representation. Based on the global feature representation, a global prediction result is generated. The client inputs the global feature representation into the regional model, performs region-specific feature modulation on the global feature representation through the regional model corresponding to the regional model parameters, obtains regional features, and generates regional prediction results based on the regional features. The client inputs regional features into the local model, maps the regional features through the local model to obtain local features, and generates local prediction results based on the local features. The local model is unique to the client and is used to enhance the client's specific adaptability to its own data distribution. The client weights the global prediction results, regional prediction results, and local prediction results to obtain the final prediction result, and determines the road defect detection result based on the final prediction result.

2. The method according to claim 1, characterized in that, The process of regional division includes: The client extracts feature vectors representing the statistical characteristics of the local road images and uploads these feature vectors to the server. Based on the similarity between feature vectors, the server performs clustering processing on feature vectors from different clients, dividing all clients into multiple region clusters, with each cluster corresponding to a region; The server constructs a region distance matrix based on the feature mean vectors of each region; the elements of the region distance matrix represent the Euclidean distance between the feature mean vectors of two corresponding regions. The server sends the corresponding region identifier to each client.

3. The method according to claim 2, characterized in that, The process of determining global model parameters, regional model parameters, and local model parameters includes: For any client, the server will send the initialized global model parameters, the regional model parameters of the client's region, the client's local model parameters, and the fusion weights of each model to the client. The client uses the global model, regional model, local model, and the fusion weights of each model to update the parameters of the global model, regional model, and local model. The client updates and saves the updated local model parameters as parameters of the local model, and uploads the updated global model parameters and regional model parameters as aggregate parameters to the server; The server performs global aggregation based on the global model parameters sent by all clients to obtain aggregated global model parameters. At the regional level, it performs regional aggregation on the regional model parameters sent by clients within the same region, based on the regional distance matrix, to obtain aggregated regional model parameters. Based on the model training status and performance feedback, the server adjusts the fusion weights of each model and redistributes the updated global model parameters, regional model parameters, and fusion weights to the client, entering the next round of federated training until the preset termination condition is met, thus obtaining the road defect detection model. The parameters of the road defect detection model include the global model parameters, regional model parameters, local model parameters, and fusion weights corresponding to the termination condition.

4. The method according to claim 3, characterized in that, The parameters of the global model, regional model, and local model are updated using the global model, regional model, local model, and the fusion weights of each model. This includes: For each training sample, the global model, regional model, local model, and the fusion weights of each model are used to determine the global prediction result, regional prediction result, local prediction result, and final prediction result of the sample road image. The main loss for the segmentation task is determined based on the global prediction results, regional prediction results, local prediction results, and final prediction results of the sample. Obtain a snapshot of the client's regional teacher model; the snapshot is generated by the regional model parameters of the client's region in the previous round or a historical round. Based on the sample region features extracted from the client's regional model and the sample region features extracted from the snapshot of the regional teacher model, as well as the sample region features extracted from the snapshot of the regional teacher model in other regions, the regional distillation constraints are determined, and the total loss is determined based on the main loss of the segmentation task and the regional distillation constraints. Backpropagation is performed on the global model parameters, regional model parameters, and local model parameters using the total loss; Once the local training reaches the preset number of iterations, the global model parameters, regional model parameters, and local model parameters corresponding to the preset number of iterations will be determined as the final global model parameters, regional model parameters, and local model parameters for this round.

5. The method according to claim 4, characterized in that, Based on the sample region features extracted from the client's regional model and the sample region features extracted from the snapshot of the regional teacher model, as well as the sample region features extracted from the snapshot of the regional teacher model in other regions, regional distillation constraints are determined, including: Based on the sample region features extracted from the client's regional model and the sample region features extracted from the snapshot of the regional teacher model, the distillation loss within the region is determined; Based on the regional distance matrix and the sample regional features extracted from the regional model, as well as the sample regional features extracted from the regional teacher model snapshots of other regions, cross-regional distillation terms are determined. Regional distillation constraints are determined based on intra-regional distillation losses and inter-regional distillation terms.

6. The method according to claim 5, characterized in that, The distillation loss function within the region is: ； in, Distillation loss within the region, Sample region features extracted from the region model of the client's region. The sample region features are extracted from the snapshot of the regional teacher model for the client's region. Indicates Euclidean distance; The cross-regional distillation term is: ； ； in, For cross-regional distillation, For the region With the region Distillation weights between For the region The sample region features extracted from the snapshot of the regional-level teacher model. For regions in the region distance matrix With the region The Euclidean distance between them This is the attenuation coefficient due to distillation distance; The zone distillation constraint is: ； in, For regional distillation constraints, This is the inter-regional distillation adjustment coefficient; The total loss function is: ； in, For the total loss, To divide the main loss of the task, This is the distillation loss weighting coefficient.

7. The method according to claim 3, characterized in that, The global aggregation formula for global model parameters is: ； in, These are the aggregated global model parameters. For the total number of clients, For the first The number of training samples per client. The total number of training samples across all clients. For the first Global model parameters for each client; The formula for calculating region aggregation is: ； ； ； in, These are the parameters of the aggregated region model. To achieve cross-regional sharing ratios, For the region With the region Similarity weights between them For the region Regional adaptation parameters, For the region The number of Chinese clients For the first The number of training samples per client. For the region The total number of training samples across all clients. For the first Regional model parameters for each client, For the region With the region The Euclidean distance between them This is the polymerization distance attenuation coefficient.

8. The method according to claim 3, characterized in that, The model training status includes the current training epoch, and the performance feedback includes the global average intersection-over-union ratio (IoU) of the defect detection task corresponding to the previous epoch. Based on the model training status and performance feedback, the fusion weights of each model are adjusted, including: Obtain the regional distribution difference index; the regional distribution difference index is derived from the regional distance matrix. The global average intersection-over-union ratio, regional distribution difference index, and the current round are input into a pre-trained reinforcement learning algorithm to obtain the action set of the reinforcement learning algorithm. Action set for: ； in, For the fusion weights of the global model, For the fusion weights of the regional model, For the fusion weights of the local model; .

9. The method according to claim 8, characterized in that, The reinforcement learning algorithm uses the Q-learning algorithm, and the reward function of the Q-learning algorithm is: ； ； in, For the first The reward function value for each round, Indicates performance gain. This is the regional difference penalty coefficient. This is an indicator of regional distribution differences. For the first The global average intersection-union ratio of the rounds, For the first The global average intersection-union ratio of the rounds.

10. A road defect detection system based on hierarchical federated learning, characterized in that, The system includes multiple clients and one server; For any client, the client receives global model parameters and regional model parameters of its own region from the server; the global model parameters are shared by all clients and are used to learn the common structure that is invariant across regions; the regional model parameters are shared by clients in the same region and are used to compensate for consistency deviations within the region without destroying the global expressive power. The client inputs the road image of the target road into the global model corresponding to the global model parameters. The global model performs cross-regional shared feature extraction on the road image to obtain a global feature representation. Based on the global feature representation, a global prediction result is generated. The client inputs the global feature representation into the regional model corresponding to the regional model parameters, performs regional-specific feature modulation on the global feature representation through the regional model to obtain regional features, and generates regional prediction results based on the regional features. The client inputs regional features into the local model, maps the regional features through the local model to obtain local features, and generates local prediction results based on the local features. The local model is unique to the client and is used to enhance the client's specific adaptability to its own data distribution. The client weights the global prediction results, regional prediction results, and local prediction results to obtain the final prediction result, and determines the road defect detection result based on the final prediction result.

Images