A Three-Dimensional Inspection Method for Surface Defects of Dams Using Multiple Unmanned Vehicles in a Cloud-Edge Collaborative Environment

Abstract

This invention discloses a three-dimensional inspection method for dam surface defects using multiple unmanned vehicles in a cloud-edge collaborative environment. To train dam defect detection models for unmanned equipment, it introduces equipment index metrics, metric-based equipment grouping, and intra-group collaborative aggregation techniques. By dynamically adjusting the topology of the hierarchical federated network, it alleviates the loss of defect detection model accuracy caused by inconsistent data distribution among different unmanned vehicles while reducing the energy consumption required for unmanned equipment training. Equipment index metrics reflect the data distribution of data collected by the unmanned vehicles and their own communication capabilities, allowing for a trade-off between the two to control the importance of certain components. This invention solves the problem of excessive communication energy consumption when unmanned vehicles train dam defect detection models through hierarchical federated learning, reducing energy consumption while ensuring the accuracy of the detection model, thereby improving the three-dimensional inspection effect of dam defects based on unmanned vehicles.

Description

Technical Field

[0001] This invention relates to a three-dimensional inspection method for surface defects of dams using multiple unmanned vehicles in a cloud-edge collaborative environment, belonging to the field of computer vision technology. Background Technology

[0002] Traditional dam defect inspection relies primarily on manual inspection, a method with numerous drawbacks, including being time-consuming, labor-intensive, inefficient, and susceptible to human error, making it difficult to achieve efficient and accurate defect identification. To address these issues, machine learning techniques have been introduced to train dam defect detection models, improving accuracy and efficiency through automated analysis. With the widespread use of unmanned equipment (such as drones and smart sensors), data collection has become more convenient and comprehensive. However, the massive amounts of data collected by unmanned equipment consume significant network resources during transmission, leading to a substantial increase in communication pressure. Furthermore, this data may contain sensitive information, making it difficult to centralize data collection for machine learning training. Therefore, a hierarchical federated learning framework can be introduced into three-dimensional dam defect inspection. Hierarchical federated learning, through collaborative model training among cloud servers, edge servers, and unmanned equipment, effectively integrates local data from various unmanned devices while ensuring data privacy and security, achieving efficient defect identification and prediction. The hierarchical structure makes the model training process more flexible and scalable, adapting to the complex topology and dynamic changes of the dam inspection network, significantly reducing communication overhead and computational latency. The multi-unmanned vehicle three-dimensional inspection of dam surface defects in a cloud-edge collaborative environment can improve the accuracy and real-time performance of defect detection, providing solid technical support for the intelligent management of water conservancy projects.

[0003] While multi-unmanned vehicle (UAV) 3D inspection of dam surface defects in a cloud-edge collaborative environment offers significant advantages in improving detection accuracy and protecting data privacy, it still faces several key challenges in practical applications. First, the data collected by UAVs exhibits heterogeneity. Differences in acquisition methods, sensor types, and data quality among different devices lead to lower accuracy in the trained dam defect detection models. This data heterogeneity causes inconsistent model performance across different devices, reducing overall detection accuracy. Second, the communication conditions between UAVs and edge servers vary, manifesting as differences in network bandwidth, latency, and connection stability, resulting in significant communication heterogeneity. Devices with poor communication conditions consume more energy during data transmission, increasing energy consumption during training. Some UAVs, such as drones and unmanned vehicles, typically rely on limited battery power, making energy consumption particularly significant for these resource-constrained devices. High energy consumption not only shortens equipment operating time, limiting inspection coverage and frequency, but can also lead to premature equipment failure and increased maintenance costs. Since dam inspection tasks often require long-term, high-frequency monitoring, energy consumption directly impacts the system's continuous operation and inspection efficiency. Therefore, how to effectively reduce the energy consumption of unmanned equipment while ensuring the accuracy of the defect detection model has become a key challenge that needs to be addressed in the multi-unmanned vehicle dam surface defect three-dimensional inspection system in a cloud-edge collaborative environment. Summary of the Invention

[0004] Purpose of the invention: To address the problem that the heterogeneity of data and communication between unmanned equipment leads to the loss of accuracy of dam defect detection models trained in a cloud-edge collaborative environment and the high energy consumption of communication during the training process, this invention provides a three-dimensional inspection method for dam surface defects using multiple unmanned vehicles in a cloud-edge collaborative environment. By adjusting the hierarchical federated network topology, the method reduces the communication energy consumption of unmanned equipment during the training process while ensuring the accuracy of the trained model, thereby improving the accuracy of dam defect detection and the energy efficiency of the inspection process.

[0005] Technical Solution: A three-dimensional inspection method for dam surface defects using multiple unmanned vehicles in a cloud-edge collaborative environment. This method trains a dam defect detection model for unmanned vehicles based on a three-layer cloud, edge, and terminal architecture. The method includes local model training, local model uploading, edge-terminal model aggregation, and cloud-based model aggregation stages. Unmanned vehicles collect dam defect image data, and a local model is trained on the unmanned vehicles based on this image data. Finally, the local models on the unmanned vehicles are hierarchically aggregated to obtain a global model, i.e., the dam defect detection model. The method includes the following steps:

[0006] Step 1) During the local model training phase, each unmanned device in each edge server group trains a local dam defect detection model based on the dam defect image data it has collected. In addition, the unmanned device records its communication speed with each edge server and other unmanned devices in the group at certain time intervals and stores it in the communication speed recording queue. After the unmanned device completes a specified number of rounds of local model training, it triggers the local model upload.

[0007] Step 2) In the local model upload stage, each unmanned device completes local model upload and local model aggregation according to the model aggregation tree generated in the previous edge aggregation stage. At the same time, each unmanned device calculates the communication metric with its own edge server and other unmanned devices in the group based on the communication speed recording queue, and uploads it directly to its own edge server.

[0008] Step 3) In the edge model aggregation stage, the edge server performs weighted aggregation on the received local models to obtain the edge model. In addition, it calculates the model aggregation tree for the next edge iteration based on the communication metric matrix uploaded by the unmanned devices in the group. Then, the edge server distributes the edge model and the model aggregation tree to each unmanned device in the group. The unmanned devices use the edge model to update their local models and enter the next edge iteration. After the edge iteration reaches a specified number, the edge model upload is triggered.

[0009] Step 4) During the edge model upload phase, each edge server uploads its own edge model to the cloud server. At the same time, each group of unmanned devices is required to calculate and upload communication metrics and local data distribution metrics, and forward the two metrics to the cloud server for calculating the grouping of unmanned devices in the next round of cloud iteration.

[0010] Step 5) In the cloud model aggregation stage, the cloud server performs weighted aggregation on the received edge models to obtain the cloud model. In addition, it calculates the grouping results of the unmanned devices based on the communication metrics and data distribution metrics of each unmanned device, and adjusts the federated network structure according to the grouping results. Finally, the cloud model is sent to each edge server to enter the next round of cloud iteration. Steps 1)-5) are repeated until the cloud model reaches the specified accuracy or the cloud iteration reaches the specified number of rounds, and finally the dam defect detection model is obtained and sent to each unmanned device to perform the dam defect detection task.

[0011] Furthermore, the specific steps for training the local dam defect detection model using unmanned equipment in step 1) are as follows:

[0012] Unmanned equipment c i In the r-th local iteration, stochastic gradient descent is used to update the local dam defect detection model:

[0013]

[0014] in, Unmanned equipment c i The parameters of the local dam defect detection model in the (r+1)th and rth rounds of local iteration are given, where η is the learning rate in stochastic gradient descent. For the r-th round local model In local dataset D i The gradient calculated above is used to trigger the local model upload after the unmanned device performs a specified number of local iterations by applying the above formula.

[0015] Furthermore, in step 1), the unmanned device records its communication speed with each edge server and other unmanned devices in the group at regular time intervals. The specific steps are as follows:

[0016] Edge server e j Subordinate unmanned equipment c i With a node within the group k The communication speed between them can be expressed as:

[0017]

[0018] Among them, node k ∈e j ∪C j C j For edge server e j Subordinate unmanned equipment collection, B ik For node k Assigned to unmanned equipment c i bandwidth, p i For unmanned equipment c i The upload power, N0 is the power spectral density, For unmanned equipment c i With node k The channel gain between, where d ik For unmanned equipment c i With node k The distance between them, α is the path decay constant, L ik Let c be a random variable; during the local computation phase, the unmanned device c i Calculate the relationship with node at regular intervals. k The communication speed between them is recorded and stored in the communication speed record queue. In this context, l represents the maximum length of the queue. When the actual number of elements in the queue is less than l, the communication speed to be recorded is directly stored at the end of the queue; otherwise, the first element at the head of the queue is deleted before being stored.

[0019] Furthermore, in step 2), local model uploading and local model aggregation are completed according to the model aggregation tree, and model parameters are uploaded to the edge server; the specific steps are as follows:

[0020] In the r-th edge iteration, edge server e j The model aggregation tree of the group can be represented as e j ∪C j Let C be the set of nodes in the tree. j For edge server e j Subordinate unmanned equipment collection, R j To represent the node relationships in the model aggregation tree, some unmanned devices are represented as the leaf node set C in the model aggregation tree. leaf Edge server e j The root node is the model aggregation tree, and the remaining unmanned devices form the intermediate node set C. mid During the aggregation process, the local model of the unmanned device is uploaded and aggregated along each path from the leaf node to the root node. The roles of different types of nodes in the aggregation process are as follows:

[0021] For unmanned equipment c at leaf nodes i ∈C leaf Simply upload the local model parameters to the parent node;

[0022] For the unmanned equipment c at the intermediate node i ∈C mid It needs to wait for its child node set. After all unmanned devices upload their local model parameters, their own model parameters are then locally weighted and aggregated with the model parameters of the child unmanned devices:

[0023]

[0024] Among them, |D j |For unmanned equipment c j The number of images of dam defects collected. For edge server e j The total amount of data collected by all subordinate unmanned equipment. For the unmanned equipment c at the intermediate node after aggregation i The parameters of the local aggregation model, For unmanned equipment c j The local model parameters are uploaded from the intermediate node to the parent node after local aggregation.

[0025] For the root node edge server e j Once all local models within the group have been aggregated and uploaded, edge model aggregation is triggered.

[0026] Furthermore, the specific steps for the unmanned equipment to calculate the communication metric in step 2) are as follows:

[0027] For unmanned equipment c i Records and nodes k Communication speed recording queue From the head of the team To the end of the line The order is according to Calculate communication metrics In the formula This is the smoothing coefficient.

[0028] Furthermore, the specific steps of the edge server computing model aggregation tree in step 3) are as follows:

[0029] Step 31: In the r-th edge iteration, define the communication metric matrix. Define a loop detection set list, Loop, with an initial length of |C|. j |+1, and each element in the list is an empty set, initialize the model aggregation tree. At this time R j It is an empty set;

[0030] Step 32, search Largest element If c i With node k If they do not belong to the same set in the Loop, then c i With node k Add to model aggregation tree In the set of nodes, c i With node k The connection was added to Node association relationship R j If the condition is met, skip step 33 and proceed to step 34; otherwise, execute step 33.

[0031] Step 33, merge c i With node k The two corresponding sets in the loop detection set list;

[0032] Step 34, final settings Largest element

[0033] Step 35: Repeat steps 32-34 until all nodes are added to the model aggregation tree. middle.

[0034] Furthermore, the specific steps for the unmanned equipment to calculate the local data distribution metric in step 4) are as follows:

[0035] For unmanned equipment c i The dataset of images of dam defects held by D i Its data distribution metric can be expressed as dis(D) i )={(c,n c )|c∈{1,2,...}}, where c is the data sample category, n c The number of data samples for category c.

[0036] Furthermore, the specific steps for calculating the grouping results of unmanned devices on the cloud server in step 5) are as follows:

[0037] Step 51, initialize the grouping results for each edge server e. j ∈E, and its subordinate unmanned equipment set C j Let E be an empty string, and E be the set of edge servers. Define a matrix. C represents a collection of unmanned equipment;

[0038] Step 52, for each unmanned device c in the set of unmanned devices C i ∈C, calculate its addition to each edge server e j After ∈E, the richness of data distribution within the edge server group In the formula p c Define a matrix to represent the probability of a class c data sample appearing in the dataset.

[0039] Step 53, calculate the combined metric matrix In the formula, λ is used to control the proportion of communication energy consumption optimization. This is a regularization function used to map the values ​​of matrix elements to the range of 0-1; for the grouped unmanned equipment c i Combined metric matrix The value of the element in the i-th row is set to -1;

[0040] Step 54, find the combined metric matrix The largest element u in the middle ij Unmanned equipment c i Add edge server e j C, a collection of subordinate unmanned equipment j In the middle, and mark the unmanned equipment c i Grouped;

[0041] Repeat steps 52-54 until all unmanned devices have been grouped.

[0042] A three-dimensional inspection system for surface defects of dams using multiple unmanned vehicles in a cloud-edge collaborative environment includes the following modules:

[0043] The local model training module allows each unmanned device in each edge server group to train a local dam defect detection model based on the dam defect image data it collects. In addition, the unmanned device records its communication speed with each edge server and other unmanned devices in the group at regular intervals and stores it in the communication speed recording queue. After the unmanned device completes a specified number of rounds of local model training, it triggers the local model upload.

[0044] In the local model upload module, each unmanned device completes local model upload and local model aggregation according to the model aggregation tree generated in the previous edge aggregation stage. At the same time, each unmanned device calculates the communication metric with its own edge server and other unmanned devices in the group based on the communication speed recording queue, and uploads it directly to its own edge server.

[0045] The edge model aggregation module involves the edge server performing weighted aggregation on the received local models to obtain the edge model. In addition, it calculates the model aggregation tree for the next edge iteration based on the communication metric matrix uploaded by the unmanned devices in the group. Subsequently, the edge server distributes the edge model and the model aggregation tree to each unmanned device in the group. The unmanned devices use the edge model to update their local models and enter the next edge iteration. After the edge iteration reaches a specified number, the edge model upload is triggered.

[0046] The edge model upload module requires each edge server to upload its own edge model to the cloud server. At the same time, it requires each group of unmanned devices to calculate and upload communication metrics and local data distribution metrics, and forward the two metrics to the cloud server for calculating the grouping of unmanned devices in the next round of cloud iteration.

[0047] The cloud model aggregation module performs weighted aggregation on the received edge models to obtain the cloud model. In addition, it calculates the grouping results of the unmanned devices based on the communication metrics and data distribution metrics of each unmanned device, and adjusts the federated network structure according to the grouping results. Finally, the cloud model is distributed to each edge server to enter the next round of cloud iteration. When the cloud model reaches the specified accuracy or the cloud iteration reaches the specified number of rounds, the final cloud model is the dam defect detection model.

[0048] The specific implementation process and methods of the system are the same, and will not be repeated here.

[0049] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the three-dimensional inspection method for surface defects of dams using multiple unmanned vehicles in a cloud-edge collaborative environment, as described above.

[0050] Beneficial Effects: Compared with existing technologies, the cloud-edge collaborative method for three-dimensional inspection of dam surface defects using multiple unmanned vehicles in a cloud environment combines equipment index measurement, metric-based equipment grouping, and intra-group collaborative aggregation technology. This reduces the communication energy consumption of unmanned vehicles during the training process while ensuring the accuracy of the dam defect detection model, thus optimizing the energy efficiency of the inspection system. Equipment index measurement considers both the communication capabilities of the equipment and the local data distribution characteristics, and can balance these two aspects according to different scenarios. Metric-based equipment grouping is used to adjust the network topology between edge servers and unmanned vehicles, controlling the differences in data distribution within different edge server groups and the global data distribution while improving the overall system communication rate. This ensures the final accuracy of the dam defect detection model and reduces the communication energy consumption of unmanned vehicles. Intra-group collaborative aggregation allows communication between different unmanned vehicles, further reducing the overall system communication energy consumption by adjusting the communication network structure within the edge server groups. This invention solves the problems of excessive network resource consumption and privacy leakage when unmanned devices upload defect image samples they have collected, alleviates the problem of decreased accuracy of dam defect detection models caused by the distribution differences of data collected by unmanned devices, and solves the problem of excessive energy consumption during training caused by differences in network bandwidth, latency and connection stability of unmanned devices. Attached Figure Description

[0051] Figure 1 This is a flowchart of a method according to an embodiment of the present invention. Detailed Implementation

[0052] The present invention will be further illustrated below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. After reading the present invention, any modifications of the present invention in various equivalent forms by those skilled in the art will fall within the scope defined by the appended claims.

[0053] A three-dimensional inspection method for dam surface defects using multiple unmanned vehicles in a cloud-edge collaborative environment combines equipment index measurement, measurement-based equipment grouping, and intra-group collaborative aggregation technology to adjust the topology of the federated training network. This method can reduce the communication energy consumption of unmanned equipment while ensuring the accuracy of the dam surface defect detection model, thereby improving the service life of unmanned equipment and optimizing the energy efficiency of the inspection system.

[0054] like Figure 1 As shown, the three-dimensional inspection method for surface defects of dams using multiple unmanned vehicles in a cloud-edge collaborative environment includes the following steps:

[0055] Step 1) During the local model training phase, the edge server e j Unmanned equipment within the group c i In the r-th local iteration, the local dam defect detection model is updated using stochastic gradient descent based on the dam defect image data collected by the model itself.

[0056]

[0057] in, Unmanned equipment c i The parameters of the local dam defect detection model in the (r+1)th and rth rounds of local iteration are given, where η is the learning rate in stochastic gradient descent. For the r-th round local model In local dataset D i The gradient calculated above; in addition, the unmanned device c i Record its relationship with a node in the group at regular intervals according to the following formula. k Inter-communication speed:

[0058]

[0059] In the formula, node k ∈e j ∪C j C j For edge server e j Subordinate unmanned equipment collection, B ik For node k Assigned to unmanned equipment c i bandwidth, p i For unmanned equipment c i The upload power, N0 is the power spectral density, For unmanned equipment c i With node k The channel gain between, where d ik For unmanned equipment c i With node k The distance between them, α is the path decay constant, L ik Let c be a random variable; during the local computation phase, the unmanned device c i Calculate the relationship with node at regular intervals. k The communication speed between them is recorded and stored in the communication speed record queue. In this context, l represents the maximum queue length. When the actual number of elements in the queue is less than l, the communication speed to be recorded is directly stored at the end of the queue; otherwise, the first element at the head of the queue is deleted before being stored. After the unmanned device executes a specified number of local iterations, it triggers the local model upload.

[0060] Step 2) In the local model upload phase, each unmanned device completes local model upload and local model aggregation according to the model aggregation tree generated in the previous edge aggregation phase. In the r-th edge iteration, the edge server e j The model aggregation tree of the group can be represented as ej ∪C j Let C be the set of nodes in the tree. j For edge server e j Subordinate unmanned equipment collection, R j To represent the node relationships in the model aggregation tree, some unmanned devices are represented as the leaf node set C in the model aggregation tree. leaf Edge server e j The root node is the model aggregation tree, and the remaining unmanned devices form the intermediate node set C. mid During the aggregation process, the local model of the unmanned device is uploaded and aggregated along each path from the leaf node to the root node. The roles of different types of nodes in the aggregation process are as follows:

[0061] For unmanned equipment c at leaf nodes i ∈C leaf Simply upload the local model parameters to the parent node;

[0062] For the unmanned equipment c at the intermediate node i ∈C mid It needs to wait for its child node set. After all unmanned devices upload their local model parameters, their own model parameters are then locally weighted and aggregated with the model parameters of the child unmanned devices:

[0063]

[0064] Among them, |D j |For unmanned equipment c j The number of images of dam defects collected. For edge server e j The total amount of data collected by all subordinate unmanned equipment. For the unmanned equipment c at the intermediate node after aggregation i The parameters of the local aggregation model, For unmanned equipment c j The local model parameters are collected, and after local aggregation, the intermediate node uploads the locally aggregated model parameters to the parent node; simultaneously, each unmanned device c i ∈ j Based on nodes k Communication speed recording queue From the head of the team To the end of the line The order is according to Calculate communication metrics In the formula As a smoothing coefficient, the calculated communication metric is then directly uploaded to its respective edge server e. j ;

[0065] Step 3) In the edge model aggregation stage, the edge server performs weighted aggregation on the received local models to obtain the edge model. In addition, it calculates the model aggregation tree for the next round of edge iteration based on the communication metric matrix uploaded by unmanned devices within the group.

[0066] In the r-th edge iteration, define the communication metric matrix. Define a loop detection set list, Loop, with an initial length of |C|. j |+1, and each element in the list is an empty set, initialize the model aggregation tree. At this time R j It is an empty set;

[0067] Search Largest element If c i With node k If they do not belong to the same set in the Loop, then c i With node k Add to model aggregation tree In the set of nodes, c i With node k The connection was added to Node association relationship R j In the middle, simultaneously merge c i With node k The two corresponding sets in the loop detection set list; finally set Largest element Repeat this process until all nodes are added to the model aggregation tree. In the middle. Then the edge server will combine the edge model and the model aggregation tree. The data is distributed to all unmanned devices within the group. The unmanned devices use the edge model to update their local model and enter the next edge iteration. After the edge iteration reaches a specified number of times, the edge model is uploaded.

[0068] Step 4) During the edge model upload phase, each edge server uploads its own edge model to the cloud server. Simultaneously, each group's unmanned devices are required to calculate and upload communication metrics and local data distribution metrics. For unmanned device c... i The dataset of images of dam defects held by D i Its data distribution metric can be expressed as dis(D) i )={(c,n c )|c∈{1,2,...}}, where c is the data sample category, n c The number of data samples for category c; after the edge server collects the communication metrics and local data distribution metrics of its subordinate unmanned devices, it forwards and uploads them to the cloud server to calculate the grouping of unmanned devices in the next round of cloud iteration;

[0069] Step 5) In the cloud model aggregation stage, the cloud server performs weighted aggregation on the received edge models to obtain the cloud model. In addition, the unmanned device grouping results are calculated based on the communication metrics and data distribution metrics of each unmanned device:

[0070] First, initialize the grouping results for each edge server e. j ∈E, and its subordinate unmanned equipment set C j Let E be an empty string, and E be the set of edge servers. Define a matrix. C represents a collection of unmanned equipment;

[0071] For each unmanned device c in the set of unmanned devices C i ∈C, calculate its addition to each edge server e j After ∈E, the richness of data distribution within the edge server group In the formula p c Define a matrix to represent the probability of a class c data sample appearing in the dataset.

[0072] Calculate the combined metric matrix In the formula, λ is used to control the proportion of communication energy consumption optimization. This is a regularization function used to map the values ​​of matrix elements to the range of 0-1; for the grouped unmanned equipment c i Combined metric matrix The value of the element in the i-th row is set to -1;

[0073] Finding the combined metric matrix The largest element u in the middle ij Unmanned equipment c i Add edge server e j C, a collection of subordinate unmanned equipment j In the middle, and mark the unmanned equipment c i Grouped; repeat the above steps until all unmanned devices have been grouped, and finally obtain the unmanned device grouping results for the next round of cloud iteration, and adjust the federated network structure according to the grouping results; finally, the cloud server will send the cloud model to each edge server to enter the next round of cloud iteration; repeat steps 1)-5) until the cloud model reaches the specified accuracy or the cloud iteration reaches the specified number of rounds, and finally obtain the dam defect detection model, and send it to each unmanned device to perform the dam defect detection task.

[0074] Obviously, those skilled in the art should understand that the cloud-edge collaborative three-dimensional inspection method for surface defects of dams using multiple unmanned vehicles in the above-described embodiments of the present invention can be implemented using general-purpose computing devices. These methods can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by the computing device. Furthermore, in some cases, the steps shown or described can be performed in a different order than presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the embodiments of the present invention are not limited to any particular hardware and software combination.

Claims

1. A three-dimensional inspection method for surface defects of dams using multiple unmanned vehicles in a cloud-edge collaborative environment, characterized in that, The process includes local model training, local model uploading, edge model aggregation, edge model uploading, and cloud model aggregation, and includes the following steps: Step 1) During the local model training phase, each unmanned device in each edge server group trains a local dam defect detection model based on the dam defect image data it has collected. In addition, the unmanned device records its communication speed with each edge server and other unmanned devices in the group at certain time intervals and stores it in the communication speed recording queue. After the unmanned device completes a specified number of rounds of local model training, it triggers the local model upload. Step 2) In the local model upload stage, each unmanned device completes local model upload and local model aggregation according to the model aggregation tree generated in the previous edge aggregation stage. At the same time, each unmanned device calculates the communication metric with its own edge server and other unmanned devices in the group based on the communication speed recording queue, and uploads it directly to its own edge server. Step 3) In the edge model aggregation stage, the edge server performs weighted aggregation on the received local models to obtain the edge model. In addition, it calculates the model aggregation tree for the next edge iteration based on the communication metric matrix uploaded by the unmanned devices in the group. Then, the edge server distributes the edge model and the model aggregation tree to each unmanned device in the group. The unmanned devices use the edge model to update their local models and enter the next edge iteration. After the edge iteration reaches a specified number, the edge model upload is triggered. Step 4) During the edge model upload phase, each edge server uploads its own edge model to the cloud server. At the same time, each group of unmanned devices is required to calculate and upload communication metrics and local data distribution metrics, and forward the two metrics to the cloud server for calculating the grouping of unmanned devices in the next round of cloud iteration. Step 5) In the cloud model aggregation stage, the cloud server performs weighted aggregation on the received edge models to obtain the cloud model. In addition, it calculates the grouping results of the unmanned devices based on the communication metrics and data distribution metrics of each unmanned device, and adjusts the federated network structure according to the grouping results. Finally, the cloud model is sent to each edge server to enter the next round of cloud iteration. Steps 1)-5) are repeated until the cloud model reaches the specified accuracy or the cloud iteration reaches the specified number of rounds, and finally the dam defect detection model is obtained and sent to each unmanned device to perform the dam defect detection task.

2. The method for three-dimensional inspection of dam surface defects using multiple unmanned vehicles in a cloud-edge collaborative environment as described in claim 1, characterized in that, The specific steps for training the local dam defect detection model using unmanned equipment in step 1) are as follows: Unmanned equipment c i In the r-th local iteration, stochastic gradient descent is used to update the local dam defect detection model: in, Unmanned equipment c i The parameters of the local dam defect detection model in the (r+1)th and rth rounds of local iteration are given, where η is the learning rate in stochastic gradient descent. For the r-th round local model In local dataset D i The gradient calculated above is used to trigger the local model upload after the unmanned device performs a specified number of local iterations by applying the above formula.

3. The method for three-dimensional inspection of dam surface defects using multiple unmanned vehicles in a cloud-edge collaborative environment as described in claim 1, characterized in that, In step 1), the unmanned device records its communication speed with each edge server and other unmanned devices in the group at regular time intervals. The specific steps are as follows: Edge server e j Subordinate unmanned equipment c i With a node within the group k The communication speed between them can be expressed as: Among them, node k ∈e j ∪C j , C j For edge server e j Subordinate unmanned equipment collection, B ik For node k Assigned to unmanned equipment c i bandwidth, p i For unmanned equipment c i The upload power, N0 is the power spectral density, For unmanned equipment c i With node k The channel gain between, where d ik For unmanned equipment c i With node k The distance between them, α is the path decay constant, L ik Let c be a random variable; during the local computation phase, the unmanned device c i Calculate the relationship with node at regular intervals. k The communication speed between them is recorded and stored in the communication speed record queue. In this context, l represents the maximum length of the queue. When the actual number of elements in the queue is less than l, the communication speed to be recorded is directly stored at the end of the queue; otherwise, the first element at the head of the queue is deleted before being stored.

4. The method for three-dimensional inspection of dam surface defects using multiple unmanned vehicles in a cloud-edge collaborative environment as described in claim 1, characterized in that, In step 2), local model uploading and partial model aggregation are completed according to the model aggregation tree, and model parameters are uploaded to the edge server; the specific steps are as follows: In the r-th edge iteration, edge server e j The model aggregation tree of the group can be represented as e j ∪C j Let C be the set of nodes in the tree. j For edge server e j Subordinate unmanned equipment collection, R j To represent the node relationships in the model aggregation tree, some unmanned devices are represented as the leaf node set C in the model aggregation tree. leaf Edge server e j The root node is the model aggregation tree, and the remaining unmanned devices form the intermediate node set C. mid During the aggregation process, the local model of the unmanned device is uploaded and aggregated along each path from the leaf node to the root node. The roles of different types of nodes in the aggregation process are as follows: For unmanned equipment c at leaf nodes i ∈C leaf Simply upload the local model parameters to the parent node; For the unmanned equipment c at the intermediate node i ∈C mid It needs to wait for its child node set. After all unmanned devices upload their local model parameters, their own model parameters are then locally weighted and aggregated with the model parameters of the child unmanned devices: Among them, |D j |For unmanned equipment c j The number of images of dam defects collected. For edge server e j The total amount of data collected by all subordinate unmanned equipment. For the unmanned equipment c at the intermediate node after aggregation i The parameters of the local aggregation model, For unmanned equipment c j The local model parameters are uploaded from the intermediate node to the parent node after local aggregation. For the root node edge server e j Once all local models within the group have been aggregated and uploaded, edge model aggregation is triggered.

5. The method for three-dimensional inspection of dam surface defects using multiple unmanned vehicles in a cloud-edge collaborative environment as described in claim 1, characterized in that, The specific steps for the unmanned equipment to calculate communication metrics in step 2) are as follows: For unmanned equipment c i Records and nodes k Communication speed recording queue From the head of the team To the back of the line The order is according to Calculate communication metrics In the formula This is the smoothing coefficient.

6. The method for three-dimensional inspection of dam surface defects using multiple unmanned vehicles in a cloud-edge collaborative environment as described in claim 1, characterized in that, The specific steps of the edge server computing model aggregation tree in step 3) are as follows: Step 31, in the r-th edge iteration, define the communication metric matrix. Define a loop detection set list, Loop, with an initial length of |C|. j |+1, and each element in the list is an empty set, initialize the model aggregation tree. At this time R j It is an empty set; Step 32, search Largest element If c i With node k If they do not belong to the same set in the Loop, then c i With node k Add to model aggregation tree In the set of nodes, c i With node k The connection was added to Node association relationship R j If the condition is met, skip step 33 and proceed to step 34; otherwise, execute step 33. Step 33, merge c i With node k The two corresponding sets in the loop detection set list; Step 34, final settings Largest element Step 35: Repeat steps 32-34 until all nodes are added to the model aggregation tree. middle.

7. The method for three-dimensional inspection of dam surface defects using multiple unmanned vehicles in a cloud-edge collaborative environment as described in claim 1, characterized in that, The specific steps for the unmanned equipment to calculate the local data distribution metric in step 4) are as follows: For unmanned equipment c i The dataset of images of dam defects held by D i Its data distribution metric can be expressed as dis(D) i )={(c,n c )|c∈{1,2,...}}, where c is the data sample category, n c The number of data samples for category c.

8. The method for three-dimensional inspection of dam surface defects using multiple unmanned vehicles in a cloud-edge collaborative environment as described in claim 1, characterized in that, The specific steps for calculating the grouping results of unmanned equipment on the cloud server in step 5) are as follows: Step 51, initialize the grouping results for each edge server e. j ∈E, and its subordinate unmanned equipment set C j Let E be an empty string, and E be the set of edge servers. Define a matrix. C represents a collection of unmanned equipment; Step 52, for each unmanned device c in the set of unmanned devices C i ∈C, calculate its addition to each edge server e j After ∈E, the richness of data distribution within the edge server group In the formula p c Define a matrix to represent the probability of a class c data sample appearing in the dataset. Step 53, calculate the combined metric matrix In the formula, λ is used to control the proportion of communication energy consumption optimization. This is a regularization function used to map the values ​​of matrix elements to the range of 0-1; for the grouped unmanned equipment c i Combined metric matrix The value of the element in the i-th row is set to -1; Step 54, find the combined metric matrix The largest element u in the middle ij Unmanned equipment c i Add edge server e j C, a collection of subordinate unmanned equipment j In the middle, and mark the unmanned equipment c i Grouped; Repeat steps 52-54 until all unmanned devices have been grouped.

9. A three-dimensional inspection system for surface defects of dams using multiple unmanned vehicles in a cloud-edge collaborative environment, characterized in that, Includes the following modules: The local model training module allows each unmanned device in each edge server group to train a local dam defect detection model based on the dam defect image data it collects. In addition, the unmanned device records its communication speed with each edge server and other unmanned devices in the group at regular intervals and stores it in the communication speed recording queue. After the unmanned device completes a specified number of rounds of local model training, it triggers the local model upload. In the local model upload module, each unmanned device completes local model upload and local model aggregation according to the model aggregation tree generated in the previous edge aggregation stage. At the same time, each unmanned device calculates the communication metric with its own edge server and other unmanned devices in the group based on the communication speed recording queue, and uploads it directly to its own edge server. The edge model aggregation module involves the edge server performing weighted aggregation on the received local models to obtain the edge model. In addition, it calculates the model aggregation tree for the next edge iteration based on the communication metric matrix uploaded by the unmanned devices in the group. Subsequently, the edge server distributes the edge model and the model aggregation tree to each unmanned device in the group. The unmanned devices use the edge model to update their local models and enter the next edge iteration. After the edge iteration reaches a specified number, the edge model upload is triggered. The edge model upload module requires each edge server to upload its own edge model to the cloud server. At the same time, it requires each group of unmanned devices to calculate and upload communication metrics and local data distribution metrics, and forward the two metrics to the cloud server for calculating the grouping of unmanned devices in the next round of cloud iteration. The cloud model aggregation module performs weighted aggregation on the received edge models to obtain the cloud model. In addition, it calculates the grouping results of the unmanned devices based on the communication metrics and data distribution metrics of each unmanned device, and adjusts the federated network structure according to the grouping results. Finally, the cloud model is distributed to each edge server to enter the next round of cloud iteration. When the cloud model reaches the specified accuracy or the cloud iteration reaches the specified number of rounds, the final cloud model is the dam defect detection model.

10. A computer device, characterized in that, The computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the three-dimensional inspection method for surface defects of dams using multiple unmanned vehicles in a cloud-edge collaborative environment as described in any one of claims 1-7.

Images