Three-dimensional inspection method for surface defects of multi-unmanned-vehicle dam in cloud-edge collaborative environment

Abstract

The invention discloses a three-dimensional inspection method for surface defects of a multi-unmanned-vehicle dam in a cloud edge collaborative environment, and the method comprises the steps: training a dam defect detection model for unmanned equipment, introducing equipment index measurement, equipment grouping based on measurement and an intra-group collaborative aggregation technology, and dynamically adjusting the topological structure of a hierarchical federated network, thereby achieving the three-dimensional inspection of the surface defects of the multi-unmanned-vehicle dam. And the energy consumption required in the training process of the unmanned equipment is reduced while the defect detection model precision damage caused by inconsistent distribution of acquired data of different unmanned equipment is relieved. The equipment index measurement is used for reflecting data distribution of data collected by the unmanned equipment and the communication capability of the unmanned equipment, and the data distribution and the communication capability can be balanced to control the importance proportion of a certain part. The problem that communication energy consumption is too high when the unmanned equipment trains the dam defect detection model through hierarchical federal learning is solved, the precision of the detection model is fully ensured while energy consumption is reduced, and the dam defect three-dimensional inspection effect based on the unmanned equipment is further improved.

Description

Technical Field

[0001] The present invention relates to a method for three-dimensional inspection of surface defects of dams by multiple unmanned vehicles in a cloud-edge collaborative environment, belonging to the technical field of computer vision. Background Art

[0002] Traditional inspection of dam defects mainly relies on manual detection. This method has many disadvantages, such as being time-consuming, laborious, inefficient, and vulnerable to human factors, making it difficult to achieve efficient and accurate defect identification. To solve these problems, machine learning technology has been introduced to train dam defect detection models, improving the accuracy and efficiency of detection through automated analysis. With the wide application of unmanned devices (such as drones, intelligent sensors, etc.), data collection has become more convenient and comprehensive. However, the massive data collected by unmanned devices occupies a large amount of network resources during transmission, resulting in a significant increase in communication pressure. In addition, this data may contain sensitive information, making it difficult to gather the data to a unified center for centralized machine learning training. Therefore, a hierarchical federated learning framework can be introduced into the three-dimensional inspection of dam defects. Hierarchical federated learning collaboratively trains models among cloud servers, edge servers, and unmanned devices, and can effectively integrate the local data of each unmanned device on the premise of ensuring data privacy and security, achieving efficient defect identification and prediction. The hierarchical structure makes the model training process more flexible and has better scalability, capable of adapting to the complex topology and dynamic changes of the dam inspection network, significantly reducing communication overhead and computing latency. The three-dimensional inspection of surface defects of dams by multiple unmanned vehicles in a cloud-edge collaborative environment can improve the accuracy and real-time performance of defect detection, providing strong technical support for the intelligent management of water conservancy projects.

[0003] Although the three-dimensional inspection of surface defects of dams by multiple unmanned vehicles in the cloud-edge collaborative environment has significant advantages in improving detection accuracy and ensuring data privacy, it still faces some key problems in practical applications. First, the data collected by unmanned devices is heterogeneous. Due to differences in data collection methods, sensor types, and data quality among different devices, the accuracy of the dam defect detection model obtained through training is relatively low. This data heterogeneity causes the model to perform inconsistently on different devices, reducing the overall detection accuracy. Second, the communication conditions between unmanned devices and edge servers vary, manifested as differences in network bandwidth, latency, and connection stability, resulting in significant communication heterogeneity problems. Devices with poor communication conditions consume more energy during data transmission, increasing the energy consumption during the training process. Some unmanned devices, such as drones and unmanned vehicles, usually rely on limited battery power, and the energy consumption problem is particularly prominent in these resource-constrained unmanned devices. High energy consumption not only shortens the working time of the devices, limits the coverage and frequency of inspections, but also may cause premature damage to the devices, increasing the maintenance cost. Since dam inspection tasks often require long-term and high-frequency monitoring, the energy consumption problem directly affects the continuous operation ability and inspection efficiency of the entire system. Therefore, how to effectively reduce the energy consumption of unmanned devices while ensuring the accuracy of the defect detection model has become a key challenge that urgently needs to be solved for the three-dimensional inspection system of surface defects of dams by multiple unmanned vehicles in the cloud-edge collaborative environment. Summary of the Invention

[0004] Object of the Invention: Aiming at the problems that the data heterogeneity of unmanned devices and communication heterogeneity lead to the damage of the accuracy of the dam defect detection model obtained through model training in the cloud-edge collaborative environment and the large communication energy consumption during the training process, the present invention provides a method for three-dimensional inspection of surface defects of dams by multiple unmanned vehicles in the cloud-edge collaborative environment, which reduces the communication energy consumption brought to unmanned devices during the training process by adjusting the hierarchical federated network topology structure, thereby improving the accuracy of dam defect detection and the energy efficiency of the inspection process.

[0005] Technical Solution: A method for three-dimensional inspection of surface defects of dams by multiple unmanned vehicles in the cloud-edge collaborative environment. This method trains a dam defect detection model for unmanned devices based on a three-layer architecture of cloud, edge, and terminal, including local model training, local model uploading, edge-terminal model aggregation, edge-terminal model uploading, and cloud model aggregation stages. The dam defect image data is collected by unmanned devices, and the local model is trained on the unmanned devices according to the defect image data. Finally, the local models of the unmanned devices are hierarchically aggregated to obtain a global model, that is, the dam defect detection model. The method includes the following steps:

[0006] Step 1) In the local model training stage, each unmanned device in each edge server group trains a local dam defect detection model based on the dam defect image data collected by itself. In addition, the unmanned device records the communication speed between itself and each edge server and other unmanned devices in the group at regular time intervals and stores it in the communication speed record queue. After the unmanned device completes the specified number of rounds of local model training, it triggers the upload of the local model.

[0007] Step 2) In the local model upload stage, each unmanned device completes the upload of the local model and the aggregation of the local model according to the model aggregation tree generated in the previous edge aggregation stage. At the same time, each unmanned device calculates the communication metrics with the affiliated edge server and other unmanned devices in the group based on the communication speed record queue and directly uploads them to the affiliated edge server.

[0008] Step 3) In the edge model aggregation stage, the edge server performs weighted aggregation on the received local models to obtain the edge model. In addition, it calculates the model aggregation tree in the next edge iteration based on the communication metric matrix uploaded by the unmanned devices in the group. Subsequently, the edge server distributes the edge model and the model aggregation tree to each unmanned device in the group. The unmanned device uses the edge model to update the local model and enters the next edge iteration. After the edge iteration reaches the specified number of times, it triggers the upload of the edge model.

[0009] Step 4) In the edge model upload stage, each edge server uploads its own edge model to the cloud server. At the same time, it requires the unmanned devices in each group to calculate and upload the communication metrics and local data distribution metrics, and forwards the two metrics to the cloud server for calculating the grouping situation of the unmanned devices in the next cloud iteration.

[0010] Step 5) In the cloud model aggregation stage, the cloud server performs weighted aggregation on the received edge models to obtain the cloud model. In addition, it calculates the grouping results of the unmanned devices based on the communication metrics and data distribution metrics of each unmanned device, adjusts the federated network structure according to the grouping results, and finally distributes the cloud model to each edge server to enter the next cloud iteration. Repeat steps 1)-5) until the cloud model reaches the specified accuracy or the cloud iteration reaches the specified number of rounds, and finally obtain the dam defect detection model and send it to each unmanned device to perform the dam defect detection task.

[0011] Further, the specific steps for the unmanned device to train the local dam defect detection model in step 1) are as follows:

[0012] Unmanned device c i In the r-th local iteration, the local dam defect detection model is updated using stochastic gradient descent:

[0013]

[0014] Among them, are the local dam defect detection model parameters of the unmanned device c i in the (r + 1)-th and r-th local iterations respectively, η is the learning rate in stochastic gradient descent, is the local model in the r-th round calculated on the local dataset D i The gradient obtained. After the unmanned device executes the local iteration for the specified number of rounds using the above formula, it triggers the upload of the local model.

[0015] Furthermore, in step 1), the unmanned device records the communication speed between itself and each edge server and other unmanned devices in the group at regular time intervals. The specific steps are as follows:

[0016] Edge server e j Subordinate unmanned device c i The communication speed between and a certain node node k in the group can be expressed as:

[0017]

[0018] Where node k ∈ e j ∪ C j , C j is the set of unmanned devices subordinate to the edge server e j , B ik is the bandwidth allocated to the unmanned device c k by the node node i , p i is the upload power of the unmanned device c i , N 0 is the power spectral density, is the channel gain between the unmanned device c i and the node node k . In the formula, d ik is the distance between the unmanned device c i and the node node k , α is the path attenuation constant, L ik is a random variable; in the local calculation stage, the unmanned device c i calculates the communication speed with the node node k at regular time intervals and stores it in the communication speed record queue . l is the maximum length of the queue. When the actual number of elements in the queue is less than l, the communication speed to be recorded is directly stored at the end of the queue. Otherwise, the first element at the head of the queue is deleted and then stored.

[0019] Furthermore, in step 2), local model uploading and local model aggregation are completed according to the model aggregation tree, and model parameters are uploaded to the edge server; the specific steps are as follows:

[0020] In the rth edge iteration, the edge server e j The model aggregation tree of the group can be expressed as e j ∪C j is the set of nodes in the tree, C j For edge servers j Subordinate unmanned equipment collection, R j is the node association relationship in the model aggregation tree, and some unmanned equipment is the leaf node set C in the model aggregation tree leaf , edge server j is the root node of the model aggregation tree, and the remaining unmanned devices form the intermediate node set C mid During the aggregation process, the local model of the unmanned equipment is uploaded and aggregated along each path from the leaf node to the root node. The roles of different types of nodes in the aggregation process are as follows:

[0021] For leaf nodes with unmanned devices c i ∈C leaf , only need to upload local model parameters to the parent node;

[0022] For unmanned devices at intermediate nodes c i ∈C mid , need to wait for its child node set After all unmanned devices upload their local model parameters, they perform local weighted aggregation on their own model parameters and the model parameters of the unmanned devices of the child nodes:

[0023]

[0024] Among them, |D j | For unmanned equipment c j The number of dam defect image data collected, For edge servers j The total amount of data collected by all subordinate unmanned equipment, c is the unmanned device at the intermediate node after aggregation i The local aggregation model parameters, For unmanned equipment j After local aggregation, the intermediate node uploads the local aggregated model parameters to the parent node;

[0025] For the root edge server e j ,After all local models are aggregated and uploaded in the group, the edge model aggregation is triggered.

[0026] Further, the specific steps for the unmanned device to calculate the communication metric in step 2) are as follows:

[0027] For unmanned device c i The recorded communication speed record queue with node k According to the order from the head of the queue to the end of the queue calculate the communication metric where is the smoothing coefficient.

[0028] Further, the specific steps for the edge server to calculate the model aggregation tree in step 3) are as follows:

[0029] Step 31, in the r-th round of edge iteration, define the communication metric matrix Define the loop detection set list Loop, the initial length of the list is |C j | + 1, and each element in the list is an empty set, initialize the model aggregation tree At this time, R j is an empty set;

[0030] Step 32, find the largest element in If c i and node k do not belong to the same set in Loop, then add c i and node k to the node set of the model aggregation tree add the connection between c i and node k to the node association relationship R j in, skip step 33 and enter step 34; otherwise, execute step 33;

[0031] Step 33, merge the two sets corresponding to c i and node k in the loop detection set list;

[0032] Step 34, finally set the largest element in

[0033] Step 35, repeat steps 32 - 34 until all nodes are added to the model aggregation tree

[0034] Further, the specific steps for the unmanned device to calculate the local data distribution metric in step 4) are as follows:

[0035] ​​​For the unmanned device c i The dataset D of dam defect images held i , and its data distribution metric can be expressed as dis(D i ) = {(c, n c )|c ∈ {1, 2,...}}, where c is the data sample category and n c is the number of data samples in category c.

[0036] Furthermore, the specific steps for calculating the grouping result of unmanned devices in the cloud server in step 5) are as follows:

[0037] Step 51, initialize the grouping result. For each edge server e j ∈ E, the set C of its subordinate unmanned devices is j empty. E is the set of edge servers. Define the matrix C as the set of unmanned devices;

[0038] Step 52, for each unmanned device c i ∈ C in the set C of unmanned devices, calculate the richness of data distribution within the edge server group after it joins each edge server e j ∈ E where p c is the occurrence probability of data samples of class c in the dataset. Define the matrix

[0039] Step 53, calculate the combined metric matrix where λ is used to control the optimization ratio of communication energy consumption, is the regularization function, which is used to map the numerical values of matrix elements to the range of 0 - 1; for the already grouped unmanned device c i , set the element values of the i-th row of the combined metric matrix to -1;

[0040] Step 54, find the maximum element u in the combined metric matrix ij , add the unmanned device c i to the set C of subordinate unmanned devices of the edge server e j , and mark the unmanned device c j as grouped; i

[0041] Repeat the process of steps 52 - 54 until all unmanned devices are grouped.

[0042] A multi-unmanned vehicle three-dimensional inspection system for dam surface defects in a cloud-edge collaborative environment includes the following modules:

[0043] ​Local model training module. Each unmanned device in each edge server group trains a local dam defect detection model based on the dam defect image data collected by itself. In addition, each unmanned device records the communication speed between itself and each edge server and other unmanned devices in the group at regular time intervals and stores it in the communication speed record queue. After the unmanned device completes the specified number of rounds of local model training, it triggers the upload of the local model.

[0044] Local model upload module. Each unmanned device completes the upload of the local model and the aggregation of the local models according to the model aggregation tree generated in the previous edge aggregation stage. At the same time, each unmanned device calculates the communication metrics with the edge server it belongs to and other unmanned devices in the group based on the communication speed record queue and directly uploads them to the edge server it belongs to.

[0045] Edge-side model aggregation module. The edge server performs weighted aggregation on the received local models to obtain the edge-side model. In addition, it calculates the model aggregation tree in the next edge-side iteration based on the communication metric matrix uploaded by the unmanned devices in the group. Subsequently, the edge server distributes the edge-side model and the model aggregation tree to each unmanned device in the group. The unmanned device uses the edge-side model to update the local model and enters the next edge-side iteration. After the edge-side iteration reaches the specified number of times, it triggers the upload of the edge-side model.

[0046] Edge-side model upload module. Each edge server uploads its own edge-side model to the cloud server. At the same time, it requires the unmanned devices in each group to calculate and upload the communication metrics and local data distribution metrics, and forwards the two metrics to the cloud server for calculating the grouping situation of the unmanned devices in the next cloud iteration.

[0047] Cloud model aggregation module. The cloud server performs weighted aggregation on the received edge-side models to obtain the cloud model. In addition, it calculates the grouping results of the unmanned devices based on the communication metrics and data distribution metrics of each unmanned device, adjusts the federated network structure according to the grouping results, and finally distributes the cloud model to each edge server to enter the next cloud iteration. When the cloud model reaches the specified accuracy or the cloud iteration reaches the specified number of rounds, the finally obtained cloud model is the dam defect detection model.

[0048] The specific implementation process and method of the system are the same and will not be elaborated here.

[0049] A computer device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the above computer program, it implements the method for three-dimensional inspection of the surface defects of the dam by multiple unmanned vehicles in the cloud-edge collaborative environment as described above.

[0050] Beneficial effects: Compared with the prior art, the method for three-dimensional inspection of surface defects of dams by multiple unmanned vehicles in a cloud-edge collaborative environment combines device metric measurement, metric-based device grouping, and intra-group collaboration aggregation technologies, reduces the communication energy consumption brought to unmanned devices during the training process while ensuring the accuracy of the dam defect detection model, and optimizes the energy efficiency of the inspection system. Device metric measurement takes into account both the communication capabilities of devices and the characteristics of local data distribution, and the two can be weighed according to different scenarios; metric-based device grouping is used to adjust the network topology between edge servers and unmanned devices, control the differences between the data distribution within different edge server groups and the global data distribution while improving the overall communication rate of the system, thereby ensuring the final accuracy of the dam defect detection model and reducing the communication energy consumption of unmanned devices; intra-group collaboration aggregation allows communication between different unmanned devices, and further reduces the overall communication energy consumption of the system by adjusting the communication network structure within the edge server group. The present invention solves the problems that unmanned devices occupy excessive network resources and there is a risk of privacy leakage when uploading the defect image samples they collect, alleviates the problem of the decline in the accuracy of the dam defect detection model caused by the distribution differences in data collected by unmanned devices, and solves the problem of excessive energy consumption during the training process due to the differences in network bandwidth, latency, and connection stability of unmanned devices. Description of the Drawings

[0051] Figure 1 It is a flowchart of the method according to an embodiment of the present invention. Detailed Embodiments

[0052] The following further clarifies the present invention in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and not to limit the scope of the present invention. After reading the present invention, various equivalent modifications of the present invention by those skilled in the art fall within the scope defined by the appended claims of this application.

[0053] The method for three-dimensional inspection of surface defects of dams by multiple unmanned vehicles in a cloud-edge collaborative environment combines device metric measurement, metric-based device grouping, and intra-group collaboration aggregation technologies to adjust the topology of the federated training network, which can reduce the communication energy consumption of unmanned devices on the basis of ensuring the accuracy of the dam surface defect detection model, thereby extending the service life of unmanned devices and optimizing the energy efficiency of the inspection system.

[0054] As Figure 1 shown, the method for three-dimensional inspection of surface defects of dams by multiple unmanned vehicles in a cloud-edge collaborative environment includes the following steps:

[0055] Step 1) In the local model training stage, the edge server e j intra-group unmanned device c i updates the local dam defect detection model using stochastic gradient descent based on the dam defect image data collected by itself in the r-th local iteration:

[0056]

[0057] Among them, are the local dam defect detection model parameters of the unmanned device c i in the (r + 1)-th and r-th local iterations respectively, η is the learning rate in stochastic gradient descent, is the local model in the r-th round calculated on the local dataset D i ; in addition, the unmanned device c i records its communication speed with a certain node node k in the group at a certain time interval according to the following formula:

[0058]

[0059] where node k ∈e j ∪C j , C j is the set of unmanned devices subordinate to the edge server e j , B ik is the bandwidth allocated to the unmanned device c k by the node node i , p i is the upload power of the unmanned device c i , N 0 is the power spectral density, is the channel gain between the unmanned device c i and the node node k ; in the formula, d ik is the distance between the unmanned device c i and the node node k , α is the path attenuation constant, L ik is a random variable; in the local calculation stage, the unmanned device c i calculates the communication speed with the node node k at a certain time interval and stores it in the communication speed record queue ; l is the maximum length of the queue. When the actual number of elements in the queue is less than l, the communication speed to be recorded is directly stored at the end of the queue; otherwise, the first element at the head of the queue is deleted and then stored. After the unmanned device performs the local iteration of the specified round, the local model upload is triggered;

[0060] Step 2) In the local model upload stage, each unmanned device completes the local model upload and local model aggregation according to the model aggregation tree generated in the previous round of edge aggregation stage. In the r-th edge iteration, the model aggregation tree of the group where the edge server e j is located can be expressed as e j ∪C j is the set of nodes in the tree, and C j is the set of unmanned devices subordinate to the edge server e j and R j is the node association relationship in the model aggregation tree. Some unmanned devices are used as the set of leaf nodes C leaf in the model aggregation tree, and the edge server e j is the root node of the model aggregation tree, and the remaining unmanned devices form the set of intermediate nodes C mid During the aggregation process, the local models of unmanned devices are uploaded and aggregated along each path from the leaf node to the root node. The roles of different types of nodes in the aggregation process are as follows:

[0061] For the leaf node unmanned device c i ∈C leaf , only the local model parameters need to be uploaded to the parent node;

[0062] For the intermediate node unmanned device c i ∈C mid , it is necessary to wait for all unmanned devices in its child node set to upload their local model parameters, and then perform local weighted aggregation of its own model parameters with the model parameters of the child node unmanned devices:

[0063]

[0064] where |D j | is the number of dam defect image data collected by the unmanned device c j , is the total number of data collected by all unmanned devices subordinate to the edge server e j , is the local aggregation model parameter of the intermediate node unmanned device c i after aggregation, is the local model parameter of the unmanned device c j . After local aggregation, the intermediate node uploads the local aggregation model parameter to the parent node; at the same time, each unmanned device c i ∈ j calculates the communication metric k based on the communication speed record queue with the node node in the order from the head to the tail of the queue according to ; In the formula is the smoothing coefficient, and then the calculated communication metric is directly uploaded to its affiliated edge server e j ;

[0065] Step 3) In the edge model aggregation stage, the edge server performs weighted aggregation on the received local models to obtain the edge model. In addition, based on the communication metric matrix uploaded by the devices without a person in the group, the model aggregation tree in the next round of edge iteration is calculated:

[0066] In the r-th round of edge iteration, the communication metric matrix is defined Define the loop detection set list Loop, the initial length of the list is |C j | + 1, and each element in the list is an empty set, and the model aggregation tree is initialized At this time, R j is an empty set;

[0067] Search for the largest element in If c i and node k do not belong to the same set in Loop, then add c i and node k to the node set of the model aggregation tree , add the connection between c i and node k to 's node association relationship R j , and at the same time merge the two sets corresponding to c i and node k in the loop detection set list; finally set the largest element in Repeat this process until all nodes are added to the model aggregation tree . Subsequently, the edge server distributes the edge model and the model aggregation tree to each device without a person in the group. The device without a person uses the edge model to update the local model and enters the next edge iteration; after the edge iteration reaches the specified number of times, the edge model upload is triggered;

[0068] Step 4) In the edge model upload stage, each edge server uploads its own edge model to the cloud server. At the same time, it requires the devices without a person in each group to calculate and upload the communication metric and the local data distribution metric. For the dam defect image dataset D i held by the device without a person c i , its data distribution metric can be expressed as dis(D i ) = {(c, n c )|c ∈ {1, 2,...}}, where c is the data sample category, n cis the number of data samples of category c; after the edge server collects the communication metrics and local data distribution metrics of the subordinate unmanned devices, it forwards and uploads them to the cloud server for calculating the grouping situation of the unmanned devices in the next round of cloud iteration;

[0069] Step 5) In the cloud model aggregation stage, the cloud server performs weighted aggregation on the received edge models to obtain the cloud model. In addition, it calculates the unmanned device grouping result based on the communication metrics and data distribution metrics of each unmanned device:

[0070] First, initialize the grouping result. For each edge server e j ∈E, the set C j of its subordinate unmanned devices is empty. E is the set of edge servers. Define the matrix C as the set of unmanned devices;

[0071] For each unmanned device c i ∈C in the set C of unmanned devices, calculate the richness of data distribution within the edge server group after it joins each edge server e j ∈E. In the formula, p c is the occurrence probability of data samples of category c in the dataset. Define the matrix

[0072] Calculate the combined metric matrix In the formula, λ is used to control the optimization proportion of communication energy consumption, is the regularization function, which is used to map the numerical values of matrix elements to the range of 0 - 1; for the grouped unmanned device c i , set the element values of the i-th row of the combined metric matrix to -1;

[0073] Find the maximum element u in the combined metric matrix ij , add the unmanned device c i to the set C j of the subordinate unmanned devices of the edge server e j , and mark the unmanned device c i as grouped; repeat the above steps until all unmanned devices are grouped. Finally, obtain the unmanned device grouping result for the next round of cloud iteration, and adjust the federated network structure according to the grouping result; finally, the cloud server sends the cloud model to each edge server and enters the next round of cloud iteration; repeat steps 1) - 5) until the cloud model reaches the specified accuracy or the cloud iteration reaches the specified number of rounds. Finally, obtain the dam defect detection model and send it to each unmanned device for performing the dam defect detection task.

[0074] Obviously, those skilled in the art should understand that the above-described method for three-dimensional inspection of surface defects of dams by multiple unmanned vehicles in a cloud-edge collaborative environment of the present invention can be implemented using a general-purpose computing device. They can be concentrated on a single computing device or distributed over a network composed of multiple computing devices. Optionally, they can be implemented using program code executable by the computing device. Thus, they can be stored in a storage device and executed by the computing device. And in some cases, the steps shown or described can be executed in a sequence different from that here, or they can be separately fabricated into individual integrated circuit modules, or multiple modules or steps among them can be fabricated into a single integrated circuit module for implementation. In this way, the embodiments of the present invention are not limited to any specific combination of hardware and software.

Claims

1. A three-dimensional inspection method for dam surface defects using multiple unmanned vehicles in a cloud-edge collaborative environment, characterized in that: It includes local model training, local model upload, edge model aggregation, edge model upload and cloud model aggregation stages, including the following steps: Step 1) In the local model training stage, each unmanned device in each edge server group trains a local dam defect detection model based on the dam defect image data collected by itself. In addition, the unmanned device records the communication speed between itself and each edge server and other unmanned devices in the group at a certain time interval, and stores it in the communication speed record queue; after the unmanned device completes the specified number of rounds of local model training, it triggers the local model upload; Step 2) In the local model upload phase, each unmanned device completes local model upload and local model aggregation according to the model aggregation tree generated in the previous round of edge aggregation phase. At the same time, each unmanned device calculates the communication metrics with the edge server to which it belongs and other unmanned devices in the group based on the communication speed record queue, and uploads it directly to the edge server to which it belongs; Step 3) In the edge model aggregation phase, the edge server performs weighted aggregation on the received local models to obtain the edge model. In addition, the model aggregation tree in the next round of edge iteration is calculated based on the communication metric matrix uploaded by the unmanned devices in the group. The edge server then sends the edge model and the model aggregation tree to each unmanned device in the group. The unmanned device uses the edge model to update the local model and enters the next edge iteration. When the edge iteration reaches the specified number of times, the edge model upload is triggered; Step 4) In the edge model upload phase, each edge server uploads its own edge model to the cloud server, and requires the unmanned devices in each group to calculate and upload the communication metric and the local data distribution metric, and forward the two metrics to the cloud server for calculating the grouping of unmanned devices in the next round of cloud iteration; Step 5) In the cloud model aggregation stage, the cloud server performs weighted aggregation on the received edge models to obtain the cloud model. In addition, the unmanned device grouping results are calculated based on the communication metrics and data distribution metrics of each unmanned device, and the federal network structure is adjusted according to the grouping results. Finally, the cloud model is sent to each edge server to enter the next round of cloud iteration; repeat steps 1)-step 5) until the cloud model reaches the specified accuracy or the cloud iteration reaches the specified round, and finally the dam defect detection model is obtained and sent to each unmanned device for performing the dam defect detection task.

2. According to the method for three-dimensional inspection of dam surface defects by multiple unmanned vehicles in a cloud-edge collaborative environment according to claim 1, it is characterized in that: The specific steps of unmanned equipment training the local dam defect detection model in step 1) are as follows: Unmanned equipment i In the rth round of local iteration, stochastic gradient descent is used to update the local dam defect detection model: in, They are unmanned equipment c i The parameters of the local dam defect detection model in the r+1th round and the rth round of local iteration, η is the learning rate in the stochastic gradient descent, is the local model of round r In the local dataset D i The unmanned device uses the above formula to perform a specified number of local iterations and triggers the upload of the local model.

3. The method for three-dimensional inspection of dam surface defects by multiple unmanned vehicles in a cloud-edge collaborative environment according to claim 1 is characterized in that: In step 1), the unmanned device records the communication speed between itself and each edge server and other unmanned devices in the group at regular intervals. The specific steps are as follows: Edge Server j Subordinate unmanned equipment c i With a node in the group k The communication speed between can be expressed as: where node k ∈e j ∪C j , C j For edge servers j Subordinate unmanned equipment collection, B ik For node k Assigned to unmanned equipment c i The bandwidth, p i For unmanned equipment i The upload power, N0 is the power spectrum density, For unmanned equipment i With node k The channel gain between ik For unmanned equipment i With node k The distance between them, α is the path attenuation constant, L ik is a random variable; in the local computing stage, the unmanned device c i Calculate the node at regular intervals k The communication speed between the two nodes is recorded and stored in the communication speed record queue. In the figure, l is the maximum length of the queue. When the actual number of elements in the queue is less than l, the communication speed to be recorded is directly stored at the end of the queue. Otherwise, the first element at the head of the queue is deleted and then stored.

4. The method for three-dimensional inspection of dam surface defects by multiple unmanned vehicles in a cloud-edge collaborative environment according to claim 1 is characterized in that: In step 2), local model uploading and local model aggregation are completed according to the model aggregation tree, and model parameters are uploaded to the edge server; the specific steps are as follows: In the rth edge iteration, the edge server e j The model aggregation tree of the group can be expressed as e j ∪C j is the set of nodes in the tree, C j For edge servers j Subordinate unmanned equipment collection, R j is the node association relationship in the model aggregation tree, and some unmanned equipment is the leaf node set C in the model aggregation tree leaf , edge server j is the root node of the model aggregation tree, and the remaining unmanned devices form the intermediate node set C mid In the aggregation process, the local model of the unmanned equipment is uploaded and aggregated along each path from the leaf node to the root node. The roles of different types of nodes in the aggregation process are as follows: For leaf nodes with unmanned devices c i ∈C leaf , only need to upload local model parameters to the parent node; For unmanned devices at intermediate nodes c i ∈C mid , need to wait for its child node set After all unmanned devices upload their local model parameters, they perform local weighted aggregation on their own model parameters and the model parameters of the unmanned devices of the child nodes: Among them, |D j | For unmanned equipment c j The number of dam defect image data collected, For edge servers j The total amount of data collected by all subordinate unmanned equipment, c is the unmanned device at the intermediate node after aggregation i The local aggregation model parameters, For unmanned equipment j After local aggregation, the intermediate node uploads the local aggregated model parameters to the parent node; For the root edge server e j ,After all local models are aggregated and uploaded in the group, the edge model aggregation is triggered.

5. The method for three-dimensional inspection of dam surface defects by multiple unmanned vehicles in a cloud-edge collaborative environment according to claim 1 is characterized in that: The specific steps of calculating the communication metric of the unmanned device in step 2) are as follows: For unmanned equipment i Recorded with node node k The communication speed record queue Press from the team leader To the end of the line The order is based on Calculating communication metrics In the formula is the smoothing coefficient.

6. The method for three-dimensional inspection of dam surface defects by multiple unmanned vehicles in a cloud-edge collaborative environment according to claim 1 is characterized in that: The specific steps of edge server computing model aggregation tree in step 3) are as follows: Step 31, in the rth edge iteration, define the communication metric matrix Define the loop detection set list Loop, the initial length of the list is |C j |+1, and each element in the list is an empty set, initializing the model aggregation tree At this time R j is an empty collection; Step 32, search The largest element in If c i With node k If they do not belong to the same set in Loop, c i With node k Join model aggregation tree In the node set of i With node k Add the connection to The node association relationship R j , skip step 33 and proceed to step 34; otherwise, execute step 33; Step 33, merge c i With node k The two corresponding sets in the loop detection set list; Step 34, Final Settings The largest element in Step 35, repeat steps 32-34 until all nodes are added to the model aggregation tree middle.

7. The method for three-dimensional inspection of dam surface defects by multiple unmanned vehicles in a cloud-edge collaborative environment according to claim 1 is characterized in that: The specific steps for the unmanned device to calculate the local data distribution metric in step 4) are as follows: For unmanned equipment i The dam defect image dataset D i , its data distribution measure can be expressed as dis(D i )={(c,n c )|c∈{1,2,...}}, where c is the data sample category, n c is the number of data samples of category c.

8. The method for three-dimensional inspection of dam surface defects by multiple unmanned vehicles in a cloud-edge collaborative environment according to claim 1 is characterized in that: The specific steps of calculating the unmanned device grouping results on the cloud server in step 5) are as follows: Step 51, initialization grouping results, for each edge server e j ∈E, its subordinate unmanned equipment set C j is empty, E is the edge server set, and the matrix is ​​defined C is a collection of unmanned equipment; Step 52: for each unmanned device c in the unmanned device set C i ∈C, calculate its addition to each edge server e j ∈E, the data distribution richness within the edge server group Where p c For the probability of occurrence of class c data samples in the data set, define the matrix Step 53, calculate the combined metric matrix In the formula, λ is used to control the optimization ratio of communication energy consumption. is a regularization function used to map the values ​​of matrix elements to the range of 0-1; for the grouped unmanned devices c i , the combined metric matrix The value of the element in the i-th row is set to -1; Step 54, find the combined metric matrix The largest element u in ij , unmanned equipment c i Add edge server j The subordinate unmanned equipment set C j and mark unmanned equipment c i Grouped; Repeat steps 52-54 until all unmanned devices have completed grouping.

9. A three-dimensional inspection system for dam surface defects using multiple unmanned vehicles in a cloud-edge collaborative environment, characterized in that: Includes the following modules: Local model training module: each unmanned device in each edge server group trains a local dam defect detection model based on the dam defect image data collected by itself. In addition, the unmanned device records the communication speed between itself and each edge server and other unmanned devices in the group at a certain time interval and stores it in the communication speed record queue; After the unmanned device completes the specified number of rounds of local model training, it triggers the upload of the local model; In the local model upload module, each unmanned device completes local model upload and local model aggregation according to the model aggregation tree generated in the previous round of edge aggregation stage. At the same time, each unmanned device calculates the communication metrics with its edge server and other unmanned devices in the group based on the communication speed record queue, and uploads it directly to its edge server; Edge model aggregation module: The edge server performs weighted aggregation on the received local models to obtain the edge model. In addition, the model aggregation tree in the next round of edge iteration is calculated based on the communication metric matrix uploaded by the unmanned devices in the group. The edge server then sends the edge model and the model aggregation tree to each unmanned device in the group. The unmanned device uses the edge model to update the local model and enter the next edge iteration. When the edge iteration reaches the specified number of times, the edge model upload is triggered; In the edge model upload module, each edge server uploads its own edge model to the cloud server, and requires the unmanned devices in each group to calculate and upload the communication metrics and local data distribution metrics, and forward the two metrics to the cloud server for calculating the grouping of unmanned devices in the next round of cloud iteration; In the cloud model aggregation module, the cloud server performs weighted aggregation on the received edge models to obtain the cloud model. In addition, the unmanned device grouping results are calculated based on the communication metrics and data distribution metrics of each unmanned device, and the federal network structure is adjusted according to the grouping results. Finally, the cloud model is sent to each edge server to enter the next round of cloud iteration; when the cloud model reaches the specified accuracy or the cloud iteration reaches the specified round, the final cloud model is the dam defect detection model.

10. A computer device, characterized in that: The computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, a three-dimensional inspection method for dam surface defects using multiple unmanned vehicles in a cloud-edge collaborative environment is implemented as described in any one of claims 1 to 7.

Images