A Method and System for Updating Defect Identification Models for Unmanned Aerial Vehicle Inspection Based on Federated Learning

Abstract

This invention discloses a method and system for updating a defect identification model for UAV inspection based on federated learning, relating to the field of UAV inspection technology. The method includes: acquiring performance evaluation information of a local model in a local scenario; transmitting local identification model parameters, scene description information of the current inspection environment, and performance evaluation information to a central server; classifying each UAV based on the scene description information and evaluating the activity of the local model; and weighting and aggregating the local identification model parameters based on the performance evaluation information to obtain a global model; responding to a UAV's model update request, adjusting the parameters of the global model based on the UAV's scene description information to generate a personalized adaptation model and distributing it to the corresponding UAV. This application avoids the problem of diluted professional knowledge in specific scenarios caused by indiscriminate aggregation, achieves on-demand customization in the model distribution stage, and improves the model's recognition accuracy and adaptability in complex and changing inspection scenarios.

Description

Technical Field

[0001] This application relates to the field of drone inspection technology, specifically to a method and system for updating a drone inspection defect identification model based on federated learning. Background Technology

[0002] To further enhance the intelligence level of drone inspections, deep learning-based automatic defect identification technology has become crucial. However, the performance of these identification models heavily relies on a large amount of high-quality defect data. In practice, this data is often scattered across various inspection terminals and contains sensitive geographical location and asset information. Uploading this raw dataset centrally to the cloud for model training poses serious risks of data privacy breaches and security compliance. To achieve continuous optimization and iteration of model performance while protecting data privacy, federated learning, an innovative distributed machine learning method, has emerged. It allows various terminal devices to collaboratively train a global model without directly sharing raw data. Although federated learning provides a feasible technical path to solve data privacy issues, some challenges remain in practical applications. Traditional federated averaging methods typically aggregate parameters indiscriminately across all participating terminal device models, ignoring the significant differences that may exist between different drones inspecting equipment types, environmental conditions (e.g., lighting, weather), and defect type distributions. This "one-size-fits-all" aggregation strategy can impair the model's adaptability and recognition accuracy in different specific scenarios, making it difficult for the updated global model to achieve optimal performance on all terminals and limiting the model's ability to generalize in real-world, complex, and ever-changing inspection tasks. Summary of the Invention

[0003] This application addresses the issues of low adaptability and recognition accuracy of UAV inspection defect recognition models in specific scenarios due to data security risks and indiscriminate model parameter aggregation strategies during the updating process of UAV inspection defect recognition models based on federated learning. It proposes a method and system for updating UAV inspection defect recognition models based on federated learning. By classifying the UAV's decision branches according to the scenario and obtaining model parameters based on its performance in that scenario, the model is then personalized according to inspection needs. This ensures that the UAV acquires a customized recognition capability that includes general knowledge and is highly suited to its specific working environment, thereby significantly improving the model's recognition accuracy and adaptability in diverse inspection scenarios while protecting data privacy.

[0004] To achieve the above objectives, the technical solutions adopted in the embodiments of this application are as follows: In a first aspect, embodiments of this application provide a method for updating a defect identification model for UAV inspection based on federated learning. The method includes: acquiring performance evaluation information of a local identification model in a local scenario; transmitting local identification model parameters, scenario description information of the current inspection environment, and performance evaluation information to a central server; classifying each UAV according to the scenario description information and evaluating the activity of the local model, and weighting and aggregating the local identification model parameters in conjunction with the performance evaluation information to obtain a global model; responding to a UAV's model update request, adjusting the parameters of the global model according to the UAV's scenario description information, generating a personalized adaptation model, and distributing it to the corresponding UAV.

[0005] In this solution, performance evaluation information of the local recognition model in the local scenario is obtained, and the local recognition model parameters, scene description information of the current inspection environment, and performance evaluation information are transmitted to the central server. This allows the central server to understand the specific operating environment and model performance of each UAV. By classifying each UAV in the scene description information on the central server, and then weighting and aggregating the local recognition model parameters based on the scene classification results and performance evaluation information, the problem of diluted professional knowledge in specific scenarios caused by traditional indiscriminate aggregation is avoided. This allows the generated global model to integrate specialized recognition capabilities across different scenarios. In response to UAV model update requests, the global model parameters are adjusted according to the UAV's scene description information to generate personalized adapted models and distribute them to the corresponding UAVs. This achieves "on-demand customization" in the model distribution stage, ensuring that each UAV receives a recognition model highly suited to its current inspection environment. Thus, without sharing the original data, the recognition accuracy and adaptability of the model in complex and changing inspection scenarios are significantly improved.

[0006] Optionally, the step of obtaining the performance evaluation information of the local recognition model in the local scene and transmitting the local recognition model parameters, the scene description information of the current inspection environment, and the performance evaluation information to the central server includes: obtaining the environmental feature information of the UAV when performing the inspection task; identifying and dividing different environmental units according to the changes in the environmental feature information; generating a corresponding scene feature description for each environmental unit and associating the defect image data collected in the environmental unit with the scene feature description; training the corresponding decision branch in the UAV local defect recognition system according to the current scene feature description, adjusting the parameters through the defect image data associated with the current scene feature description, and obtaining the parameter update amount of the decision branch; calculating the accuracy, recall, and F1 score of the parameter-adjusted decision branch based on the subset in the UAV local test dataset that matches the current environmental unit, and generating a scene-specific performance evaluation summary; and encapsulating the parameter update amount of each scene-specific adjusted decision branch, the corresponding scene feature description, and the performance evaluation summary into an independent knowledge module and transmitting it to the central server.

[0007] Optionally, the step of classifying each UAV according to scene description information and evaluating the activity of the local model, and combining performance evaluation information to perform weighted aggregation of the local identification model parameters to obtain a global model includes: the central server decrypting the received knowledge module and creating a virtual knowledge agent, the agent attributes of which include at least the physical UAV ID, environmental unit characteristics, decision branch type, and scene-specific performance indicators; dividing all virtual knowledge agents into different scene professional groups according to the environmental unit characteristics of the virtual knowledge agents; and within each scene professional group, performing weighted aggregation of the update amount of decision branch parameters from different virtual knowledge agents to generate a global model.

[0008] Optionally, within each scenario-specific group, the weighted aggregation of decision branch parameter updates from different virtual knowledge agents to generate a global model includes: evaluating the activity of virtual knowledge agents in each scenario category based on the frequency and number of knowledge modules uploaded by virtual knowledge agents in a recent period; identifying environmental features corresponding to scenario categories with activity levels below a preset threshold, and selecting general decision branch parameters matching the environmental features from a preset general knowledge reserve as supplementary knowledge; retrieving the scenario-specific accuracy of each virtual knowledge agent in identifying specific defects; dynamically adjusting the aggregation weight of virtual knowledge agents based on the activity evaluation results and scenario-specific accuracy, and combining the supplementary knowledge to weighted aggregate the decision branch parameter updates from different virtual knowledge agents to generate a global decision branch; and integrating the global decision branch with a shared general feature extraction backbone network to form the global model.

[0009] Optionally, the step of dynamically adjusting the aggregation weight of the virtual knowledge agent based on the activity assessment result and the scene-specific accuracy includes: comparing the activity assessment result with the historical activity data of the virtual knowledge agent, and evaluating the degree of matching between the scene-specific accuracy and the scene feature description of the virtual knowledge agent and the amount of local test data to identify abnormal assessment results; the central server verifies the model parameters uploaded by the drone in response to the abnormal assessment result; obtains the scene-specific accuracy after parameter verification, and obtains a confidence factor based on the degree of abnormality of the abnormal assessment result and the parameter verification result; identifies the nonlinear correlation pattern between the confidence factor, the activity assessment result, and the scene-specific accuracy, and dynamically adjusts the weight mapping function according to the nonlinear correlation pattern; and uses the weight mapping function to map the activity assessment result, the scene-specific accuracy, and the confidence factor into corrected aggregation weights.

[0010] Optionally, identifying the nonlinear correlation pattern between the confidence factor, the activity assessment result, and the scene-specific accuracy, and dynamically adjusting the weight mapping function according to the nonlinear correlation pattern, includes: the central server dividing virtual knowledge agents into multiple scene-specific groups based on the scene feature descriptions of each virtual knowledge agent, with each group corresponding to an inspection area or device type; obtaining the nonlinear correlation pattern of the scene-specific group based on the interaction relationship between the activity assessment result, the scene-specific accuracy, and the confidence factor within each group; continuously monitoring the real-time changing trends of the activity assessment result, scene-specific accuracy, and confidence factor within each scene-specific group; comparing the real-time changing trends with the historical correlation patterns of the scene-specific group to identify any deviations from the real-time changing trends and the historical correlation patterns exceeding a threshold, or any new combination patterns that have not been recorded; updating the nonlinear correlation pattern of the scene-specific group in response to the new combination pattern, and adjusting the weight mapping function according to the updated nonlinear correlation pattern.

[0011] This solution completely abandons the traditional indiscriminate averaging aggregation method by decomposing the professional knowledge of each UAV in different micro-environments into independent virtual knowledge agents and clustering them according to scene characteristics. First, it quantifies the timeliness and persistence of each agent's knowledge contribution based on activity assessment with time decay. It automatically supplements general knowledge for low-activity scenes to solve the problem of knowledge scarcity in low-frequency scenes. Then, it ensures the quality of knowledge participating in aggregation by accurately filtering and correcting abnormal assessment results and erroneous model parameters through anomaly identification and confidence factor. At the same time, it dynamically adapts to different inspection areas and equipment types based on scene-specific nonlinear association patterns. By understanding the inherent dynamics of activity, accuracy, and confidence, a nonlinear weighting mapping function is used to give higher weights to knowledge modules with high scene adaptability, high recognition accuracy, and high contribution credibility in the aggregation. This avoids the dilution of high-quality scene-specific knowledge by general knowledge and ensures that the global model can fully integrate the specialized defect recognition capabilities in various complex inspection scenarios. This overcomes the problems of poor model scene adaptability, impaired recognition accuracy, and uneven terminal performance caused by traditional "one-size-fits-all" aggregation, and significantly improves the generalization ability and optimal performance adaptability of the global model in complex and ever-changing UAV inspection tasks.

[0012] Optionally, the step of responding to the model update request from the UAV, adjusting the parameters of the global model based on the scene description information of the UAV, generating a personalized adaptation model, and distributing it to the corresponding UAV includes: receiving the model update request from the UAV, combining the current scene features uploaded by the UAV and historical scene preference data, matching one or at least one global decision branch with the highest relevance to the current and historical scenes of the UAV; fine-tuning the connection weights between the global decision branches and the backbone network according to the scene adaptation rule base, and adjusting the convolution kernel parameters related to the target defects within the branches; and generating a personalized adaptation model for the UAV based on the adjusted connection weights and convolution kernel parameters.

[0013] Optionally, the method further includes: when the central server generates a personalized adaptation model and distributes it to the corresponding UAV, it adopts either a server-side adaptation distribution mode or an edge-side adaptation distribution mode; when the distribution mode is server-side adaptation, a complete personalized adaptation model is generated, encrypted, and then sent to the UAV; the UAV decrypts the received personalized adaptation model file, replaces the original local defect identification model, and activates the corresponding decision branch; when the distribution mode is edge-side adaptation, the backbone network, all global decision branches, and scene adaptation configuration files are distributed; the UAV activates the corresponding decision branch and completes its parameter adjustment according to the configuration file.

[0014] Secondly, embodiments of this application provide a drone inspection defect identification model update system based on federated learning, comprising: an information acquisition module, used to acquire performance evaluation information of the local identification model in the local scene, and transmit the local identification model parameters, scene description information of the current inspection environment, and performance evaluation information to a central server; a model aggregation module, used to classify each drone in the scene and evaluate the activity of the local model according to the scene description information, and to perform weighted aggregation of the local identification model parameters in combination with the performance evaluation information to obtain a global model; and a model update module, used to respond to the model update request of the drone, adjust the parameters of the global model according to the scene description information of the drone, generate a personalized adaptation model, and distribute it to the corresponding drone.

[0015] Thirdly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the method described in the first aspect above.

[0016] The beneficial effects of this application are: 1. This solution uses scene classification results and performance evaluation information to weighted aggregate the parameters of the local UAV identification model. It can effectively integrate the common features of different UAVs in similar scenarios, enhance the generalization ability of the model by utilizing the diversity of group data, and avoid interference from heterogeneous environmental data through scene classification. This enables the global model to accurately capture the distribution of defect features in specific scenarios, overcoming the problem of poor adaptability of the model in specific scenarios caused by the traditional indiscriminate aggregation strategy. 2. By responding to the model update request from the UAV, the global model is adjusted according to the scene description information of the UAV to generate a personalized adapted model. This hierarchical architecture of "central aggregation - edge adaptation" makes full use of the knowledge accumulation of the global model. Through parameter sharing and fine-tuning at the structural level, it not only retains the general feature extraction capability of the global model in a wide range of scenarios, but also combines the specific needs of the UAV's local scene. It effectively solves the problem of low recognition accuracy caused by the single data of the local model, thereby significantly improving the defect recognition efficiency in specific scenarios while ensuring the robustness of the model. Attached Figure Description

[0017] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings. The drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings.

[0018] Figure 1 A flowchart of the method for updating the defect identification model of UAV inspection based on federated learning provided in the embodiments of this application.

[0019] Figure 2 A schematic diagram of a drone inspection defect identification model update system module based on federated learning provided in this application embodiment. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description of this application is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely one preferred embodiment of this application and are only used to explain this application. They do not limit the scope of protection of this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] Example 1: As Figure 1 As shown, a method for updating a defect identification model for UAV inspection based on federated learning includes steps S1-S3, wherein: S1. Obtain the performance evaluation information of the local recognition model in the local scene, and transmit the local recognition model parameters, the scene description information of the current inspection environment, and the performance evaluation information to the central server.

[0022] In an optional embodiment, step S1 includes: Acquire environmental characteristic information of the drone when it performs inspection tasks, and identify and classify different environmental units based on changes in environmental characteristic information; A corresponding scene feature description is generated for each environmental unit, and the defect image data collected within the environmental unit is associated with the scene feature description; The decision branch in the local defect identification system of the UAV is trained based on the current scene feature description. The parameters are adjusted by the defect image data associated with the current scene feature description, and the parameter update amount of the decision branch is obtained. Based on the subset of the local test dataset of the drone that matches the current environmental unit, calculate the accuracy, recall and F1 score of the decision branch after parameter adjustment, and generate a scene-specific performance evaluation summary. The parameter update amount, corresponding scene feature description, and performance evaluation summary of each scenario-specific decision branch are encapsulated into an independent knowledge module and transmitted to the central server.

[0023] In this embodiment, environmental feature information includes at least light intensity, temperature, humidity, latitude and longitude, and altitude data; inspection images are used to identify infrastructure types (such as power towers / oil pipelines / bridges) and preliminary defect types. The above data is integrated into a real-time environmental feature vector E=[light intensity, temperature, humidity, longitude, latitude, altitude, equipment type code], and this environmental feature vector is used as a scene feature description. The environmental unit determination and division step includes: continuously calculating the environmental feature vector of consecutive sampling points and the average vector of the first m sampling points. The Euclidean distance, when When a preset threshold is reached, a new environmental unit is determined and assigned a unique identifier. All defect image data collected within the environmental unit are automatically labeled with the corresponding unique identifier and a detailed scene feature description of that unit, forming a local dataset with "scene-data" binding.

[0024] Furthermore, the UAV local defect recognition system consists of a modular model with a shared general feature extraction backbone and multiple scene-specific decision branches. The backbone adopts a ResNet-50 / YOLOv5 architecture, and the decision branches are used to identify specific defect types or adapt to specific environmental conditions, such as a high humidity corrosion recognition branch, a low light crack recognition branch, and a foreign object recognition branch. Each branch corresponds to a typical scene / defect. During decision branch training, the scene features of the current environmental unit are matched through a rule engine, and only the 1-2 most relevant decision branches are activated. The parameters of the inactive branches are frozen. For example, if (humidity > 80% AND geographical location = coastal) THEN, the "high humidity corrosion recognition branch" is activated. During parameter adjustment, only the labeled dataset of the current environmental unit is used to fine-tune the activated decision branches and the last two layers of the backbone network. A subset matching the current environmental unit is selected from the local privacy-free test set, and the accuracy, precision, recall, and F1 score of the activated branch are calculated to generate a scene-specific performance evaluation summary (clearly labeled with the corresponding branch ID and environmental unit ID).

[0025] Furthermore, homomorphic encryption or secure multi-party computation protocols are used to encrypt the parameter update amount of each active branch individually; the encrypted branch parameter update amount, the corresponding environment unit ID, the detailed scene feature description, and the scene-specific performance evaluation summary are encapsulated into an independent knowledge module and then encrypted and transmitted to the central server.

[0026] In this embodiment, by acquiring environmental feature information (including illumination, humidity, geographical location, and inspection equipment type) of the UAV during its inspection mission and dividing the environment into units, the physical space inspection mission is transformed into logical scene units, providing a foundation for the refined management of the model. By generating scene feature descriptions and associating them with defect image data, a mapping relationship between environmental semantics and defect data is established, enabling the model to perceive the environmental context. Furthermore, the corresponding decision branches in the local defect identification system are trained specifically based on the scene feature descriptions, updating only parameters related to the current environment, thus avoiding catastrophic forgetting caused by training the entire network. By calculating the accuracy, recall, and F1 score of the adjusted decision branches on the matching subset, a scene-specific performance evaluation summary is generated. This not only quantifies the effectiveness of the local model in a specific environment but also encapsulates the model parameter update amount, scene description, and performance indicators into independent knowledge modules. Thus, only model parameters and metadata are included during transmission, avoiding the uploading of original sensitive image data and effectively solving the risk of data privacy leakage. At the same time, it provides accurate evaluation basis for the subsequent scene-based weighted aggregation of the central server. Meanwhile, the independent knowledge module simplifies knowledge in complex mixed environments, facilitating subsequent classification and aggregation operations by the central server and improving the overall efficiency and effectiveness of the federated learning system.

[0027] S2. Based on the scene description information, classify each drone in terms of scene and evaluate the activity of the local model. Combine the performance evaluation information to perform weighted aggregation of the local recognition model parameters to obtain the global model.

[0028] In an optional embodiment, step S2 includes: S21. After decrypting the received knowledge module, the central server creates a virtual knowledge agent. The agent's attributes include at least the physical drone ID, environmental unit characteristics, decision branch type, and scene-specific performance indicators. S22. Based on the environmental unit characteristics of virtual knowledge agents, all virtual knowledge agents are divided into different scenario professional groups; S23. Within each scenario professional group, the update amounts of decision branch parameters from different virtual knowledge agents are weighted and aggregated to generate a global model.

[0029] In this embodiment, the virtual knowledge agent is an abstract mapping of the knowledge contributed by a physical drone in a specific environmental unit. If a physical drone traverses multiple environmental units and uploads multiple independent knowledge modules in a single mission, then multiple virtual knowledge agents will be generated in the logical space of the central server. Each virtual knowledge agent carries rich attribute information, including its physical drone ID (for traceability), environmental unit characteristics (for scene classification), decision branch type (for distinguishing between corrosion recognition branches, crack recognition branches, etc.), and scene-specific performance indicators (for evaluating the quality of the knowledge module). By using virtual knowledge agents, complex physical entities are decoupled into fine-grained logical units, laying the foundation for subsequent refined aggregation.

[0030] Furthermore, the central server uses clustering algorithms (such as K-means or DBSCAN) to analyze the environmental unit characteristics of each virtual knowledge agent. For example, agents whose environmental feature descriptions contain tags such as "high humidity" and "coastal" are grouped into the "coastal high salt fog scene professional group," while agents containing tags such as "low light" and "mountainous area" are grouped into the "mountainous low light scene professional group." This grouping operation ensures that the subsequent aggregation process is carried out within a knowledge set with similar environmental backgrounds, avoiding the erroneous mixing of knowledge from different scenarios, and thus effectively preventing the dilution of professional knowledge during the aggregation process.

[0031] In this embodiment, parameter aggregation is performed within each scenario professional group, enabling model parameters from similar environments (such as UHV substations and coastal transmission lines) to mutually enhance each other, thus avoiding parameter conflicts caused by heterogeneous environmental data (such as significant differences in lighting and background) in traditional federated learning. Through the isolation aggregation strategy based on scenario classification, the problem of poor model adaptability in specific scenarios caused by indiscriminate aggregation is effectively overcome, enabling the generated global model to accurately capture the distribution of defect features in specific scenarios and significantly improving the model's generalization ability in complex and variable inspection environments.

[0032] In an optional embodiment, step S23 includes: The activity level of virtual knowledge agents in each scenario category is evaluated based on the frequency and number of knowledge modules uploaded by virtual knowledge agents in a recent period. Identify the environmental characteristics corresponding to scene categories with activity levels below a preset threshold, and select general decision branch parameters that match the environmental characteristics from a preset general knowledge reserve as supplementary knowledge; Retrieve the scenario-specific accuracy of each virtual knowledge agent in identifying specific defects; The aggregation weight of the virtual knowledge agent is dynamically adjusted based on the activity assessment results and the scene-specific accuracy. Combined with the supplementary knowledge, the update amount of decision branch parameters from different virtual knowledge agents is weighted and aggregated to generate a global decision branch. The global decision branch is integrated with the shared general feature extraction backbone network to form the global model.

[0033] Specifically, activity level refers to the continuous contribution of the virtual knowledge agent to the scene's professional group. It is obtained based on the frequency and quantity of knowledge module uploads within a sliding time window, combined with a time decay factor. The activity level expression is: (1); In the formula, The normalized activity of the j-th virtual knowledge agent at time t (value range [0,1], the sum of the activity of all agents in the group is 1). , Let represent the number of times virtual knowledge agents j and i upload knowledge modules within the sliding time window [tT,t], where T is the length of the sliding time window. , Let J and i be the times when virtual knowledge agents j and i upload the knowledge module for the kth time, respectively. , Let j and i be the number of knowledge modules uploaded by virtual knowledge agents j and i in the kth instance, respectively. The time decay coefficient ( (The larger the value, the more emphasis is placed on recently uploaded knowledge). Let be the total number of virtual knowledge agents within the scenario professional group S. Here, virtual knowledge agent j refers to the target agent, and virtual knowledge agent i refers to any agent within the scenario professional group S.

[0034] In this embodiment, by introducing a time-decaying activity metric, recently uploaded knowledge reflecting the latest environmental changes receives higher weight, preventing the global model from being dominated by outdated historical knowledge. Activity changes also allow for the rapid identification of abnormal agents (such as drones that suddenly stop uploading or upload large amounts of abnormal data in a short period). Combined with subsequent confidence checks, this effectively filters out invalid parameter updates caused by malicious attacks or device malfunctions. For scenario categories with activity levels below a threshold, supplementary knowledge from a general knowledge reserve is introduced, effectively preventing model degradation in cold-start or data-sparse scenarios and ensuring the robustness of the global model. By dynamically adjusting the aggregation weights based on the scenario-specific accuracy of specific defect identification and the activity evaluation results, high-quality, high-activity data sources have a greater impact on the global model, achieving adaptive aggregation through selection and elimination. Integrating the global decision branch with a shared general feature extraction backbone network retains general feature extraction capabilities while enhancing the recognition accuracy for specific scenarios, thus ensuring model versatility while avoiding low recognition accuracy in specific scenarios due to indiscriminate aggregation.

[0035] In an optional embodiment, dynamically adjusting the aggregation weight of the virtual knowledge agent based on the activity assessment results and the scene-specific accuracy includes: The activity evaluation results are compared with the historical activity data of the virtual knowledge agent, and the matching degree between the scene-specific accuracy and the scene feature description of the virtual knowledge agent and the amount of local test data is evaluated to identify abnormal evaluation results. The central server verifies the model parameters uploaded by the UAV in response to the abnormal evaluation results; Obtain the scenario-specific accuracy after parameter validation, and obtain the confidence factor based on the degree of anomaly in the anomaly assessment results and the parameter validation results; Identify the non-linear correlation pattern between the confidence factor, the activity assessment result, and the scene-specific accuracy, and dynamically adjust the weight mapping function according to the non-linear correlation pattern; The activity assessment result, the scene-specific accuracy, and the confidence factor are mapped to the corrected aggregate weights using the weight mapping function.

[0036] Specifically, the steps for identifying abnormal assessment results include: based on the current activity level of the virtual knowledge agent j. Compared with the historical average activity The activity deviation is obtained and represented as: If the activity deviation exceeds the activity anomaly threshold (e.g., 0.5), it is marked as agent j having an activity anomaly, and the corresponding anomaly indicator type is... Based on the scenario-specific accuracy of agent j. (i.e., the original scene-specific accuracy for agent j) and the average accuracy of the same scene group The accuracy deviation is obtained and expressed as: If the accuracy deviation exceeds the accuracy anomaly threshold (e.g., 0.3), and the amount of local test data for the drone is less than the minimum effective test sample size, then it is marked as an accuracy anomaly, and the corresponding anomaly index type is specified. .

[0037] Specifically, for agents marked as abnormal, the decision branch parameters they uploaded are extracted. Calculate the average parameters of the same scene group. The similarity is used to obtain the parameter similarity deviation, which is expressed as: If the parameter similarity deviation is less than the parameter similarity threshold, it is determined to be a severe parameter anomaly, triggering historical parameter rollback verification: retrieve the parameters uploaded normally in the last few (e.g., 3 times) by the proxy, calculate the similarity deviation with the current parameter, confirm whether it is a sudden anomaly, and generate a verification result: if the verification passes, the corresponding anomaly indicator type is determined. If it fails .

[0038] Specifically, obtaining the confidence factor includes: determining the degree of anomaly based on the activity deviation, accuracy deviation, and parameter similarity deviation; and calculating the confidence factor based on the degree of anomaly and the model parameter validation results. The expression for the degree of anomaly is: (2) In the formula, The number of exception types. (Values ​​1 / 2 / 3) For extremely small positive numbers, avoid dividing by 0; Confidence factor The expression is: (3); In the formula, This is the anomaly penalty coefficient, ranging from 0.8 to 1, used to control the degree to which the anomaly severity weakens the confidence level. This is an anomaly severity attenuation term; the greater the anomaly severity, the lower the confidence level. The basic confidence coefficient (based on the number of anomaly types) Decision), for example, ; Specifically, the scenario-specific accuracy expression after parameter validation is as follows: (4); In the formula, The corrected scene-specific accuracy is used to characterize the defect identification accuracy of the virtual knowledge agent j in the corresponding scene. This represents the average accuracy of all normal agents within the scenario-specific group S. is the confidence factor for virtual knowledge agent j. This indicates that the parameter consistency check passed and there were no anomalies. This indicates that the verification failed or that a serious anomaly exists.

[0039] In this embodiment, a dual verification mechanism is constructed by comparing the activity evaluation results with historical data and assessing the matching degree of scene-specific accuracy. This effectively identifies model parameter deviations caused by data anomalies or malicious uploads. By introducing a confidence factor, the weights of virtual knowledge agents with artificially high accuracy, sudden changes in activity, and abnormal parameters are reduced, thereby preventing data from polluting the global model and enhancing the system's security and anti-interference capabilities. A penalty coefficient is added to the confidence factor so that minor anomalies still retain a certain contribution, while only serious anomalies are masked, thus improving overall robustness.

[0040] In an optional embodiment, identifying the nonlinear correlation pattern between the confidence factor, the activity assessment result, and the scene-specific accuracy, and dynamically adjusting the weight mapping function according to the nonlinear correlation pattern, includes: The central server divides the virtual knowledge agents into multiple scenario-specific groups based on the scenario feature descriptions of each virtual knowledge agent. Each group corresponds to an inspection area or device type. Based on the interaction between the activity assessment results within each group, the scene-specific accuracy, and the confidence factor, a nonlinear association pattern for scene-specific groups is obtained. Continuously monitor the real-time trends of activity assessment results, scenario-specific accuracy, and confidence factors within each scenario-specific group; Compare the real-time change trend with the historical correlation pattern of the scene-specific group to identify if the real-time change trend deviates from the historical correlation pattern by more than a threshold, or if a new combination pattern that has not been recorded appears. In response to the new combination pattern, the nonlinear association pattern of the scene-specific group is updated, and the weight mapping function is adjusted according to the updated nonlinear association pattern.

[0041] Specifically, the weight mapping function expression is: (5); In the formula, The final aggregate weight of the j-th virtual knowledge agent within the scenario professional group S (the sum of the weights within the group is 1). Let be the normalized activity level of the i-th virtual knowledge agent at time t. The non-linear adjustment coefficient for the accuracy of scene S ( (The larger the value, the more significant the impact of accuracy on the weight). The nonlinear adjustment coefficient for the activity level of scenario S ( The larger the value, the more significant the impact of activity on weight. exp(·) represents the exponential function, used to achieve non-linear mapping and amplify the weight advantage of high-contribution virtual knowledge agents.

[0042] In some embodiments, based on the original scene-specific groups, the system is further divided into more granular scene-specific subgroups according to inspection area or equipment type. Each subgroup independently maintains a relational pattern database, storing the historical correspondence between activity, accuracy, confidence factor, and final weight. A dataset is built based on the historical data of each subgroup. ,in, The optimal weights for historical manual annotations are represented (derived by back-calculation based on the test accuracy of the global model in this scenario). Gradient Boosting Tree (GBDT) or a lightweight neural network (such as a 2-layer MLP) is used to fit the nonlinear relationship between the activity evaluation results, scene-specific accuracy, confidence factor, and optimal weights, resulting in an initial nonlinear association pattern, expressed as: The pattern of mutual enhancement or inhibition among the three factors is captured: ① High accuracy + high activity + high confidence, weight increases; ② High accuracy but low confidence, weight is suppressed; ③ High activity but low accuracy, weight is not amplified.

[0043] Furthermore, a sliding window (e.g., a window size of 5 rounds) is used to monitor the changing trends of three indicators within the subgroup in real time. For the activity trend, the mean and variance of activity within the window are calculated. If the variance exceeds a historical threshold, it indicates a significant change in the distribution of participating agents in this scenario. For the accuracy trend, the mean of accuracy within the window is calculated. If it shows a downward trend for three consecutive rounds, it indicates a drift in the defect features of this scenario. For the confidence trend, the proportion of low-confidence agents within the window is calculated. If the proportion exceeds a preset threshold, such as 20%, it indicates a systematic anomaly in this scenario.

[0044] Furthermore, by comparing real-time trends with historical correlation patterns, an update mode is triggered and updates are performed if either of the following two conditions are met. And simultaneously adjust the weight mapping function. and In scenario one: the similarity between the real-time indicator combination and all historical combinations is below a similarity threshold (e.g., 0.7, indicating the presence of previously unrecorded combinations). (Combination); Case 2: The test accuracy of the current weight mapping function in this subgroup drops by more than the preset value (e.g., 5%, indicating mode failure).

[0045] Furthermore, the update amounts of decision branch parameters from different virtual knowledge agents within the same scene's professional group are weighted and aggregated to obtain the scene's global decision branch parameter update amount, expressed as: (6); In the formula, The global decision branch parameter update amount for scenario professional group S; Finally, the central server integrates the global decision branches of each scenario with the shared backbone network to form a complete global model library.

[0046] In this embodiment, by identifying the nonlinear correlation pattern between confidence factors, activity assessment results, and scene-specific accuracy, and dynamically adjusting the weight mapping function accordingly, the aggregation weight is no longer a simple linear function, but can be intelligently adjusted based on data quality, activity level, and historical performance. This ensures the robustness of the model update process, avoids the pollution of the global model by low-quality or abnormal data, and enables the system to continuously optimize the aggregation strategy based on actual operating data, thereby maintaining the high performance of the model in complex and ever-changing inspection networks.

[0047] S3. In response to the model update request from the drone, adjust the parameters of the global model according to the scene description information of the drone, generate a personalized adaptation model and distribute it to the corresponding drone.

[0048] In an optional embodiment, step S3 includes: Receive the model update request from the drone, combine the current scene features uploaded by the drone with the historical scene preference data, and match one or at least one global decision branch that is most relevant to the current and historical scenes of the drone. Fine-tune the connection weights between the global decision branch and the backbone network based on the scenario adaptation rule base, and adjust the convolution kernel parameters related to the target defect within the branch; A personalized adaptation model for the drone is generated based on the adjusted connection weights and convolution kernel parameters.

[0049] Specifically, the construction of the scene adaptation rule base is based on the statistical analysis of historical inspection data and the integration of expert experience. The central server periodically analyzes performance evaluation summaries uploaded by virtual knowledge agents within each scene's professional group to identify model parameter features that perform exceptionally well in specific scenarios. For example, through comparison, in a "low-light mountainous" scenario, models with an accuracy exceeding 90% in crack detection generally have connection weights from the 3rd and 4th convolutional layers of their backbone network to the crack detection branch that are 15%-25% higher than the baseline value. Based on such statistical patterns, corresponding adaptation rules are generated or updated. The scene adaptation rule base adopts a hierarchical knowledge organization structure, comprising three levels: a scene feature layer, a defect type layer, and a parameter adjustment strategy layer. The scene feature layer defines the classification system of environmental conditions, including lighting conditions (such as strong light, normal, low light, night), meteorological conditions (such as high humidity, dryness, strong wind, salt spray), geographical environment (such as coastal, mountainous, plain, urban), and equipment type (such as power tower, transmission line, substation equipment). The defect type layer classifies typical defects of different inspection objects, such as metal corrosion, insulator damage, conductor strand breakage, tower cracks, etc. The parameter adjustment strategy layer is the core of the rule base, storing model parameter adjustment schemes for specific scene-defect combinations.

[0050] Furthermore, each scenario adaptation rule includes the following key elements: rule identifier, description of applicable scenario features, target defect type, parameter adjustment strategy, adjustment range, and expected effect metrics. The parameter adjustment strategy includes connection weight adjustment strategy and convolutional kernel parameter adjustment strategy. The connection weight adjustment strategy defines how the weights of each layer's output to the decision branch are adjusted, including the layer number to be adjusted, the adjustment direction (enhancement or suppression), the range of adjustment coefficients, and the adjustment priority. The convolutional kernel parameter adjustment strategy defines the fine-tuning scheme for the convolutional kernels within the decision branch, including the location of the target convolutional kernel, the parameter adjustment type (such as weight scaling, bias correction, filter reorganization), the calculation formula for the adjustment parameters, and the constraints.

[0051] In this embodiment, by combining the current scene features and historical scene preference data uploaded by the UAV, the most relevant global decision branch is matched to achieve accurate knowledge delivery from the model. The connection weights between the global decision branch and the backbone network are fine-tuned according to the scene adaptation rule base, and the convolution kernel parameters related to the target defect within the branch are adjusted. This achieves lightweight fine-tuning of the model, preserving the general knowledge of the global model while injecting specific features of the local scene. A personalized adaptation model is generated based on the adjusted parameters, enabling the distributed model to accurately match the needs of the UAV's current and historical inspection scenarios. This solves the problem of general models not being suitable for specific scenarios, thereby reducing the computational burden on the UAV end and significantly improving the accuracy of defect identification, achieving the best balance between model performance and computational efficiency.

[0052] In an optional embodiment, the method further includes: When the central server generates a personalized adaptation model and distributes it to the corresponding drone, it adopts either server-side adaptation and distribution mode or end-side adaptation and distribution mode. When the distribution mode is server-side adaptation, a complete personalized adaptation mode is generated, encrypted, and sent to the drone. The drone decrypts the received personalized adaptation model file, replaces the original local defect identification model, and activates the corresponding decision branch. When the distribution mode is edge-side adaptation, the backbone network, all global decision branches, and scenario adaptation configuration files are distributed. The UAV activates the corresponding decision branch according to the configuration file and completes its parameter adjustment.

[0053] Specifically, in the server-side adaptation and distribution mode, the central server completes all the matching, fine-tuning, and integration work mentioned above, generating a complete, directly loadable, personalized adaptation model file. The server then encrypts this file and distributes it to the drone via a secure communication link. Upon receiving the file, the drone simply decrypts and replaces it to activate the new model. This mode is suitable for scenarios with ample network bandwidth, high requirements for timely model loading, or where drone-side computing resources are limited and unable to handle complex parameter adjustments. Its advantage lies in the extremely simple operation on the drone side, allowing for immediate use and rapid response to environmental changes.

[0054] In the edge-adaptive distribution mode, the central server does not generate a complete personalized model file. Instead, it packages and distributes a shared general feature extraction backbone network, all relevant global decision branches, and a lightweight "scene adaptation configuration file" to the drone. This configuration file contains parameter adjustment instructions for the drone, such as "activate branch A, adjust the weight of convolutional kernel B to X". After receiving this data, the drone utilizes its local edge computing capabilities to complete the activation of decision branches and parameter adjustment operations locally according to the instructions in the configuration file. This mode is suitable for scenarios with limited network bandwidth, high communication costs, or where the drone has strong edge computing capabilities. Its advantage lies in significantly reducing the amount of communication data, as the configuration file size is much smaller than the complete model file, while also giving the drone the flexibility to perform secondary fine-tuning locally.

[0055] In this embodiment, two strategies are used to meet the needs of drones with different hardware configurations and network environments, improve the flexibility and compatibility of the system, and optimize the utilization efficiency of computing and communication resources while ensuring the quality of model updates.

[0056] Based on the same inventive concept, this application also provides a federated learning-based UAV inspection defect identification model update system, corresponding to the federated learning-based UAV inspection defect identification model update method, such as... Figure 2 As shown, it includes: The information acquisition module is used to acquire the performance evaluation information of the local recognition model in the local scene, and transmit the local recognition model parameters, the scene description information of the current inspection environment, and the performance evaluation information to the central server. The model aggregation module is used to classify each drone in terms of scene and evaluate the activity of the local model based on the scene description information, and to perform weighted aggregation of the local recognition model parameters in combination with the performance evaluation information to obtain the global model. The model update module is used to respond to the model update request of the UAV, adjust the parameters of the global model according to the scene description information of the UAV, generate a personalized adaptation model, and distribute it to the corresponding UAV.

[0057] In this embodiment, information acquisition is deployed on the UAV inspection terminal side to collect environmental characteristic information such as light intensity, temperature and humidity, air pressure, latitude and longitude in real time. The model aggregation module and model update module are deployed on the central server to execute the weighted aggregation algorithm. Based on activity, professionalism score and confidence factor, the aggregation weight is calculated to generate a global decision branch, which is then integrated with the shared backbone network to form a global model. The aggregated global knowledge is then transformed into customized services for specific UAVs. Through the collaborative operation of each module, the hardware architecture achieves collaborative computing power between the edge and the cloud. In terms of data flow, a closed-loop control is achieved from scene perception, knowledge uploading, intelligent aggregation to personalized delivery. In terms of functional logic, fine-grained adaptation to diverse inspection scenarios is achieved. Thus, while ensuring data privacy, the generalization ability and recognition accuracy of the defect identification model in complex environments are effectively improved.

[0058] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements a method for updating a drone inspection defect identification model based on federated learning.

[0059] The above-described embodiments are preferred embodiments of this application and are not intended to limit the specific scope of this application. The scope of this application includes but is not limited to the specific embodiments described above. All equivalent changes made in accordance with the shape, structure, and method of this application are within the protection scope of this application.

Claims

1. A method for updating a defect identification model for UAV inspection based on federated learning, characterized in that: Includes the following steps: Obtain performance evaluation information of the local recognition model in the local scene, and transmit the local recognition model parameters, scene description information of the current inspection environment, and performance evaluation information to the central server; Based on scene description information, each drone is classified into scenes and its local model activity is evaluated. Combined with performance evaluation information, the local recognition model parameters are weighted and aggregated to obtain a global model. The central server decrypts the received knowledge modules and creates a virtual knowledge agent. The agent's attributes include at least the physical drone ID, environmental unit characteristics, decision branch type, and scene-specific performance indicators. Based on the environmental unit characteristics of the virtual knowledge agents, all virtual knowledge agents are divided into different scene-specific groups. The activity of virtual knowledge agents in each scene category is evaluated based on the frequency and number of knowledge modules uploaded by the virtual knowledge agents in a recent period. Environmental characteristics corresponding to scene categories with activity levels below a preset threshold are identified, and general decision branch parameters matching these environmental characteristics are selected from a preset general knowledge reserve as supplementary knowledge. The scene-specific accuracy of each virtual knowledge agent in identifying specific defects is retrieved. The aggregation weights of the virtual knowledge agents are dynamically adjusted based on the activity evaluation results and scene-specific accuracy. Combined with the supplementary knowledge, the update amounts of decision branch parameters from different virtual knowledge agents are weighted and aggregated to generate a global decision branch. The global decision branch is integrated with a shared general feature extraction backbone network to form the global model. In response to the drone's model update request, the parameters of the global model are adjusted based on the drone's scene description information to generate a personalized adaptation model, which is then distributed to the corresponding drone. Specifically, combining the current scene features uploaded by the drone with historical scene preference data, one or more global decision branches with the highest relevance to the drone's current and historical scenes are matched. The connection weights between the global decision branches and the backbone network are fine-tuned according to the scene adaptation rule base, and the convolutional kernel parameters related to target defects within the branches are adjusted. Based on the adjusted connection weights and convolutional kernel parameters, a personalized adaptation model for the drone is generated, and the personalized adaptation model is distributed to the corresponding drone using either a server-side adaptation distribution mode or an edge-side adaptation distribution mode.

2. The method for updating the UAV inspection defect identification model based on federated learning according to claim 1, characterized in that: The step of obtaining performance evaluation information of the local recognition model in the local scene and transmitting the local recognition model parameters, scene description information of the current inspection environment, and performance evaluation information to the central server includes: Acquire environmental characteristic information of the drone when it performs inspection tasks, and identify and classify different environmental units based on changes in environmental characteristic information; A corresponding scene feature description is generated for each environmental unit, and the defect image data collected within the environmental unit is associated with the scene feature description; The decision branch in the local defect identification system of the UAV is trained based on the current scene feature description. The parameters are adjusted by the defect image data associated with the current scene feature description, and the parameter update amount of the decision branch is obtained. Based on the subset of the local test dataset of the drone that matches the current environmental unit, calculate the accuracy, recall and F1 score of the decision branch after parameter adjustment, and generate a scene-specific performance evaluation summary. The parameter update amount, corresponding scene feature description, and performance evaluation summary of each scenario-specific decision branch are encapsulated into an independent knowledge module and transmitted to the central server.

3. The method for updating the UAV inspection defect identification model based on federated learning according to claim 1, characterized in that: The dynamic adjustment of the aggregation weight of the virtual knowledge agent based on the activity assessment results and scenario-specific accuracy includes: The activity evaluation results are compared with the historical activity data of the virtual knowledge agent, and the matching degree between the scene-specific accuracy and the scene feature description of the virtual knowledge agent and the amount of local test data is evaluated to identify abnormal evaluation results. The central server verifies the model parameters uploaded by the UAV in response to the abnormal evaluation results; Obtain the scenario-specific accuracy after parameter validation, and obtain the confidence factor based on the degree of anomaly in the anomaly assessment results and the parameter validation results; Identify the non-linear correlation pattern between the confidence factor, the activity assessment result, and the scene-specific accuracy, and dynamically adjust the weight mapping function according to the non-linear correlation pattern; The activity assessment result, the scene-specific accuracy, and the confidence factor are mapped to the corrected aggregate weights using the weight mapping function.

4. The method for updating the UAV inspection defect identification model based on federated learning according to claim 3, characterized in that: The process of identifying the non-linear correlation pattern between the confidence factor, the activity assessment result, and the scene-specific accuracy, and dynamically adjusting the weight mapping function based on the non-linear correlation pattern, includes: The central server divides the virtual knowledge agents into multiple scenario-specific groups based on the scenario feature descriptions of each virtual knowledge agent. Each group corresponds to an inspection area or device type. Based on the interaction between the activity assessment results within each group, the scene-specific accuracy, and the confidence factor, a nonlinear association pattern for scene-specific groups is obtained. Continuously monitor the real-time trends of activity assessment results, scenario-specific accuracy, and confidence factors within each scenario-specific group; Compare the real-time change trend with the historical correlation pattern of the scene-specific group to identify if the real-time change trend deviates from the historical correlation pattern by more than a threshold, or if a new combination pattern that has not been recorded appears. In response to the new combination mode, the nonlinear association mode of the scene-specific group is updated, and the weight mapping function is adjusted according to the updated nonlinear association mode.

5. The method for updating the UAV inspection defect identification model based on federated learning according to claim 1, characterized in that: The method further includes: When the distribution mode is server-side adaptation, a complete personalized adaptation mode is generated, encrypted, and sent to the drone. The drone decrypts the received personalized adaptation model file, replaces the original local defect identification model, and activates the corresponding decision branch. When the distribution mode is edge-side adaptation, the backbone network, all global decision branches, and scenario adaptation configuration files are distributed. The UAV activates the corresponding decision branch according to the configuration file and completes its parameter adjustment.

6. A UAV inspection defect identification model update system based on federated learning, characterized in that: The method for updating the defect identification model of UAV inspection based on federated learning as described in any one of claims 1-5 includes: The information acquisition module is used to acquire the performance evaluation information of the local recognition model in the local scene, and transmit the local recognition model parameters, the scene description information of the current inspection environment, and the performance evaluation information to the central server. The model aggregation module is used to classify each drone in a scene and evaluate the activity of its local model based on scene description information. It then combines performance evaluation information to weighted aggregate the parameters of the local recognition model to obtain a global model. The central server decrypts the received knowledge modules and creates a virtual knowledge agent. The agent's attributes include at least the physical drone ID, environmental unit characteristics, decision branch type, and scene-specific performance metrics. Based on the environmental unit characteristics of the virtual knowledge agents, all virtual knowledge agents are divided into different scene-specific groups. The module further analyzes the frequency and quantity of knowledge modules uploaded by the virtual knowledge agents within each scene category. The system assesses the activity level of virtual knowledge agents; identifies environmental features corresponding to scene categories with activity levels below a preset threshold, and selects general decision branch parameters matching these environmental features from a preset general knowledge base as supplementary knowledge; retrieves the scene-specific accuracy of each virtual knowledge agent in identifying specific defects; dynamically adjusts the aggregation weights of virtual knowledge agents based on the activity assessment results and scene-specific accuracy, and combines the supplementary knowledge to weighted aggregate the update amounts of decision branch parameters from different virtual knowledge agents to generate a global decision branch; and integrates the global decision branch with a shared general feature extraction backbone network to form the global model. The model update module responds to model update requests from drones by adjusting the parameters of the global model based on the scene description information of the drones, generating personalized adaptation models, and distributing them to the corresponding drones. Specifically, it combines the current scene features uploaded by the drones with historical scene preference data to match one or more global decision branches with the highest relevance to the drone's current and historical scenes. It then fine-tunes the connection weights between the global decision branches and the backbone network based on the scene adaptation rule base, and adjusts the convolutional kernel parameters within the branches related to target defects. Based on the adjusted connection weights and convolutional kernel parameters, it generates personalized adaptation models for the drones and distributes these models to the corresponding drones using either a server-side adaptation distribution mode or an edge-side adaptation distribution mode.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the method for updating the defect identification model of UAV inspection based on federated learning as described in any one of claims 1-5.

Images