A method and system for unmanned aerial vehicle inspection image recognition based on federated continual learning

Abstract

The application discloses a kind of unmanned aerial vehicle inspection image recognition method and system based on federal continuous learning, belong to artificial intelligence and computer vision technical field, including the average prediction entropy of sample image is obtained by the probability distribution of sample image in known defect category;In response to the difference of average prediction entropy between adjacent training rounds is located outside mutation threshold, determine new defect category;Based on the joint dynamic sampling of sample image and sample memory bank containing new defect category, obtain training sequence;Based on training sequence, carry out deep feature extraction and joint loss calculation, obtain joint loss;Based on joint loss, adjust the weight parameter of perception model, obtain intermediate model;The accuracy index of intermediate model on known defect category and new defect category distribution is obtained;Based on the data amount of sample image in accuracy index and training sequence, obtain the adaptive aggregation weight of unmanned aerial vehicle, and based on adaptive aggregation weight, parameter aggregation is carried out to intermediate model, and the perception model is updated.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence and computer vision technology, specifically relating to a method and system for drone inspection image recognition based on federated continuous learning. Background Technology

[0002] With the rapid development of the low-altitude economy and drone technology, the use of swarms of patrol drones for autonomous inspection of critical infrastructure across wide areas (such as inter-regional ultra-high-voltage power grids and long-distance oil and gas pipelines) has become an important means of ensuring national industrial and energy security. Existing drone systems have matured in platform mobility control technologies such as flight control navigation and lidar obstacle avoidance. However, when faced with complex aerial images, drones urgently need a "visual perception brain" to accurately identify various equipment defects (such as insulator damage, cable icing, and metal corrosion).

[0003] In wide-area inspection scenarios, high-definition inspection images collected by drones often contain sensitive geographic mapping information or core confidential information. Due to national data security regulations and commercial barriers between cross-regional operating units (such as different provincial network companies), it is absolutely forbidden to directly upload this raw image data to a single cloud server for unified model training via the public network. To break down these "data silos," federated learning has been introduced into drone inspection networks. Each base station or airborne edge computing node trains its model locally using its own image data, while the server only aggregates the model parameters, thereby achieving joint modeling while ensuring data privacy.

[0004] Conventional federated learning frameworks typically assume that edge nodes possess a static and closed dataset, and perform one-time joint training based on this. In real-world wide-area UAV power line inspection missions, aerial data is not static but arrives continuously in a streaming form as the flight path progresses. Due to the complex and ever-changing inspection environment, new equipment defect categories (such as "cable icing," "bird nests," or "severe hardware corrosion" that occur seasonally and geographically) emerge sequentially over time. UAV systems cannot and should not wait until all potential defect types have been collected before global retraining; instead, they require the onboard visual perception center to have the ability to "learn while flying," that is, to continuously and incrementally absorb new knowledge from newly received streaming tasks while maintaining the coordination of the federated architecture. Therefore, there is an urgent need to extend the traditional federated learning framework to the Federated Continual Learning paradigm.

[0005] However, during this streaming incremental update process, the UAV visual perception model faces a severe dual challenge of "spatiotemporal catastrophic forgetting": (1) Catastrophic amnesia in the time dimension: As the inspection route progresses, the types of equipment defects faced by the UAV will continue to evolve. For example, in the early stages of the flight, the UAV only grasps the characteristics of "damaged insulators" and "normal insulators". However, as the inspection deepens, new and unknown defect categories such as cable icing and metal component corrosion will continuously emerge (i.e., incremental new tasks). Due to the limitations of onboard micro storage, the UAV cannot retain a large number of images of historical defects for a long time. When the onboard model directly uses the newly arrived icing defect data for incremental fine-tuning, it will generate serious gradient interference. The new features forcibly cover the old features, causing the UAV to completely lose the ability to identify the original insulator damage after learning to identify icing.

[0006] (2) Catastrophic forgetting in the spatial dimension: The types of defects collected by drone formations in different geographical regions (such as the Gobi Desert in the northwest and the southeast coast) vary greatly across regions and the data distribution is unbalanced. Traditional federated learning methods (such as FedAvg) blindly perform weighted averaging based on the amount of data at each node when aggregating model parameters in the cloud. This can lead to some poor-quality airborne models that are severely "unbalanced" or forgotten due to limited data, polluting the global model, forcibly smoothing out high-value sparse defect features, and causing the global recognition performance to collapse in the spatial dimension.

[0007] More problematic is that these two forgetting effects intertwine and amplify in drone federation networks: spatial aggregation bias in the cloud worsens the temporal forgetting of local drones as the model is deployed; and the amnesia of local drones further reduces the aggregation quality of the next round in the cloud.

[0008] Therefore, there is an urgent need in this field for a federated continuous learning image recognition method that can simultaneously mitigate catastrophic forgetting from both temporal and spatial perspectives, thus opening up the engineering closed loop of autonomous inspection and endowing UAV swarms with lifelong visual perception capabilities that are continuously evolving and do not forget. Summary of the Invention

[0009] Purpose of the invention: This application develops a method and system for image recognition of unmanned aerial vehicle (UAV) inspections based on federated continuous learning, aiming to solve the technical problems in the prior art.

[0010] Technical Solution: In a first aspect, embodiments of this application provide a method for drone inspection image recognition based on federated continuous learning, including: Inspection images are recognized using a pre-set perception model; The perception model is updated by training it with sample images collected by drone inspections, including: The average prediction entropy of the sample image is obtained by the probability distribution of the sample image in the known defect category; In response to the difference in the average prediction entropy between adjacent training rounds being outside the mutation threshold, a new defect category is determined, and sample images containing the new defect category are added to the sample memory bank. A training sequence is obtained by performing joint dynamic sampling based on sample images containing new defect categories and the sample memory. The training sequence contains sample images with known defect category identifiers and new defect category identifiers. Based on the training sequence, deep feature extraction and joint loss calculation are performed to obtain the joint loss; Adjust the weight parameters of the perception model based on the joint loss to obtain an intermediate model; Obtain the accuracy index of the intermediate model on the distribution of known defect categories and new defect categories; Based on the accuracy metric and the amount of data in the sample images of the training sequence, the adaptive aggregation weights of the UAV are obtained, and the intermediate model is aggregated based on the adaptive aggregation weights to update the perception model.

[0011] In some embodiments, the step of adding the sample image containing the new defect category to the sample memory includes: The sensory model is used to extract features from the sample images of the new defect category to obtain sample features; Obtain the class center of the sample features in the feature space, and select safe samples among the sample images of the new defect category based on the Euclidean distance between the sample features and the class centers to construct a safe candidate pool; Based on the Euclidean distance, a target number of safe samples are selected as representative samples in ascending order, and the complement sample images of the representative samples in the safe candidate pool are determined as the remaining samples; The remaining samples, selected in descending order of normalized entropy value as the target number, are used as marginal samples, and the sample memory increment for a new defect category is constructed by combining these representative samples; the formula for representing the normalized entropy value includes: ; in, For the corresponding sample image The normalized entropy value; The number of known defect categories; The number of new defect categories; For traversing the defect categories; For sample images Belongs to the The probability of a defect category; The sample memory is incremented and added to the sample memory.

[0012] In some embodiments, the method further includes: Based on the updated perception model, feature reprojection is performed on sample images of defect categories in the sample memory to obtain projected features; The average value of the projected features is normalized twice to update the class centers of the defect categories. The characterization formula includes: ; in, The class center for the defect category; An index for inspection tasks; For training rounds; For drones Sample memory bank In the middle, it belongs to the defect category. A collection of sample images; for The total number of sample images in the dataset; To utilize the updated perception model parameters For sample images The extracted high-dimensional feature vector is used to characterize the sample features; for Norms are used to perform spatial normalization operations on eigenvectors.

[0013] In some embodiments, the known defect category identifier includes: The first sub-identifier used to characterize a representative sample of a known defect category; A second sub-identifier used to characterize edge samples of known defect categories.

[0014] In some embodiments, the joint loss includes memory-aware gradient coordination loss and adaptive semantic decoupling loss.

[0015] In some embodiments, the step of obtaining the memory-aware gradient coordination loss includes: Obtain the absolute value of the gradient of the sample images in the training sequence; Based on the known defect category identifier and the new defect category identifier, the absolute gradient values ​​corresponding to the sample images in the training sequence are grouped, and the average value of the absolute gradient values ​​in the group is obtained as the local mean. An adaptive normalization factor is obtained based on the local mean, and a memory-aware gradient coordination loss is obtained by combining the absolute value of the gradient. The representation formula of the memory-aware gradient coordination loss includes: ; in, For memory-aware gradient coordination loss; The total number of sample images in the training sequence; For inspection tasks Next, the The absolute value of the gradient generated by a sample image at its corresponding ground truth node in the output layer of the model classifier; A minimal constant introduced to prevent the denominator from being zero; The standard cross-entropy loss function; For inspection tasks The next One sample image; for Corresponding defect category identifier; For drones During inspection mission The The model parameters of the perception model after each training round; To utilize right The predicted probability distribution output after feature extraction; This is the adaptive normalization factor.

[0016] In some embodiments, the step of obtaining the adaptive semantic decoupling loss includes: Construct a hybrid target vector based on the perception model to be updated; Use the Sigmoid response of the perceptual model to be updated as a soft label for the known defect category dimension; Obtain the true unique heat hard label for the new defect category; The binary cross-entropy between the updated perceptual model's prediction result and the mixed target vector, obtained based on the hybrid target vector, the soft label, and the one-hot hard label, is used as the adaptive semantic decoupling loss. The formula for representing the adaptive semantic decoupling loss includes: ; in, For adaptive semantic decoupling loss; The total number of sample images in the training sequence; This represents the total number of known defect categories. This represents the total number of new defect categories. For traversing the defect categories; For the perception model to be updated, the first The sample image in the th... The predicted response value for the defect category is the independent probability strength of the Logits output by the perception model to be updated after being mapped by the Sigmoid function. hour, Used to characterize uniquely heated hard labels. For the true unique hot hard label The indicator value for the defect category; hour, Used to characterize soft tags. This is the balance weight constant; In order to carry out inspection tasks The following is the first The mixed target vector constructed from sample images In the The values ​​that the elements of a class can take.

[0017] In some embodiments, the step of obtaining the accuracy metric of the intermediate model on the distribution of known defect categories and new defect categories includes: Obtain a dynamic evaluation set that is physically isolated from the sample memory; By performing closed-loop inference on the dynamic evaluation set using an intermediate model, the accuracy index of the intermediate model on the distribution of known defect categories and new defect categories is obtained. The formula for representing the accuracy index includes: ; in, For inspection tasks The Drones in each training round Accuracy metrics; For dynamic evaluation set The total number of test samples in the sample; for A single test sample image of a drone and its corresponding real defect category identifier. For sample images The corresponding actual defect category identifier; For drones During inspection mission The The model parameters of the perception model after each training round; For the perceptual model to sample images The predicted probability distribution output after feature extraction; The function is used to extract the category index corresponding to the maximum value in the predicted probability distribution, i.e., the predicted category of the perceptual model; This is an indicator function that takes the value 1 when the logical condition within the parentheses is true, and 0 otherwise.

[0018] In some embodiments, the formula for representing the adaptive aggregation weights of the UAV includes: ; in, For inspection tasks The Drones in each training round Adaptive aggregation weights; For inspection tasks The Drones in each training round Accuracy metrics; For inspection tasks Chinese drones The amount of data in the sample images in the training sequence used for model training; For inspection tasks The set of drone nodes selected for parameter aggregation; For set The index.

[0019] Secondly, embodiments of this application also provide a drone inspection image recognition system based on federated continuous learning, comprising: The inspection and recognition module is used to recognize inspection images through a preset perception model. A model update module, used for training the perception model using sample images collected by the UAV during inspection, and comprising: A task mutation detection submodule is used to obtain the average prediction entropy of the sample image through the probability distribution of the sample image in the known defect category; A dual-path memory management submodule is used to determine a new defect category and add the sample image containing the new defect category to the sample memory bank in response to the difference in the average prediction entropy between adjacent training rounds being outside the mutation threshold. The training batch construction submodule is used to perform joint dynamic sampling based on sample images containing new defect categories and the sample memory to obtain training sequences, wherein the training sequences contain sample images with known defect category identifiers and new defect category identifiers; A joint loss calculation submodule is used to perform deep feature extraction and joint loss calculation based on the training sequence to obtain the joint loss. A local model optimization submodule is used to adjust the weight parameters of the perceptual model based on the joint loss to obtain an intermediate model. The dynamic evaluation set management submodule is used to obtain the accuracy index of the intermediate model on the distribution of known defect categories and new defect categories. The global adaptive aggregation submodule is used to obtain the adaptive aggregation weight of the UAV based on the accuracy index and the amount of data of sample images in the training sequence, and to perform parameter aggregation on the intermediate model based on the adaptive aggregation weight to update the perception model.

[0020] Beneficial Effects: Compared with existing technologies, the present application provides a method for UAV inspection image recognition based on federated continuous learning, including inspection image recognition through a preset perception model, model training through sample images collected by UAV inspection, and updating the perception model, including: obtaining the average prediction entropy of sample images through the probability distribution of sample images in known defect categories; determining a new defect category in response to the difference in average prediction entropy between adjacent training rounds being outside the mutation threshold, and adding sample images containing the new defect category to the sample memory; performing joint dynamic sampling based on the sample images containing the new defect category and the sample memory to obtain a training sequence, the training sequence containing sample images of known defect category identifiers and new defect category identifiers; performing deep feature extraction and joint loss calculation based on the training sequence to obtain a joint loss; adjusting the perception model weight parameters based on the joint loss to obtain an intermediate model; obtaining the accuracy index of the intermediate model on the distribution of known defect categories and new defect categories; obtaining the adaptive aggregation weight of the UAV based on the accuracy index and the amount of data of sample images in the training sequence, and performing parameter aggregation on the intermediate model based on the adaptive aggregation weight to update the perception model. This application overcomes two key problems faced by existing federated continuous learning in scenarios involving streaming and non-independent identically distributed (Non-IID) aerial data from drones: local new knowledge overwriting old knowledge (catastrophic temporal forgetting) and global aggregation bias caused by heterogeneous data (catastrophic spatial forgetting). It greatly improves the accuracy and generalization ability of airborne vision models when evolving across domains. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 A flowchart illustrating the steps of the drone inspection image recognition method based on federated continuous learning provided in this application embodiment; Figure 2 A flowchart illustrating the steps of updating the perception model in the drone inspection image recognition method based on federated continuous learning provided in this application embodiment; Figure 3 A flowchart illustrating the steps of updating the sample memory containing sample images of defect categories in the drone inspection image recognition method based on federated continuous learning provided in this application embodiment; Figure 4 A flowchart illustrating the steps of updating the defect category center in the UAV inspection image recognition method based on federated continuous learning provided in this application embodiment; Figure 5A flowchart illustrating the steps of obtaining the memory-aware gradient coordination loss in the UAV inspection image recognition method based on federated continuous learning provided in this application embodiment; Figure 6 A flowchart illustrating the steps for obtaining adaptive semantic decoupling loss in the UAV inspection image recognition method based on federated continuous learning provided in this application embodiment; Figure 7 A flowchart illustrating the steps for obtaining accuracy metrics in the drone inspection image recognition method based on federated continuous learning provided in this application embodiment; Figure 8 A module connection diagram of a drone inspection image recognition system based on federated continuous learning provided in an embodiment of this application; Figure 9 A hardware structure framework diagram of the drone inspection image recognition method and system based on federated continuous learning provided in the embodiments of this application; Figure 10 A flowchart illustrating the drone inspection image recognition method based on federated continuous learning provided in this application embodiment; Figure 11 A flowchart illustrating the process of updating the sample memory containing sample images of defect categories in the drone inspection image recognition method based on federated continuous learning provided in this application embodiment; Figure 12 A flowchart illustrating the local optimization of the UAV in the UAV inspection image recognition method based on federated continuous learning provided in the embodiments of this application; Figure 13 A flowchart of the global two-factor adaptive aggregation in the UAV inspection image recognition method based on federated continuous learning provided in the embodiments of this application; Attached reference numerals: 1. Inspection and identification module; 2. Model update module; 10. Task mutation detection submodule; 20. Dual-path memory management submodule; 30. Training batch construction submodule; 40. Joint loss calculation submodule; 50. Local model optimization submodule; 60. Dynamic evaluation set management submodule; 70. Global adaptive aggregation submodule. Detailed Implementation

[0023] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0024] This application provides an image recognition method for UAV inspection based on federated continuous learning. The method includes: recognizing inspection images using a pre-set perception model; training the model using sample images collected by the UAV during inspection; and updating the perception model. The method includes: obtaining the average prediction entropy of sample images based on the probability distribution of known defect categories; determining a new defect category in response to the difference in average prediction entropy between adjacent training rounds being outside a mutation threshold, and adding sample images containing the new defect category to a sample memory; performing joint dynamic sampling based on the sample images containing the new defect category and the sample memory to obtain a training sequence, which includes sample images containing known defect category identifiers and new defect category identifiers; performing deep feature extraction and joint loss calculation based on the training sequence to obtain a joint loss; adjusting the perception model weight parameters based on the joint loss to obtain an intermediate model; obtaining an accuracy index of the intermediate model on the distributions of known and new defect categories; obtaining adaptive aggregation weights for the UAV based on the accuracy index and the amount of data in the sample images of the training sequence, and performing parameter aggregation on the intermediate model based on the adaptive aggregation weights to update the perception model. This application overcomes two key problems faced by existing federated continuous learning in scenarios involving streaming and non-independent identically distributed (Non-IID) aerial data from drones: local new knowledge overwriting old knowledge (catastrophic temporal forgetting) and global aggregation bias caused by heterogeneous data (catastrophic spatial forgetting). It greatly improves the accuracy and generalization ability of airborne vision models when evolving across domains.

[0025] Accordingly, embodiments of this application also provide a drone inspection image recognition system based on federated continuous learning. Please refer to [link to relevant documentation]. Figure 8 , Figure 8 This is a module connection diagram of the drone inspection image recognition system based on federated continuous learning provided in this application embodiment. The drone inspection image recognition system based on federated continuous learning provided in this application embodiment includes: Inspection and recognition module 1 is used to recognize inspection images through a preset perception model. Model update module 2 is used to train and update the perception model using sample images collected by the UAV inspection. Model update module 2 includes: Task mutation detection submodule 10 is used to obtain the average prediction entropy of the sample image through the probability distribution of the sample image in the known defect category. Dual-path memory management submodule 20 is used to determine a new defect category in response to the difference in average prediction entropy between adjacent training rounds being outside the mutation threshold, and to add the sample image containing the new defect category to the sample memory bank. Training batch construction submodule 30 is used to perform joint dynamic sampling based on sample images containing new defect categories and sample memory to obtain training sequences. The training sequences contain sample images with known defect category identifiers and new defect category identifiers. Joint loss calculation submodule 40 is used to perform deep feature extraction and joint loss calculation based on training sequences to obtain joint loss. Local model optimization submodule 50 is used to adjust the weight parameters of the perceptual model based on the joint loss to obtain an intermediate model. The dynamic evaluation set management submodule 60 is used to obtain the accuracy index of the intermediate model on the distribution of known defect categories and new defect categories. The global adaptive aggregation submodule 70 is used to obtain the adaptive aggregation weight of the UAV based on the accuracy index and the amount of data of sample images in the training sequence, and to perform parameter aggregation on the intermediate model based on the adaptive aggregation weight to update the perception model.

[0026] Specifically, please refer to Figure 9 , Figure 9 This diagram illustrates the hardware architecture of the drone inspection image recognition method and system based on federated continuous learning provided in this application. The method and system are deployed based on a "cloud-edge collaboration" architecture, forming a complete federated learning closed loop. The cloud scheduling server, equipped with a high-performance GPU (or equivalent computing power), serves as the central brain of the federated network; the drone formation performing defect detection tasks is equipped with a micro edge computing platform as an onboard visual perception node.

[0027] Understandably, after receiving a clear mission instruction, the UAV takes off and executes it, subsequently entering a local inspection and continuous learning phase on its onboard terminal. During actual flight, the UAV not only utilizes the current perception model for real-time dynamic perception and defect identification of targets, but also, for new scenarios or new feature data encountered during inspection, the UAV's onboard nodes directly use this new data to train and update the perception model locally, enabling it to quickly adapt to the current real-world environment. After completing the local intermediate model update, each onboard node only transmits the parameters of the trained intermediate model back to the cloud server via a wireless network, without uploading massive amounts of raw image data. After collecting the concurrent parameters, the cloud server performs a global aggregation operation, integrating the learning experience of each node to generate a new generation of global visual perception model with stronger generalization capabilities. Thus, the system completes a full cloud-edge collaborative interaction loop and is ready for the next round of UAV inspection missions.

[0028] In some embodiments, please refer to Figure 1 and Figure 10 , Figure 1 A flowchart illustrating the steps of the drone inspection image recognition method based on federated continuous learning provided in this application embodiment. Figure 10 This is a flowchart of the drone inspection image recognition method based on federated continuous learning provided in this application embodiment. The drone inspection image recognition method based on federated continuous learning provided in this application embodiment is specifically implemented through steps 100 and 200: Step 100: Perform inspection image recognition using a preset perception model.

[0029] Step 200: Use sample images collected by drone inspections to train the model and update the perception model.

[0030] In some embodiments, please refer to Figure 2 , Figure 2 This is a flowchart illustrating the steps of updating a preset perception model in the UAV inspection image recognition method based on federated continuous learning provided in this application embodiment. The method for updating the preset perception model in this application embodiment is specifically implemented through steps 210 to 270: Step 210: Obtain the average prediction entropy of the sample images by using the probability distribution of the sample images in the known defect categories.

[0031] Specifically, when the drone receives the current training round Local streaming data batches At that time, the parameters of the perception model that converged in the previous training round are used. Inference is then performed on this batch of data. For each sample image, its probability distribution across all known defect categories is extracted. Calculate the average prediction entropy of this batch of data. : ; in, It is a drone client. The amount of data used in training the current task; This serves as the traversal index for sample images within a local streaming data batch. ; For traversing the defect categories; The sample image is predicted by the current perceptual model as the first... The probability value of the defect category is derived from the aforementioned probability distribution. Give; It is a logarithmic function; A minimal constant introduced to prevent the argument of the logarithmic function from being zero, such as Information entropy In engineering terms, this represents the level of surprise the perception model has towards the current data.

[0032] Step 220: In response to the difference in average prediction entropy between adjacent training rounds being outside the mutation threshold, a new defect category is determined, and sample images containing the new defect category are added to the sample memory.

[0033] Understandably, when newly acquired images during drone inspections highly overlap with the feature spaces of known defect categories, the probability distribution output by the perception model is typically very concentrated. In this case, the prediction entropy is low, the system determines the environment is stable, and maintains routine incremental fine-tuning. However, when the drone enters a completely new terrain or encounters an unseen equipment defect category, the feature extractor of the old model cannot effectively identify the new features, causing the prediction probability distribution to tend to spread out evenly, thus increasing the average prediction entropy. The data shows significant fluctuations. Therefore, this application calculates the difference between the average prediction entropy of the current training epoch and the previous training epoch. This is used to determine the task boundaries. Specifically, a mutation threshold is defined. In this embodiment, it is preferably set to 1.2, if the following conditions are met: If the new task boundary is reached, it is determined that the boundary has been reached. At this point, the system will automatically trigger the subsequent local sample memory update and semantic center anchoring process.

[0034] In some embodiments, please refer to Figure 3 and Figure 11 , Figure 3 The flowchart illustrates the steps of updating the sample memory containing sample images of defect categories in the UAV inspection image recognition method based on federated continuous learning provided in this application embodiment. Figure 11The flowchart of the procedure for updating the sample memory bank containing defect category sample images in the UAV inspection image recognition method based on federated continuous learning provided in this application embodiment is as follows: In the UAV-based IoD (UAV-based edge perception system), the UAV's onboard edge devices typically face extremely stringent physical storage limitations. When a new inspection task is triggered, the system cannot save all historical aerial image data. Traditional random sampling or single-index screening often leads to two engineering disasters: First, it is easy to store severely distorted, noisy, and dirty data collected by the UAV in harsh inspection environments (such as encountering strong turbulence causing blurred images, or encountering backlight glare at specific times) into the memory bank, directly polluting subsequent model evolution; second, it is easy to retain only a large number of highly homogeneous standard / normal equipment samples, while losing the key boundary information for distinguishing complex defect scenarios. Therefore, this application designs a dynamic memory management mechanism of pre-filtering and dual-path selection, aiming to accurately lock and retain the high-quality gold samples that are most valuable for the UAV to resist forgetting with a limited storage budget. The method for updating the sample memory bank containing defect category sample images in this application embodiment is specifically implemented through steps 221 to 225: Step 221: Extract features from sample images of new defect categories using a perceptual model to obtain sample features.

[0035] Step 222: Obtain the class centers of the sample features in the feature space, and select safe samples from the sample images of the new defect category based on the Euclidean distance between the sample features and the class centers to construct a safe candidate pool.

[0036] Specifically, considering that the airborne optical sensors of drones are prone to generating extreme outliers during high-speed cruise (such as abnormal pixel distribution caused by motion blur or abrupt changes in light and shadow), a set of all new sample images for new defect categories is extracted. Utilizing the current airborne perception model Feature extraction function Calculate its class center in feature space : ; in, Represents a set A single drone aerial sample image; Represents a set The total number of new sample images included.

[0037] Subsequently, the set is calculated. Feature extraction for each sample image With class center The algorithm calculates the Euclidean distance between the samples. It forcibly removes isolated samples that are too far away, retaining only the closest ones. One sample, preferably Set as target storage capacity Three times the size, to build a security candidate pool Understandably, this step physically filters out mechanical vibration noise and out-of-distribution abnormal samples during the inspection process, ensuring that the subsequently extracted data stream has core representativeness.

[0038] Delineating the safety candidate pool Subsequently, this application implements a parallel dual-path extraction strategy, as shown in steps 223 to 225.

[0039] Step 223: Select the number of safe samples as representative samples based on the ascending order of Euclidean distance, and determine the supplementary sample images of the representative samples in the safe candidate pool as the remaining samples.

[0040] Specifically, from the security candidate pool Directly extract the distance class center Recent A set of safe samples constitutes a representative sample. These samples are located at the absolute center of the feature cluster (e.g., typical samples of insulators exploding under sufficient light and at standard shooting angles, or cable hangers). Retaining them ensures that the model anchors the most basic and core feature distribution of this category in subsequent endless task increments, preventing fundamental temporal catastrophic forgetting.

[0041] Step 224: Select the remaining samples of the target number as marginal samples based on the normalized entropy value in descending order, and combine them with representative samples to construct an incremental sample memory bank for the new defect category.

[0042] Specifically, normalized entropy is used to measure the amount of information in the remaining samples, and the sample with the largest normalized entropy value is extracted. A set of difficult-to-classify sample images constitutes a marginal memory set. The formulas for representing normalized entropy include: ; in, For the corresponding sample image The normalized entropy value; The number of known defect categories; The number of new defect categories; For traversing the defect categories; For sample images Belongs to the The probability of a defect category.

[0043] Understandably, samples with higher normalized entropy values ​​are usually challenging samples that are occluded by tree shadows or whose defect features are highly similar to the background. Extracting this data can endow the model with strong boundary discrimination capabilities when dealing with complex inspection environments (such as changing weather or occlusion) in the future, thereby improving the model's generalization robustness.

[0044] Step 225: Add the sample memory increment to the sample memory.

[0045] Specifically, through Complete the incremental construction of the memory for the new inspection category.

[0046] In some embodiments, please refer to Figure 4 , Figure 4 The flowchart illustrates the steps of updating the defect category center in the UAV inspection image recognition method based on federated continuous learning provided in this application embodiment. During the long evolution of federated continuous learning, with the continuous addition of new tasks and iterative updates to model parameters, the data originally stored in the onboard sample memory... The coordinates of old category samples in the latest high-dimensional feature space will inevitably undergo representation drift. To prevent this drift from destroying the semantic topology of old knowledge, a method is used in each global training epoch. After completion, the local drone client must utilize the model parameters of the newly updated perception model. A re-inference process (i.e., feature reprojection) is performed on sample images of all defect categories in the sample memory. Subsequently, the mean of individual features is aggregated and a second process is performed. Normalization, recalculate each defect category class center In the embodiments of this application, the method for updating the defect category center is specifically implemented through steps 226 to 227: Step 226: Based on the updated perception model, perform feature reprojection on the sample images of the defect category in the sample memory to obtain the projected features.

[0047] Step 227: Perform a second normalization on the average value of the projected features and update the class center of the defect category.

[0048] In some embodiments, the representation formula for updating the class center of a defect category includes: ; in, The class center for the defect category; An index for inspection tasks; For training rounds; For drones Sample memory bank This belongs to the defect category. A collection of sample images; for The total number of sample images in the dataset; To utilize the updated perception model parameters For sample images The extracted high-dimensional feature vectors are used to characterize the sample features; for Norms are used to perform spatial normalization operations on eigenvectors.

[0049] Understandably, the updated class center It will serve as the most accurate semantic anchor point, directly serving the next round of sample selection and joint loss function calculation, ensuring that the airborne model always maintains an accurate mapping to the feature space of the old defect categories.

[0050] Step 230: Perform joint dynamic sampling based on sample images containing new defect categories and a sample memory to obtain a training sequence. The training sequence contains sample images with known defect category identifiers and new defect category identifiers.

[0051] In some embodiments, the sample image of a known defect category includes a first sub-identifier for representing a representative sample of a known defect category, and a second sub-identifier for representing an edge sample of a known defect category.

[0052] Understandably, to overcome the bottleneck of limited GPU memory capacity in UAV onboard computing nodes, this application abandons the full data loading method when constructing local training mini-batches. Specifically, this application uses new defect task data collected in real time during the current inspection segment. With the updated local sample memory Perform joint dynamic sampling to construct a size of Hybrid training sequences ,in, For inspection tasks The next One sample image, for The corresponding defect category identifier.

[0053] Furthermore, when encapsulating the training sequence data, the identifier of each sample image is implicitly preserved in the underlying data structure, accurately distinguishing the sample images in the training sequence into three groups: sample images with new defect category identifiers, which come from a completely new defect category encountered in the current flight segment, such as the first occurrence of cable icing; sample images of known defect categories that are difficult to forget, which come from the representative memory set. Examples include standard, clear images of insulator damage taken from historical flight segments; and sample images of easily forgotten known defect categories, derived from marginal memory sets. This includes challenging historical defect samples subjected to extreme lighting and severe background interference. These diverse mixed training sequences, each with its own identifier, are then uniformly passed to the UAV's local visual perception model for deep feature extraction and joint loss calculation. This identifier is a prerequisite for the system to accurately implement a fine-grained gradient coordination mechanism (differentiating feature updates based on varying levels of difficulty) during subsequent model backpropagation.

[0054] Step 240: Perform deep feature extraction and joint loss calculation based on the training sequence to obtain the joint loss.

[0055] In some embodiments, the joint loss includes a memory-aware gradient coordination loss and an adaptive semantic decoupling loss. Understandably, when performing local incremental training on the UAV's onboard device, using only the conventional cross-entropy loss can easily lead to two fatal problems in wide-area inspection scenarios: first, a severe imbalance in the amount of aerial data captured by new and old defects (a large number of new defects captured in new flight segments overwrite old defects in the limited memory bank), resulting in the new category gradient dominating; second, global Softmax normalization causes the addition of new defects to forcibly dilute the prediction probability of old defects (i.e., a probability squeezing effect). Therefore, this application specifically weights and sums the memory-aware gradient coordination loss and the adaptive semantic decoupling loss to calculate two joint losses, in order to finely control the feature updates of the onboard model.

[0056] In some embodiments, please refer to Figure 5 and Figure 12 , Figure 5 The flowchart illustrates the steps for obtaining the memory-aware gradient coordination loss in the UAV inspection image recognition method based on federated continuous learning provided in this application embodiment. Figure 12 The flowchart of the UAV local optimization in the UAV inspection image recognition method based on federated continuous learning provided in the embodiments of this application is shown. The method for obtaining the memory-aware gradient coordination loss in the embodiments of this application is specifically implemented through steps 241 to 243: Step 241: Obtain the absolute value of the gradient of the sample images in the training sequence; Specifically, during the backpropagation calculation of gradients in the airborne perception model, the absolute value of the gradient generated by the true defect category nodes in the classifier output layer is extracted. .

[0057] Step 242: Based on the known defect category identifier and the new defect category identifier, group the absolute gradient values ​​corresponding to the sample images in the training sequence, and obtain the average value of the absolute gradient values ​​in the group as the local mean.

[0058] Specifically, based on the identifiers of the sample images, the gradients of the samples in the current batch are divided into three groups, and their means are calculated independently for each group: the mean gradient of the sample images of the new defect category. Gradient mean of sample images of known defect categories that are difficult to forget Gradient mean of sample images of easily forgotten known defect categories Subsequently, an adaptive normalization factor was constructed using the aforementioned mean. The specific assignment logic is as follows: if the sample belongs to a new class, then If it belongs to the representative old category, then take... If it belongs to the marginal old category, then take... .

[0059] Step 243: Obtain the adaptive normalization factor based on the local mean, and obtain the memory-aware gradient coordination loss by combining the absolute value of the gradient.

[0060] In some embodiments, the representation formula for memory-aware gradient coordination loss includes: ; in, For memory-aware gradient coordination loss; The total number of sample images in the training sequence. ; For inspection tasks Next, the The absolute value of the gradient generated by a sample image at its corresponding ground truth node in the output layer of the model classifier; A minimal constant introduced to prevent the denominator from being zero; The standard cross-entropy loss function; For inspection tasks The next One sample image; for The corresponding actual defect category identifier; For drones During inspection mission The The updated model parameters of the perception model in each training round; To utilize right The predicted probability distribution output after feature extraction; This is the adaptive normalization factor.

[0061] Understandably, this application divides the optimization subspace into three groups using an indicator function. For edge defect samples that are prone to generating large gradients (such as blurred images affected by extreme lighting or complex backgrounds), a larger mean is desirable. As a denominator, it adaptively suppresses overly active updates to prevent boundary drift; for representative samples with weak gradients (such as standard defects whose features are extremely stable), a smaller mean... This amplifies the loss weights, ensuring that the model remains sensitive to core features.

[0062] In some embodiments, please refer to Figure 6 and Figure 12 , Figure 6 The flowchart illustrates the steps for obtaining adaptive semantic decoupling loss in the UAV inspection image recognition method based on federated continuous learning provided in this application embodiment. Figure 12 The flowchart of the UAV local optimization process in the UAV inspection image recognition method based on federated continuous learning provided in this application embodiment illustrates how, to overcome global probability compression (e.g., when a UAV learns to recognize cable icing, the Softmax mechanism forcibly lowers the original output probability of insulator damage), this application transforms the incremental multi-classification task into multiple independent binary classification responses (Sigmoid). The previously converged perception model, stored locally on the airborne end, serves as the knowledge teacher to construct a hybrid target vector. For dimensions with known defect categories, the sigmoid response of the previously converged perceptual model is used as a soft label; for new category dimensions, the true hard labels are retained. After construction, the current model prediction is compared with... The binary cross-entropy between them is the adaptive semantic decoupling loss. In this embodiment, the method for obtaining the adaptive semantic decoupling loss is specifically implemented through steps 244 to 247: Step 244: Construct a hybrid target vector based on the perception model to be updated.

[0063] Step 245: Use the Sigmoid response of the perceptual model to be updated as a soft label for the known defect category dimension.

[0064] Step 246: Obtain the true unique heat hard label for the new defect category.

[0065] Step 247: Obtain the binary cross-entropy of the updated perceptual model's prediction results and the mixed target vector based on the hybrid target vector, soft labels, and one-hot hard labels, which serves as the adaptive semantic decoupling loss.

[0066] In some embodiments, the representation formula for adaptive semantic decoupling loss includes: ; in, For adaptive semantic decoupling loss; The total number of sample images in the training sequence. ; This represents the total number of known defect categories. This represents the total number of new defect categories. For the index of known defect category dimensions, ; For the perception model to be updated, the first The sample image in the th... The predicted response value for a known defect category is the independent probability strength of the Logits output by the perception model to be updated after being mapped by the Sigmoid function. In order to carry out inspection tasks The following is the first The mixed target vector constructed from sample images In the The values ​​that the elements of a class can take. Specifically, for the later... A new dimension for defect categories, retaining the true unique hot hard label, i.e. , here For the true unique hot hard label The indicator value for the defect category; for the previous For each known defect category, the sigmoid response of the perceptual model to be updated is used as a soft label, combined with a balance factor. To perform adaptive coordination, i.e. , here For the previous round of inspection tasks (i.e., tasks) The legacy old model for samples In the The predicted response value of the dimension, To take values ​​in The balancing weight constants between them.

[0067] Furthermore, if the sample image is a newly acquired sample image of a defect category, its true label in the known defect category dimension... At this point, the soft knowledge distillation is entirely dependent on the perception model to be updated. If the sample image is a sample image of a known defect category from the airborne sample memory, then the true label and soft label are combined by a balancing factor to prevent the error in the distillation process from being amplified as the cruise mission accumulates, thereby effectively preventing the severe distortion of the decision boundary of complex defects.

[0068] Step 250: Adjust the weight parameters of the perception model based on the joint loss to obtain the intermediate model.

[0069] Specifically, this application utilizes a lightweight deep learning optimizer (such as a momentum-driven SGD optimizer) to collaboratively update the weight parameters of the backbone feature extraction network and the defect classifier of the airborne visual perception model through multiple rounds of forward and backward propagation on a micro edge computing node until the local epoch training converges, thus obtaining the locally updated intermediate model.

[0070] Understandably. This local optimization process enables the UAV to smoothly and stably absorb newly encountered defect features (such as cable icing) during physical flight propulsion, while solidifying the already mastered defect feature representations on the underlying weight parameters.

[0071] Step 260: Obtain the accuracy metrics of the intermediate model on the distribution of known defect categories and new defect categories.

[0072] In some embodiments, please refer to Figure 7 and Figure 13 , Figure 7 The flowchart illustrates the steps involved in obtaining accuracy metrics in the drone inspection image recognition method based on federated continuous learning, as provided in this application embodiment. Figure 13 The flowchart of the global two-factor adaptive aggregation in the UAV inspection image recognition method based on federated continuous learning provided in the embodiments of this application is shown. The method for obtaining the accuracy index in the embodiments of this application is specifically implemented through steps 261 to 262: Step 261: Obtain a dynamic evaluation set that is physically isolated from the sample memory.

[0073] Specifically, the dynamic evaluation set follows a strict defect category redistribution principle. The maximum total sample size of the dynamic evaluation set is [missing information]. As the drone patrol routes extend and new defect tasks continue to evolve, the sample quota for each defect category is dynamically calculated. To make room for evaluation of newly captured defect categories, "overly easy" old defect samples with extremely high prediction confidence and no significant feature shift in the original dynamic evaluation set (such as insulator damage with extremely obvious features and no background interference) are preferentially removed, thus retaining test samples that are valuable for evaluating the model.

[0074] Step 262: Perform closed-loop inference on the dynamic evaluation set using the intermediate model to obtain the accuracy index of the intermediate model on the distribution of known defect categories and new defect categories.

[0075] In some embodiments, the formula for representing the accuracy metric includes: ; in, Index for inspection tasks; For the node index of the drone; For training rounds; For accuracy metrics; For inspection tasks The Drones in each training round Accuracy metrics; For dynamic evaluation set The total number of test samples in the sample; for A single test sample image of a drone and its corresponding real defect category identifier; For drones During inspection mission The The model parameters of the perception model after each training round; For the perceptual model to sample images The predicted probability distribution output after feature extraction; The function is used to extract the category index corresponding to the maximum value in the predicted probability distribution, i.e., the predicted category of the perceptual model; This is an indicator function that takes the value 1 when the logical condition within the parentheses is true, and 0 otherwise.

[0076] Understandably, the accuracy metric is extremely lightweight in air-to-ground collaborative communication, consisting of only a scalar value. It will serve as the core quantitative basis for subsequent cloud-based evaluation of the quality of the drone model (i.e., whether there is a serious "unevenness" or omission).

[0077] Step 270: Based on the accuracy metric and the amount of data in the sample images of the training sequence, obtain the adaptive aggregation weights of the UAV, and perform parameter aggregation on the intermediate model based on the adaptive aggregation weights to update the perception model.

[0078] In some embodiments, during the current inspection task The During round model training, after the UAV completes local fine-tuning and obtains the local intermediate model, each UAV's onboard nodes... Upload updated local model parameters to cloud server or regional base station Accuracy metrics tested on the Local Dynamic Evaluation Set (DAS) And the amount of defective data actually used for training The entire interaction process involves no image or feature vector transmission, ensuring data privacy while reducing communication overhead. After receiving the aforementioned scalar information and parameters, the cloud server dynamically calculates the weights of each UAV node participating in this aggregation round by combining two factors: "local anti-forgetting performance (accuracy)" and "local data scale (the amount of data used for training airborne nodes)". The formula for representing the adaptive aggregation weights of UAVs includes: ; in, For adaptive aggregation weights; The amount of data for sample images used in model training; For inspection tasks The Drones in each training round Adaptive aggregation weights; For inspection tasks Chinese drones The amount of data in the sample images in the training sequence used for model training; For inspection tasks The set of drone nodes selected for parameter aggregation; This is the index for traversing the collection. Logarithmic scaling. It can effectively mitigate the absolute weight bias caused by individual drones that have collected extremely redundant images; while the accuracy index The actual generalization and memory capabilities of the airborne model in dealing with the evolution of complex defects were objectively quantified.

[0079] Furthermore, the cloud server performs parameter aggregation based on this adaptive weight to generate an updated global perception model. The information is then distributed to the drone formation to prepare for inspections on subsequent flight segments. Understandably, this mechanism can adaptively identify and deprive inferior drone nodes that are severely "unbalanced" or forgotten due to harsh local environments of their global dominance, effectively preventing the spatial spread of inferior defect characteristics throughout the wide-area inspection network.

[0080] Understandably, the drone inspection image recognition method based on federated continuous learning provided in the embodiments of this application includes performing inspection image recognition through a preset perception model, training the model through sample images collected by the drone inspection, and updating the perception model, including: obtaining the average prediction entropy of the sample images through the probability distribution of the sample images in known defect categories; determining a new defect category in response to the difference in the average prediction entropy between adjacent training rounds being outside the mutation threshold, and adding the sample images containing the new defect category to the sample memory; performing joint dynamic sampling based on the sample images containing the new defect category and the sample memory to obtain a training sequence, the training sequence containing sample images of known defect category identifiers and new defect category identifiers; performing deep feature extraction and joint loss calculation based on the training sequence to obtain a joint loss; adjusting the weight parameters of the perception model based on the joint loss to obtain an intermediate model; obtaining the accuracy index of the intermediate model on the distribution of known defect categories and new defect categories; obtaining the adaptive aggregation weight of the drone based on the accuracy index and the amount of data of the sample images in the training sequence, and performing parameter aggregation on the intermediate model based on the adaptive aggregation weight to update the perception model. This application overcomes two key problems faced by existing federated continuous learning in scenarios involving streaming and non-independent identically distributed (Non-IID) aerial data from drones: local new knowledge overwriting old knowledge (catastrophic temporal forgetting) and global aggregation bias caused by heterogeneous data (catastrophic spatial forgetting). It greatly improves the accuracy and generalization ability of airborne vision models when evolving across domains.

[0081] This application has provided a detailed description of a drone inspection image recognition method and system based on federated continuous learning, as provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and its core ideas. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A drone inspection image recognition method based on federated continuous learning, characterized in that, include: Inspection images are recognized using a pre-set perception model; The perception model is updated by training it with sample images collected by drone inspections, including: The average prediction entropy of the sample image is obtained by the probability distribution of the sample image in the known defect category; In response to the difference in the average prediction entropy between adjacent training epochs being outside the mutation threshold, a new defect category is determined, and sample images containing the new defect category are added to the sample memory bank. The sensory model is used to extract features from the sample images of the new defect category to obtain sample features; Obtain the class center of the sample features in the feature space, and select safe samples among the sample images of the new defect category based on the Euclidean distance between the sample features and the class centers to construct a safe candidate pool; Based on the Euclidean distance, a target number of safe samples are selected as representative samples in ascending order, and the complement sample images of the representative samples in the safe candidate pool are determined as the remaining samples; The remaining samples, selected in descending order of normalized entropy value as the target number, are used as marginal samples, and the sample memory increment for a new defect category is constructed by combining these representative samples; the formula for representing the normalized entropy value includes: ； in, For the corresponding sample image The normalized entropy value; The number of known defect categories; The number of new defect categories; For traversing the defect categories; For sample images Belongs to the The probability of a defect category; Add the sample memory increment to the sample memory; A training sequence is obtained by performing joint dynamic sampling based on sample images containing new defect categories and the sample memory. The training sequence contains sample images with known defect category identifiers and new defect category identifiers. Deep feature extraction and joint loss calculation are performed based on the training sequence to obtain the joint loss, which includes memory-aware gradient coordination loss and adaptive semantic decoupling loss. Adjust the weight parameters of the perception model based on the joint loss to obtain an intermediate model; Obtain the accuracy index of the intermediate model on the distribution of known defect categories and new defect categories; Based on the accuracy metric and the amount of data in the sample images of the training sequence, the adaptive aggregation weights of the UAV are obtained, and the intermediate model is aggregated based on the adaptive aggregation weights to update the perception model.

2. The UAV inspection image recognition method based on federated continuous learning according to claim 1, characterized in that, The method further includes: Based on the updated perception model, feature reprojection is performed on sample images of defect categories in the sample memory to obtain projected features; The average value of the projected features is normalized twice to update the class centers of the defect categories. The characterization formula includes: ； in, The class center for the defect category; An index for inspection tasks; For training rounds; For drones Sample memory bank In the middle, it belongs to the defect category. A collection of sample images; for The total number of sample images in the dataset; To utilize the updated perception model parameters For sample images The extracted high-dimensional feature vectors are used to characterize the sample features; for Norms are used to perform spatial normalization operations on eigenvectors.

3. The UAV inspection image recognition method based on federated continuous learning according to claim 1, characterized in that, The known defect category identifiers include: The first sub-identifier used to characterize a representative sample of a known defect category; A second sub-identifier used to characterize edge samples of known defect categories.

4. The UAV inspection image recognition method based on federated continuous learning according to claim 1, characterized in that, The steps for obtaining the memory-aware gradient coordination loss include: Obtain the absolute value of the gradient of the sample images in the training sequence; Based on known defect category identifiers and new defect category identifiers, the absolute gradient values ​​corresponding to the sample images in the training sequence are grouped, and the average value of the absolute gradient values ​​in the group is obtained as the local mean. An adaptive normalization factor is obtained based on the local mean, and a memory-aware gradient coordination loss is obtained by combining the absolute value of the gradient. The representation formula of the memory-aware gradient coordination loss includes: ； in, For memory-aware gradient coordination loss; The total number of sample images in the training sequence; For inspection tasks Next, the The absolute value of the gradient generated by a sample image at its corresponding ground truth node in the output layer of the model classifier; A minimal constant introduced to prevent the denominator from being zero; The standard cross-entropy loss function; For inspection tasks The next One sample image; for Corresponding defect category identifier; For drones During inspection mission The The model parameters of the perception model after each training round; To utilize right The predicted probability distribution output after feature extraction; This is the adaptive normalization factor.

5. The UAV inspection image recognition method based on federated continuous learning according to claim 1, characterized in that, The steps for obtaining the adaptive semantic decoupling loss include: Construct a hybrid target vector based on the perception model to be updated; Use the Sigmoid response of the perceptual model to be updated as a soft label for the known defect category dimension; Obtain the true unique heat hard label for the new defect category; The binary cross-entropy between the updated perceptual model's prediction result and the mixed target vector, obtained based on the hybrid target vector, the soft label, and the one-hot hard label, is used as the adaptive semantic decoupling loss. The formula for representing the adaptive semantic decoupling loss includes: ； in, For adaptive semantic decoupling loss; The total number of sample images in the training sequence; This represents the total number of known defect categories. This represents the total number of new defect categories. For traversing the defect categories; For the perception model to be updated, the first The sample image in the th... The predicted response value for the defect category is the independent probability strength of the Logits output by the perception model to be updated after being mapped by the Sigmoid function. hour, Used to characterize uniquely heated hard labels. For the true unique hot hard label of the first The indicator value for the defect category; hour, Used to characterize soft tags. This is the balance weight constant; In order to carry out inspection tasks The following is the first The mixed target vector constructed from sample images In the The values ​​that the elements of a class can take.

6. The UAV inspection image recognition method based on federated continuous learning according to claim 1, characterized in that, The steps for obtaining the accuracy metrics of the intermediate model on the distributions of known defect categories and new defect categories include: Obtain a dynamic evaluation set that is physically isolated from the sample memory; By performing closed-loop inference on the dynamic evaluation set using an intermediate model, the accuracy index of the intermediate model on the distribution of known defect categories and new defect categories is obtained. The formula for representing the accuracy index includes: ； in, For inspection tasks The In each training round, the drone Accuracy metrics; For dynamic evaluation set The total number of test samples in the sample; for A single test sample image of a drone and its corresponding real defect category identifier. This is a single test sample image of a drone. for The corresponding actual defect category identifier; For drones During inspection mission The The model parameters of the perception model after each training round; For the perceptual model to sample images The predicted probability distribution output after feature extraction; The function is used to extract the category index corresponding to the maximum value in the predicted probability distribution, i.e., the predicted category of the perceptual model; This is an indicator function that takes the value 1 when the logical condition within the parentheses is true, and 0 otherwise.

7. The UAV inspection image recognition method based on federated continuous learning according to claim 1, characterized in that, The formula for representing the adaptive aggregation weight of the UAV includes: ； in, For inspection tasks The In each training round, the drone Adaptive aggregation weights; For inspection tasks The In each training round, the drone Accuracy metrics; For inspection tasks Chinese drones The amount of data in the sample images in the training sequence used for model training; For inspection tasks The set of drone nodes selected for parameter aggregation; For set The index.

8. A drone inspection image recognition system based on federated continuous learning, characterized in that, include: Inspection recognition module (1), the inspection recognition module (1) is used to perform inspection image recognition through a preset perception model; Model update module (2), the model update module (2) is used to train the model using sample images collected by UAV inspection and update the perception model, the model update module (2) includes: Task mutation detection submodule (10), the task mutation detection submodule (10) is used to obtain the average prediction entropy of the sample image through the probability distribution of the sample image in the known defect category; A dual-path memory management submodule (20) is configured to determine a new defect category and add sample images containing the new defect category to the sample memory bank in response to the difference in the average prediction entropy between adjacent training rounds being outside the mutation threshold. The sensory model is used to extract features from the sample images of the new defect category to obtain sample features; Obtain the class center of the sample features in the feature space, and select safe samples among the sample images of the new defect category based on the Euclidean distance between the sample features and the class centers to construct a safe candidate pool; Based on the Euclidean distance, a target number of safe samples are selected as representative samples in ascending order, and the complement sample images of the representative samples in the safe candidate pool are determined as the remaining samples; The remaining samples, selected in descending order of normalized entropy value as the target number, are used as marginal samples, and the sample memory increment for a new defect category is constructed by combining these representative samples; the formula for representing the normalized entropy value includes: ； in, For the corresponding sample image The normalized entropy value; The number of known defect categories; The number of new defect categories; For traversing the defect categories; For sample images Belongs to the The probability of a defect category; Add the sample memory increment to the sample memory; Training batch construction submodule (30) is used to perform joint dynamic sampling based on sample images containing new defect categories and the sample memory to obtain training sequences, the training sequences containing sample images with known defect category identifiers and new defect category identifiers; The joint loss calculation submodule (40) is used to perform deep feature extraction and joint loss calculation based on the training sequence to obtain the joint loss, which includes memory-aware gradient coordination loss and adaptive semantic decoupling loss. The local model optimization submodule (50) is used to adjust the weight parameters of the perceptual model based on the joint loss to obtain an intermediate model; The dynamic evaluation set management submodule (60) is used to obtain the accuracy index of the intermediate model on the distribution of known defect categories and new defect categories; The global adaptive aggregation submodule (70) is used to obtain the adaptive aggregation weight of the UAV based on the accuracy index and the amount of data of sample images in the training sequence, and to perform parameter aggregation on the intermediate model based on the adaptive aggregation weight to update the perception model.

Images