An unmanned aerial vehicle image recognition method and system based on multi-task unified learning, a terminal, and a storage medium

Abstract

This invention relates to the field of UAV vision technology and discloses a UAV image recognition method, system, terminal, and storage medium based on multi-task unified learning. The method includes: preprocessing an image dataset to obtain a target dataset, and acquiring task labels for label loading to obtain target samples; performing scene understanding based on the target dataset using an initial scene understanding model to obtain an initial scene understanding result; calculating a loss based on the task type, target samples, and the initial scene understanding result to obtain an initial task loss; scaling the initial task loss, and optimizing, iteratively training, and compressing the structure of the initial scene understanding model to obtain a target scene understanding model. This invention calculates the loss based on the task classification result and uses an adaptive loss adjustment mechanism to scale the loss, using the scaled task loss to optimize model training, thereby improving the accuracy of UAV visual scene understanding.

Description

Technical Field

[0001] This invention relates to the field of UAV vision technology, and in particular to a UAV image recognition method, system, terminal, and computer-readable storage medium based on multi-task unified learning. Background Technology

[0002] As low-altitude application scenarios become increasingly complex, drones often need to simultaneously acquire scene-level semantic information, target-level positional information, and pixel-level region information during a single flight. A single vision task can no longer support a complete perception loop. To meet the needs of multi-task collaborative perception, multi-task learning methods have been adopted for drone scene understanding. These methods incorporate multiple vision tasks into a unified network, sharing feature extraction layers and configuring different task branches to achieve multiple types of results output in a single inference.

[0003] Existing UAV visual scene understanding methods typically involve setting up a shared encoder at the front end to extract general representations, setting up corresponding decoding structures at the back end for tasks such as classification, detection, and segmentation, and then uniformly training the model using a joint loss function to obtain a model for scene understanding. However, this UAV visual scene understanding method lacks effective handling of differences in data distribution between tasks, which can easily lead to a long-term bias in the training process towards tasks with more abundant data or stronger supervision, weakening the learning effect of other tasks, resulting in poor model performance and inaccurate scene understanding after training.

[0004] Therefore, existing technologies still need to be improved and developed. Summary of the Invention

[0005] The main objective of this invention is to provide a UAV image recognition method, system, terminal, and computer-readable storage medium based on multi-task unified learning. This invention aims to address the problem that existing UAV image recognition methods based on multi-task unified learning lack effective handling of differences in data distribution between tasks, which can easily lead to a long-term bias in the training process towards tasks with more abundant data or stronger supervision, weakening the learning effect of other tasks and resulting in poor model performance and inaccurate scene understanding after training.

[0006] To achieve the above objectives, the present invention provides a UAV image recognition method based on multi-task unified learning, comprising the following steps: A drone image dataset is acquired, preprocessed and classified to obtain a first dataset, a second dataset, and a third dataset, and task labels are obtained. Based on the task labels, the first dataset, the second dataset, and the third dataset, label loading is performed to obtain the target sample. An initial scene understanding model is determined. The initial scene understanding model is used to perform scene understanding processing based on the first dataset, the second dataset, and the third dataset to obtain an initial scene understanding result. The task moving average loss is obtained. The task progress is calculated based on the task moving average loss to obtain a task learning progress ratio. The task is classified based on the task learning progress ratio to obtain a task type. The initial task loss is obtained by calculating the loss based on the task type, the target sample, and the initial scene understanding result. An initial configuration is performed to obtain an optimized configuration. Based on the optimized configuration, the initial task loss is scaled using a dynamic weighting method or an uncertain weighting method. If the dynamic weighting method is used, a weight sensitivity coefficient is obtained. The task weight is calculated based on the task learning progress ratio and the weight sensitivity coefficient to obtain the target task weight. The initial task loss is then weighted and optimized based on the target task weight to obtain the target task loss. If the aforementioned uncertain weighting method is adopted, the logarithmic variance parameter is obtained, and the initial task loss is subjected to uncertain weighting processing based on the logarithmic variance parameter to obtain the target task loss; The shared parameters and task-specific branch parameters of the initial scene understanding model are optimized according to the target task loss to obtain a first scene understanding model. The first scene understanding model is iteratively trained to obtain a second scene understanding model. The second scene understanding model is structurally compressed to obtain a target scene understanding model. An image to be recognized is obtained, and the image to be recognized is recognized using the target scene understanding model to obtain the target scene understanding result.

[0007] Optionally, the UAV image recognition method based on multi-task unified learning, wherein acquiring the UAV image dataset, preprocessing and classifying the image dataset to obtain a first dataset, a second dataset, and a third dataset, and acquiring task labels, and performing label loading processing based on the task labels, the first dataset, the second dataset, and the third dataset to obtain target samples, specifically includes: The image dataset of the UAV is acquired, and the image dataset is subjected to size adjustment, tensor quantization and normalization to obtain the target dataset; The target dataset is classified to obtain a first dataset, a second dataset, and a third dataset. Obtain task labels, and perform sample generation processing based on the first dataset, the second dataset, the third dataset, and the task labels to obtain target samples.

[0008] Optionally, in the UAV image recognition method based on multi-task unified learning, the target sample includes a first sample, a second sample, and a third sample. The task labels include image-level category labels, target category labels, bounding box labels, and pixel-level semantic labels; The step of obtaining task labels involves performing sample generation processing based on the first dataset, the second dataset, the third dataset, and the task labels to obtain target samples, specifically including: The first sample is obtained by constructing a first sample based on the image-level category label and the first dataset. The second sample is obtained by constructing a second sample based on the target category label, the bounding box label, and the second dataset. The third sample is obtained by constructing a third sample based on the pixel-level semantic labels and the third dataset; The target sample is obtained based on the first sample, the second sample, and the third sample.

[0009] Optionally, in the UAV image recognition method based on multi-task unified learning, the initial scene understanding result includes scene category probability, target category, bounding box, and pixel-level semantic segmentation result; The task moving average loss includes the current task moving average loss and the historical task moving average loss; The process of determining an initial scene understanding model involves using this model to perform scene understanding processing based on the first, second, and third datasets to obtain an initial scene understanding result. A task moving average loss is then obtained. Based on the task moving average loss, task progress is calculated to obtain a task learning progress ratio. Based on the task learning progress ratio, task classification is performed to obtain the task type. Finally, based on the task type, the target sample, and the initial scene understanding result, a loss calculation is performed to obtain the initial task loss. Specifically, this includes: An initial scene understanding model is determined, and scene understanding processing is performed based on the first dataset, the second dataset, and the third dataset using the initial scene understanding model to obtain the scene category probability, the target category, the bounding box, and the pixel-level semantic segmentation result. Obtain the current task's moving average loss and the historical task's moving average loss, and perform task progress calculation based on the current task's moving average loss and the historical task's moving average loss to obtain the task learning progress ratio; Obtain the initial sampling probability, and update the initial sampling probability according to the task learning progress ratio to obtain the target sampling probability; The task type is obtained by performing random sampling based on the target sampling probability. The target samples are filtered according to the task type to obtain task samples, and loss calculation is performed based on the task samples and the initial scene understanding results to obtain the initial task loss.

[0010] Optionally, the UAV image recognition method based on multi-task unified learning, wherein the initial configuration processing is performed to obtain an optimized configuration, and the initial task loss is scaled using a dynamic weighting method or an uncertain weighting method according to the optimized configuration; if the dynamic weighting method is used, a weight sensitivity coefficient is obtained, and task weights are calculated based on the task learning progress ratio and the weight sensitivity coefficient to obtain the target task weight; and the initial task loss is weighted and optimized based on the target task weight to obtain the target task loss, specifically including: Obtain user requirements, perform initial configuration processing based on the user requirements to obtain optimized configuration, and scale the initial task loss using a dynamic weighting method or an uncertain weighting method based on the optimized configuration; If the dynamic weighting method is adopted, the weight sensitivity coefficient is obtained, and the weight prototype generation process is performed based on the task learning progress ratio and the weight sensitivity coefficient to obtain the initial weight prototype. The initial weight prototype is normalized, pruned, and smoothed to obtain the target task weight. The initial task loss is then weighted and optimized based on the target task weight to obtain the target task loss.

[0011] Optionally, the UAV image recognition method based on multi-task unified learning, wherein if the uncertain weighting method is used, the logarithmic variance parameter is obtained, and the initial task loss is subjected to uncertain weighting processing based on the logarithmic variance parameter to obtain the target task loss, specifically includes: If the aforementioned uncertain weighting method is adopted, the internal loss balance coefficient is obtained, and the internal loss is calculated based on the internal loss balance coefficient and the initial task loss to obtain the total task loss. Obtain the logarithmic variance parameter, and perform uncertainty-weighted loss calculation based on the logarithmic variance parameter and the total task loss to obtain the target task loss.

[0012] Optionally, in the UAV image recognition method based on multi-task unified learning, the target scene understanding result includes scene-level understanding result, target-level understanding result, and pixel-level understanding result; The process involves optimizing the shared parameters and task-specific branch parameters of the initial scene understanding model based on the target task loss to obtain a first scene understanding model; iteratively training the first scene understanding model to obtain a second scene understanding model; performing structural compression on the second scene understanding model to obtain a target scene understanding model; acquiring the image to be recognized; and performing recognition processing on the image to be recognized using the target scene understanding model to obtain the target scene understanding result. Specifically, this includes: The shared parameters of the initial scene understanding model are subjected to conflict determination and projection correction to obtain a corrected scene understanding model; The shared parameters and task-specific branch parameters of the modified scene understanding model are optimized based on the target task loss to obtain the first scene understanding model; The first scene understanding model is iteratively trained, and when the iterative training is completed, the second scene understanding model is obtained. Obtain the embedding dimension and the feedforward hidden layer dimension, and perform a first compression process on the feedforward network parameters of the second scene understanding model based on the embedding dimension and the feedforward hidden layer dimension to obtain a parameter compressed model; Obtain the number of encoding layers, and perform a second compression process on the cumulative feedforward parameters of the encoding layers of the parameter compression model based on the number of encoding layers, the embedding dimension, and the feedforward hidden layer dimension to obtain the target scene understanding model; An image to be identified is acquired, and the image is processed by the target scene understanding model to obtain the scene-level understanding result, the target-level understanding result, and the pixel-level understanding result.

[0013] Furthermore, to achieve the above objectives, the present invention also provides a UAV image recognition system based on multi-task unified learning, wherein the UAV image recognition system based on multi-task unified learning includes: The sample construction module is used to acquire image datasets of UAVs, preprocess and classify the image datasets to obtain a first dataset, a second dataset, and a third dataset, and acquire task labels. Based on the task labels, the first dataset, the second dataset, and the third dataset, the module performs label loading processing to obtain target samples. The loss calculation module is used to determine an initial scene understanding model, perform scene understanding processing based on the first dataset, the second dataset, and the third dataset using the initial scene understanding model to obtain an initial scene understanding result, obtain a task moving average loss, perform task progress calculation processing based on the task moving average loss to obtain a task learning progress ratio, perform task classification processing based on the task learning progress ratio to obtain a task type, and perform loss calculation processing based on the task type, the target sample, and the initial scene understanding result to obtain an initial task loss. The dynamic weight calculation module is used to perform initial configuration processing to obtain an optimized configuration. Based on the optimized configuration, the initial task loss is scaled using a dynamic weighting method or an uncertain weighting method. If the dynamic weighting method is used, a weight sensitivity coefficient is obtained. Based on the task learning progress ratio and the weight sensitivity coefficient, the task weight is calculated to obtain the target task weight. Based on the target task weight, the initial task loss is weighted and optimized to obtain the target task loss. An uncertain weighting module is used to obtain a logarithmic variance parameter if the uncertain weighting method is adopted, and to perform uncertain weighting processing on the initial task loss based on the logarithmic variance parameter to obtain the target task loss; The scene understanding module is used to optimize the shared parameters and task-specific branch parameters of the initial scene understanding model according to the target task loss to obtain a first scene understanding model, perform iterative training on the first scene understanding model to obtain a second scene understanding model, perform structural compression on the second scene understanding model to obtain a target scene understanding model, acquire the image to be recognized, and perform recognition processing on the image to be recognized through the target scene understanding model to obtain the target scene understanding result.

[0014] Furthermore, to achieve the above objectives, the present invention also provides a terminal, wherein the terminal includes: a memory, a processor, and a UAV image recognition program based on multi-task unified learning stored in the memory and executable on the processor, wherein when the UAV image recognition program based on multi-task unified learning is executed by the processor, it implements the steps of the UAV image recognition method based on multi-task unified learning as described above.

[0015] Furthermore, to achieve the above objectives, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a UAV image recognition program based on multi-task unified learning, and the UAV image recognition program based on multi-task unified learning, when executed by a processor, implements the steps of the UAV image recognition method based on multi-task unified learning as described above.

[0016] In this invention, an image dataset of a drone is acquired, preprocessed, and classified to obtain a first dataset, a second dataset, and a third dataset. Task labels are then acquired, and label loading is performed based on the task labels, the first dataset, the second dataset, and the third dataset to obtain target samples. An initial scene understanding model is determined, and scene understanding is performed using this model based on the first dataset, the second dataset, and the third dataset to obtain an initial scene understanding result. A task moving average loss is obtained, and task progress is calculated based on the task moving average loss to obtain a task learning progress ratio. Task classification is performed based on the task learning progress ratio to obtain the task type, and loss calculation is performed based on the task type, the target samples, and the initial scene understanding result to obtain an initial task loss. Initial configuration processing is performed to obtain an optimized configuration, and a dynamic weighting method or an uncertain weighting method is used based on the optimized configuration. The initial task loss is scaled. If the dynamic weighting method is used, a weight sensitivity coefficient is obtained. Task weights are calculated based on the task learning progress ratio and the weight sensitivity coefficient to obtain the target task weight. The initial task loss is then weighted and optimized based on the target task weight to obtain the target task loss. If the uncertain weighting method is used, a log-variance parameter is obtained. The initial task loss is then weighted based on the log-variance parameter to obtain the target task loss. The shared parameters and task-specific branch parameters of the initial scene understanding model are optimized based on the target task loss to obtain a first scene understanding model. The first scene understanding model is iteratively trained to obtain a second scene understanding model. The second scene understanding model undergoes structural compression to obtain the target scene understanding model. An image to be recognized is then acquired, and the target scene understanding model is used to recognize the image to obtain the target scene understanding result. This invention calculates the loss based on the task classification result and uses an adaptive loss adjustment mechanism to scale the loss. The scaled task loss is then used to optimize model training, improving the accuracy of UAV visual scene understanding. Attached Figure Description

[0017] Figure 1 This is a flowchart of a preferred embodiment of the UAV image recognition method based on multi-task unified learning of the present invention; Figure 2 This is a system flowchart of the UAV image recognition method based on multi-task unified learning according to the present invention; Figure 3 This is a dynamic sampling flowchart of the UAV image recognition method based on multi-task unified learning according to the present invention; Figure 4This is a flowchart of the dynamic weight calculation process for the UAV image recognition method based on multi-task unified learning according to the present invention. Figure 5 This is an uncertainty weighted flowchart of the UAV image recognition method based on multi-task unified learning in this invention; Figure 6 This is a flowchart of the online gradient conflict mitigation process for the UAV image recognition method based on multi-task unified learning, as described in this invention. Figure 7 This is a flowchart of the UAV scene understanding process based on the UAV image recognition method of the present invention, which is based on multi-task unified learning. Figure 8 This is a structural diagram of a preferred embodiment of the UAV image recognition system based on multi-task unified learning of the present invention; Figure 9 This is a structural diagram of a preferred embodiment of the terminal of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0019] As low-altitude application scenarios become increasingly complex, drones often need to simultaneously acquire scene-level semantic information, target-level positional information, and pixel-level region information during a single flight. A single vision task can no longer support a complete perception loop. To meet the needs of multi-task collaborative perception, multi-task learning methods have been adopted for drone scene understanding. These methods incorporate multiple vision tasks into a unified network, sharing feature extraction layers and configuring different task branches to achieve multiple types of results output in a single inference.

[0020] Existing UAV visual scene understanding methods typically involve setting up a shared encoder at the front end to extract general representations, setting up corresponding decoding structures at the back end for tasks such as classification, detection, and segmentation, and then uniformly training the model using a joint loss function to obtain a model for scene understanding. However, this UAV visual scene understanding method lacks effective handling of differences in data distribution between tasks, which can easily lead to a long-term bias in the training process towards tasks with more abundant data or stronger supervision, weakening the learning effect of other tasks, resulting in poor model performance and inaccurate scene understanding after training.

[0021] To address the aforementioned issues, this invention proposes a UAV image recognition method based on multi-task unified learning. The method calculates the loss based on the task classification results and uses an adaptive loss adjustment mechanism to scale the loss. The scaled task loss is then used to optimize model training, thereby improving the accuracy of UAV visual scene understanding.

[0022] The UAV image recognition method based on multi-task unified learning described in the preferred embodiment of the present invention, such as... Figure 1 As shown, the UAV image recognition method based on multi-task unified learning includes the following steps: Step S10: Obtain the image dataset of the UAV, perform preprocessing and classification on the image dataset to obtain a first dataset, a second dataset, and a third dataset, and obtain the task label. Perform label loading processing based on the task label, the first dataset, the second dataset, and the third dataset to obtain the target sample.

[0023] This invention is applicable to collaborative perception tasks of scene classification, target detection, and semantic segmentation in UAV visual scene understanding. The input during the training phase consists of UAV or aerial top-down images and their corresponding annotations from classification, detection, and segmentation data streams. The input during the inference phase consists of single or batch images of the UAV to be tested. The system processing chain includes shared feature extraction, unified embedding generation, classification decoding, detection decoding, segmentation decoding, task sampling scheduling, loss weight adjustment, shared gradient conflict mitigation, and lightweight deployment optimization. The system output includes scene category probabilities, target categories and bounding boxes, pixel-level semantic segmentation results, and lightweight model parameters for edge deployment.

[0024] Specifically, an image dataset of the UAV is acquired, and the image dataset is subjected to size adjustment, tensor quantization, and normalization to obtain the target dataset.

[0025] First, read the drone image; then, perform size adjustment, tensor quantization, and necessary normalization preprocessing according to the preset input specifications.

[0026] Further, the target dataset is classified to obtain a first dataset, a second dataset, and a third dataset; the target samples include a first sample, a second sample, and a third sample; the task labels include image-level category labels, target category labels, bounding box labels, and pixel-level semantic labels; a first sample is constructed based on the image-level category labels and the first dataset to obtain the first sample; a second sample is constructed based on the target category labels, the bounding box labels, and the second dataset to obtain the second sample; a third sample is constructed based on the pixel-level semantic labels and the third dataset to obtain the third sample; and the target sample is obtained based on the first sample, the second sample, and the third sample.

[0027] The preprocessed images are categorized into three task types: classification, detection, and segmentation, establishing classification, detection, and segmentation data streams. Label information for each task is loaded synchronously: image-level category labels for classification, object category and bounding box labels for detection, and pixel-level semantic labels for segmentation.

[0028] According to the task type, the labels and images are organized into mini-batch samples that can be called in subsequent iterations. Each sample contains a drone image and the label information corresponding to the task.

[0029] Step S20: Determine the initial scene understanding model, perform scene understanding processing based on the first dataset, the second dataset, and the third dataset using the initial scene understanding model to obtain the initial scene understanding result, obtain the task moving average loss, perform task progress calculation processing based on the task moving average loss to obtain the task learning progress ratio, perform task classification processing based on the task learning progress ratio to obtain the task type, and perform loss calculation processing based on the task type, the target sample, and the initial scene understanding result to obtain the initial task loss.

[0030] like Figure 2 As shown, this invention inputs an image dataset into a shared encoder, which decodes the dataset using a classification decoder, a detection decoder, and a segmentation decoder, respectively, to obtain three types of scene understanding results.

[0031] Specifically, the task moving average loss includes the current task moving average loss and the historical task moving average loss; the initial scene understanding result includes scene category probability, target category, bounding box, and pixel-level semantic segmentation result; an initial scene understanding model is determined, and scene understanding processing is performed using the initial scene understanding model based on the first dataset, the second dataset, and the third dataset to obtain the scene category probability, the target category, the bounding box, and the pixel-level semantic segmentation result.

[0032] First, drone images from the target dataset are received. Multi-scale features are extracted using a shared encoder to generate a unified embedding. Shallow features retain texture, edge, and local structural information, while higher-level features provide stronger global semantic representation. This invention aligns and interacts features at different levels using a shared encoder to generate a unified embedding that can be used by three types of tasks, avoiding redundant computation caused by deploying multiple models in series.

[0033] Next, scene classification decoding is performed. The high-level semantic representation of the shared encoding output is used as input, and the scene category probability is obtained through global aggregation and category mapping. This is used to complete the overall recognition of aerial scenes such as airports, ports, roads, and residential areas.

[0034] Target detection and decoding are performed using a query-driven ensemble prediction approach. Target category discrimination and bounding box regression are completed on multi-scale features, adapting to the localization requirements of UAV scenarios with large differences in target scale, dense small targets, and complex backgrounds. The detection branch maintains a unified interface with the shared encoder, thereby achieving instance-level perception without adding an extra backbone.

[0035] Semantic segmentation and decoding are performed, and pixel-level semantic segmentation results are output using shared coding features and high-resolution fusion features. These results are used to depict the fine boundaries and spatial distribution of areas such as roads, buildings, water bodies, and vegetation. The segmentation branches maintain regional consistency while taking edge details into account, which helps to enhance the UAV's overall understanding of complex low-altitude scenes.

[0036] Further, the current task moving average loss and the historical task moving average loss are obtained, and the task progress is calculated based on the current task moving average loss and the historical task moving average loss to obtain the task learning progress ratio; the initial sampling probability is obtained, and the initial sampling probability is updated based on the task learning progress ratio to obtain the target sampling probability.

[0037] like Figure 2 As shown, this invention uses dynamic sampling to periodically update the sampling probability for model parameter optimization.

[0038] like Figure 3 As shown, this invention uses a dynamic sampler for sampling, based on the total training loss. L EMA (Exponential Moving Average) loss calculation is performed to obtain the task moving average loss; the task learning progress ratio is then calculated based on the task moving average loss. The sampling probabilities of each task are periodically updated based on the learning progress of the task, and the classification task probabilities are output. Detection task probability And the probability of splitting tasks Select the current iteration task based on the sampling probability of each task, and calculate the corresponding task loss.

[0039] First, update the task's moving average loss. The specific process for updating the task's moving average loss is as follows: ; in, Indicates task In the The moving average loss of the next iteration. Indicates task In the The initial loss for the next iteration is calculated from the output of the current task branch. This represents the smoothing coefficient, used to control the fusion ratio of historical loss and current loss. Indicates the first The task selected in the next iteration. The value belongs to the task set. Task set It includes three types of tasks: classification, detection, and segmentation.

[0040] The specific calculation method for the task learning progress ratio is as follows: ; in, This represents the learning progress ratio for the task; a higher value indicates that the task still requires more training resources. This indicates gradient clipping. Indicates task The moving average loss at the previous statistical time point is used to measure the rate of descent of the task at different stages. Indicates task The moving average loss, This indicates that the learning progress of the task is compared to the upper bound. This represents the lower bound of the task learning progress ratio. The upper and lower bounds of the task learning progress ratio are used to suppress abnormal fluctuations.

[0041] The task sampling distribution is updated based on the task learning progress ratio. The specific process is as follows: ; in, Indicates the intermediate sampling probability. Indicates task The learning progress compared to the progress sensitivity coefficient The adjusted result Represents any task during summation and normalization. j The learning progress compared to the progress sensitivity coefficient The adjusted result This represents the final sampling probability, used to determine the probability of the next stage task occurring. This represents the mixing coefficient for a uniform distribution, used to avoid excessive bias in the sampling distribution. This represents a set of tasks.

[0042] Further, random sampling is performed based on the target sampling probability to obtain the task type; the target sample is then filtered based on the task type to obtain the task sample; and loss calculation is performed based on the task sample and the initial scene understanding result to obtain the initial task loss.

[0043] The final sampling probabilities obtained from the dynamic sampling process constitute the discrete probability distributions for the three tasks: classification, detection, and segmentation. During training, random sampling is performed according to this probability distribution to determine the current iteration task; for example, classification, detection, or segmentation is selected based on the probability interval into which the random number falls. After the task is determined, a small batch of samples is taken from the data stream corresponding to that task, and the loss corresponding to that task is calculated.

[0044] Step S30: Perform initial configuration processing to obtain optimized configuration. Based on the optimized configuration, scale the initial task loss using dynamic weighting or uncertain weighting. If dynamic weighting is used, obtain the weight sensitivity coefficient. Calculate the task weight based on the task learning progress ratio and the weight sensitivity coefficient to obtain the target task weight. Then, perform weighted optimization processing on the initial task loss based on the target task weight to obtain the target task loss.

[0045] like Figure 2 As shown, this invention uses a dynamic weighting method or an uncertain weighting method to scale the task loss.

[0046] Specifically, user requirements are obtained, initial configuration processing is performed based on the user requirements to obtain an optimized configuration, and the initial task loss is scaled using a dynamic weighting method or an uncertain weighting method based on the optimized configuration.

[0047] The loss weight mode is pre-set according to user needs. One of the two mechanisms, dynamic weighting mechanism and uncertainty weighting mechanism, is selected and enabled. The two mechanisms are not enabled at the same time to avoid double weighting and loss scale coupling.

[0048] Furthermore, if the dynamic weighting method is adopted, the weight sensitivity coefficient is obtained, and a weight prototype generation process is performed based on the task learning progress ratio and the weight sensitivity coefficient to obtain an initial weight prototype; the initial weight prototype is then normalized, pruned, and smoothed to obtain the target task weight, and the initial task loss is then weighted and optimized based on the target task weight to obtain the target task loss.

[0049] The dynamic weighting method calculates dynamic task weights based on the task learning progress. These dynamic task weights are used to scale the task loss, thereby adjusting the intensity of the task's influence on the shared encoder and unified feature representation during backpropagation. When the loss of a task decreases slowly, stagnates, or rebounds, it indicates that the task still needs more training resources, and its corresponding weight can be appropriately increased. Conversely, when the loss of a task decreases rapidly and continuously, it indicates that the task has been sufficiently learned, and its corresponding weight can be appropriately decreased.

[0050] like Figure 4As shown, the dynamic weighting method adjusts the loss contribution based on the task learning progress; specifically, it selects the total loss at the classification task level according to the task type. Total loss at the detection task level Or split the total loss at the task level As the original loss of the current mission According to the learning progress ratio of the task Generate unnormalized weight prototypes Then, normalization and clipping are performed sequentially, followed by a momentum smoothing coefficient. After smoothing, the current task weight is finally obtained. Based on the current task type, classify the current task weight into category task weights. Detection task weight Or split task weights The weights for classification, detection, and segmentation tasks are used to scale the loss of the corresponding tasks.

[0051] First, we construct a weight prototype, the specific process of which is as follows: ; in, This represents the prototype of unnormalized weights, used to construct relative weights. Indicates task The learning progress is compared with the weighted sensitivity coefficient The adjusted result shows that the weight sensitivity coefficient is used to control the degree of non-linearity when the learning progress is mapped to weights. This represents a set of tasks.

[0052] The weight prototype is normalized to the mean, and the specific process is as follows: ; in, Indicates task The normalized task weights are used to ensure the stability of the overall update scale at different stages. This indicates any task when performing mean normalization. j The prototype of the unnormalized weights.

[0053] The normalized task weights are pruned, and the specific process is as follows: ; in, This represents the task weights after pruning, which are then used as the actual weight values ​​for training. and These represent the upper and lower bounds of the weights, respectively, to prevent a particular task from being suppressed or amplified in the long term. This indicates gradient clipping.

[0054] The clipped task weights are then subjected to momentum smoothing, the specific process of which is as follows: ; in, Indicates the first k During the second weight update, the task The current candidate weights obtained after normalization and pruning This represents the momentum smoothing coefficient, used to suppress drastic jumps caused by periodic spikes.

[0055] By iterating through the tasks, a weighted optimization objective is obtained. The specific process is as follows: ; in, Indicates the first The final optimization objective of the next iteration is used as the input for backpropagation. Indicates the first In the next iteration, the currently selected task The corresponding final effective weight, Indicates the first The task selected in the next iteration The original task was lost.

[0056] Step S40: If the uncertainty weighting method is adopted, the logarithmic variance parameter is obtained, and the initial task loss is subjected to uncertainty weighting processing based on the logarithmic variance parameter to obtain the target task loss.

[0057] Specifically, if the uncertainty weighted method is used, the internal loss balance coefficient is obtained, and the internal loss is calculated based on the internal loss balance coefficient and the initial task loss to obtain the total task loss; the logarithmic variance parameter is obtained, and the uncertainty weighted loss is calculated based on the logarithmic variance parameter and the total task loss to obtain the target task loss.

[0058] like Figure 5 As shown, by introducing learnable parameters Total loss at the classification task level Total loss at the detection task level Or split the total loss at the task level Perform adaptive scaling to obtain the effective weights of the current task in the joint optimization objective. Effective weights are used to calculate the total task loss. The target is scaled to form the current iteration optimization objective, which is used as the input for backpropagation to achieve shared parameter updates.

[0059] First, calculate the internal loss of the task. The specific process is as follows: ; in, This indicates the total loss at the detection task level. Indicates the detection and classification loss. Indicates the regression loss of the detection. This represents the total loss at the task level. Represents the classification cross-entropy loss. This represents the segmentation auxiliary loss. and This represents the loss balance coefficient within the task.

[0060] The specific process for calculating the multi-task uncertainty-weighted total loss is as follows: ; in, This represents the weighted total loss based on uncertainty across multiple tasks. Indicates task Effective weights , This represents the total loss for a classification, detection, or segmentation task. This represents learnable parameters, which in this invention are tasks. The logarithmic variance parameter.

[0061] By iterating through the tasks, we obtain an uncertainty-weighted objective: ; in, Indicates the first i The final optimization objective of the next iteration is used as the input for backpropagation. Indicates task The log-variance parameter, Indicates the first i Sub-iteration task The total loss.

[0062] If a task exhibits high noise levels, labeling uncertainty, or optimization difficulty during training, its corresponding uncertainty parameter increases, reducing the effective weight of the task and weakening its contribution. Conversely, if a task exhibits low noise levels, its corresponding uncertainty parameter decreases, increasing the effective weight of the task and increasing its contribution.

[0063] Step S50: Optimize the shared parameters and task-specific branch parameters of the initial scene understanding model according to the target task loss to obtain a first scene understanding model. Iteratively train the first scene understanding model to obtain a second scene understanding model. Perform structural compression on the second scene understanding model to obtain a target scene understanding model. Obtain the image to be recognized. Perform recognition processing on the image to be recognized through the target scene understanding model to obtain the target scene understanding result.

[0064] Specifically, the target scene understanding results include scene-level understanding results, target-level understanding results, and pixel-level understanding results; conflict determination and projection correction are performed on the shared parameters of the initial scene understanding model to obtain a corrected scene understanding model; the shared parameters and task-specific branch parameters of the corrected scene understanding model are optimized according to the target task loss to obtain a first scene understanding model.

[0065] like Figure 2 As shown, online gradient conflict mitigation is performed on the shared parameters of the model.

[0066] The scaled loss is used as the backpropagation input to update the model parameters. Specifically, the gradients of shared parameters need to undergo conflict resolution and online projection correction before updating; the task-specific branch parameters corresponding to the current task are directly updated according to the gradients generated by the scaled loss; branches not related to the current task generally do not participate in the current iteration update. Task-specific branch parameters refer to parameters dedicated to the output of a specific task and are not shared with other tasks. In this invention, these mainly include parameters in the classification decoder, the detection decoder, and the segmentation decoder.

[0067] like Figure 6 As shown, conflict detection, reference gradient reading, online projection correction, and gradient memory update are performed on the task gradients with shared parameters to reduce the negative transfer effect between tasks. Specifically, the original gradient on the shared parameters is obtained based on the current task loss. ; the original gradient Gradient memory vectors shared with classification, detection, and segmentation tasks on parameters , and Perform a direction comparison; if the inner product of the two is negative, it indicates a conflict between the current update direction and the historical representative direction of the corresponding task, which may lead to a decrease in the performance of the task. In this case, perform projection correction to remove the conflict. The harmful component along the conflict direction; if the inner product is non-negative, it means the direction does not conflict, and the current gradient can be retained; the corrected shared gradient. This is used to update shared parameters for the optimizer, and the gradient memory for the current task is then updated based on the current gradient.

[0068] The shared gradient projection correction is performed as follows: ; in, Indicates the first j Post-projection task gradient, Indicates the reference gradient direction. Represents the numerically stable term. This represents the loss due to the squared error. This indicates transpose.

[0069] The specific process for gated preheating and online projection is as follows: ; in, This represents the shared gradient after conflict mitigation. Indicates task The original gradient over the shared parameters, Indicates the number of iterations. Indicates the number of preheating steps. Indicates the online projection operator, Indicates task j The gradient memory vector.

[0070] The gradient memory is updated, and the specific process is as follows: ; in, Indicates the first k Task during the next update Gradient memory vector, This represents the memory decay coefficient.

[0071] After conflict detection and online projection correction of shared parameters, the scaled task loss is used to supervise the update of shared parameters and task-specific branch parameters.

[0072] Further, the first scene understanding model is iteratively trained, and when the iterative training is completed, a second scene understanding model is obtained; the embedding dimension and the feedforward hidden layer dimension are obtained, and the number of feedforward network parameters of the second scene understanding model is compressed in the first way according to the embedding dimension and the feedforward hidden layer dimension to obtain a parameter compressed model; the number of encoding layers is obtained, and the cumulative feedforward parameters of the encoding layer of the parameter compressed model are compressed in the second way according to the number of encoding layers, the embedding dimension and the feedforward hidden layer dimension to obtain a target scene understanding model.

[0073] Iterate through the training process until the model is fully trained, then deploy the model.

[0074] When the deployment target is an airborne terminal of a drone, an edge computing device, or other platform constrained by computing power, power consumption, storage capacity, and real-time response requirements, a lightweight implementation method is adopted. In such scenarios, to improve practical deployment feasibility, a lightweight scheme combining shared encoding compression, task decoding compression, and knowledge distillation can be used. In server-side scenarios with sufficient computing resources or offline training and evaluation scenarios, a non-lightweight implementation method can be adopted.

[0075] The lightweight design for edge deployment mainly includes two parts: structural compression and knowledge distillation. Specifically, it compresses the shared encoding and task decoding structures and introduces teacher model output and shared features to distill and supervise student models.

[0076] The approximate formula satisfied by the parameters of a two-layer feedforward network is: ; in, This represents an approximate value of the parameters of a single-layer feedforward network. Indicates the embedding dimension. This represents the dimension of the feedforward hidden layer.

[0077] The approximate formula satisfied by the cumulative feedforward parameters of the coding layer is: ; in, This represents an approximate value of the cumulative parameters of a multilayer feedforward network. Indicates the number of coding layers.

[0078] The specific process for calculating the loss during fractional distillation is as follows: ; in, The classification distillation loss is used to constrain the student model's classification output distribution to approximate the teacher model's. KL represents the calculation of the KL (Kullback-Leibler) divergence. Indicates the distillation temperature coefficient. This represents the activation function. Represents the classification logits of the student model. This represents the classification logits of the teacher model (logits are intermediate results predicted by the model).

[0079] The specific process for calculating the total loss during lightweight model training is as follows: ; in, Indicates the total loss of the target. Indicates the current task The supervised training loss (the main task loss calculated by the student model based on the real labels). Indicates the weight of the distillation process. Indicates the weight of the distillation process. Represents the characteristic distillation weight, This represents the segmentation distillation loss, used to constrain the student model's pixel-level semantic segmentation output to be close to that of the teacher model. This represents the feature distillation loss, used to constrain the consistency between the student model and the teacher model on shared features or neck features.

[0080] Furthermore, an image to be identified is acquired, and the image to be identified is processed by the target scene understanding model to obtain the scene-level understanding result, the target-level understanding result, and the pixel-level understanding result.

[0081] Input the image to be tested, and output the understanding results of three types of visual scenes simultaneously in a single inference.

[0082] The results of the three types of visual scene understanding include: 1. Scene-level results: The scene category to which the image under test belongs and its corresponding probability; 2. Target-level results: Category information of targets in the image and the location of their corresponding bounding boxes; 3. Pixel-level results: Semantic segmentation maps of areas such as roads, buildings, water bodies, and vegetation.

[0083] In summary, as Figure 7 As shown, the steps for drone scene understanding in this invention include: Step S1: Construct three training data streams for classification, detection, and segmentation, and complete UAV image reading, size normalization, and label loading; Step S2: Feed the input image into the shared encoder to extract multi-scale common features and generate a unified embedding; Step S3: The unified embedding is fed into the classification decoder, detection decoder and segmentation decoder respectively to obtain the scene category probability, target category and bounding box, and pixel-level semantic segmentation results; Step S4: During the training phase, select the current iteration task based on the task probability output by the dynamic sampling module, and calculate the corresponding task loss; Step S5: Select dynamic weights or uncertainty weighting according to the enabled strategy, and scale the loss of the current task so that the influence of different tasks in joint training remains adjustable. Step S6: Perform conflict detection and online projection correction on the gradients of shared parameters, and directly update the parameters of task-specific branches according to the current task supervision; Step S7: When using a lightweight implementation, the shared encoding and task decoding structures are compressed, and the teacher model output and shared features are introduced to supervise the distillation of the student model. Step S8: After completing the model training, deploy the model to the UAV end or edge computing device, and input the image to be tested to simultaneously output the understanding results of three types of visual scenes in a single inference.

[0084] The technical effects of this invention include: 1. Construct a unified multi-task learning framework for UAV visual scene understanding, and set up a shared encoder, classification decoder, detection decoder and segmentation decoder in the same model so that scene-level, target-level and pixel-level results can be output synchronously from a single forward inference. 2. In the joint training phase, a dynamic sampling mechanism based on training feedback is introduced to adaptively adjust the frequency of task samples entering the shared network according to the training status of each task, in order to alleviate the problem of uneven task updates. 3. Set up a loss weight adjustment mechanism for multi-task optimization, and adopt a dynamic weight scheme or an uncertain weighting scheme to coordinate the loss scale of different tasks and improve training stability. 4. Set up an online gradient conflict mitigation mechanism for shared parameters to reduce negative transfer between tasks through conflict determination and gradient correction; 5. Based on the above unified learning framework, a lightweight implementation scheme is formed. By compressing the shared encoding structure, simplifying the task decoding structure, and combining knowledge distillation, the overall perception accuracy and edge deployment efficiency are balanced.

[0085] This invention calculates the loss based on the task classification results and uses an adaptive loss adjustment mechanism to scale the loss. The scaled task loss is then used to optimize model training, thereby improving the accuracy of UAV visual scene understanding.

[0086] Alternative embodiments of the present invention include: 1. The shared encoder can be replaced by a convolutional backbone, a hierarchical visual Transformer backbone, or a hybrid convolutional and attention backbone. As long as a single shared representation is still maintained for the three types of tasks to call together, it belongs to the equivalent implementation of the present invention. 2. The scheduling signal during dynamic sampling can be replaced by the loss reduction rate, the change rate of validation set indicators, the sliding accuracy improvement rate, or the training stability score. As long as the frequency of task occurrence is still adaptively adjusted according to training feedback, it belongs to the variant of this invention. 3. Either dynamic weighting method or uncertainty weighting method can be chosen to scale the loss. Alternatively, the combination of losses within a task can be replaced with equal weight combination, segmented weight combination, or normalized combination. As long as the loss scale balance of multiple tasks is still achieved, it is an equivalent implementation of the present invention. 4. The reference gradient in the online gradient correction process can be replaced by the historical gradient memory vector with the short window average gradient, the task center gradient, or the shared parameter subspace gradient. As long as conflict determination and projection correction are still performed on the shared gradient, it belongs to the variation of the present invention. The online gradient conflict mitigation may be replaced by gradient clipping, direction constraint, gradient decoupling, shared layer grouping correction, or other projection correction strategies. 5. Lightweight deployment of the model can be achieved by using compressed coding layers, reducing channel width, replacing segmentation decoding structure, output distillation, feature distillation, or combinations thereof. As long as resource consumption is reduced and overall performance is maintained while the multi-task unified learning framework is preserved, it is an equivalent implementation of the present invention.

[0087] Furthermore, such as Figure 8 As shown, based on the above-mentioned UAV image recognition method based on multi-task unified learning, the present invention also provides a UAV image recognition system based on multi-task unified learning, wherein the UAV image recognition system based on multi-task unified learning includes: The sample construction module 51 is used to acquire an image dataset of the UAV, perform preprocessing and classification processing on the image dataset to obtain a first dataset, a second dataset, and a third dataset, and acquire task labels. Based on the task labels, the first dataset, the second dataset, and the third dataset, the module performs label loading processing to obtain target samples. The loss calculation module 52 is used to determine an initial scene understanding model, perform scene understanding processing based on the first dataset, the second dataset, and the third dataset using the initial scene understanding model to obtain an initial scene understanding result, obtain a task moving average loss, perform task progress calculation processing based on the task moving average loss to obtain a task learning progress ratio, perform task classification processing based on the task learning progress ratio to obtain a task type, and perform loss calculation processing based on the task type, the target sample, and the initial scene understanding result to obtain an initial task loss. The dynamic weight calculation module 53 is used to perform initial configuration processing to obtain an optimized configuration. Based on the optimized configuration, the initial task loss is scaled using a dynamic weighting method or an uncertain weighting method. If the dynamic weighting method is used, a weight sensitivity coefficient is obtained. Based on the task learning progress ratio and the weight sensitivity coefficient, the task weight is calculated to obtain the target task weight. Based on the target task weight, the initial task loss is weighted and optimized to obtain the target task loss. Uncertainty weighting module 54 is used to obtain the logarithmic variance parameter if the uncertainty weighting method is adopted, and to perform uncertainty weighting processing on the initial task loss according to the logarithmic variance parameter to obtain the target task loss; The scene understanding module 55 is used to optimize the shared parameters and task-specific branch parameters of the initial scene understanding model according to the target task loss to obtain a first scene understanding model, perform iterative training on the first scene understanding model to obtain a second scene understanding model, perform structural compression on the second scene understanding model to obtain a target scene understanding model, acquire the image to be recognized, and perform recognition processing on the image to be recognized through the target scene understanding model to obtain the target scene understanding result.

[0088] Furthermore, such as Figure 9 As shown, based on the above-mentioned UAV image recognition method and system based on multi-task unified learning, the present invention also provides a terminal, which includes a processor 10, a memory 20 and a display 30. Figure 9 Only some of the terminal components are shown; however, it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0089] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard drive or memory. In other embodiments, the memory 20 may be an external storage device of the terminal, such as a plug-in hard drive, smart media card (SMC), secure digital card (SD), flash card, etc. Further, the memory 20 may include both internal and external storage devices. The memory 20 is used to store application software and various types of data installed on the terminal, such as the program code installed on the terminal. The memory 20 can also be used to temporarily store data that has been output or will be output. In one embodiment, the memory 20 stores a UAV image recognition program 40 based on multi-task unified learning, which can be executed by the processor 10 to implement the UAV image recognition method based on multi-task unified learning in this application.

[0090] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor, or other data processing chip, used to run program code stored in the memory 20 or process data, such as executing the UAV image recognition method based on multi-task unified learning.

[0091] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display 30 is used to display information on the terminal and to display a visual user interface.

[0092] In one embodiment, when the processor 10 executes the UAV image recognition program 40 based on multi-task unified learning stored in the memory 20, the following steps are performed: A drone image dataset is acquired, preprocessed and classified to obtain a first dataset, a second dataset, and a third dataset, and task labels are obtained. Based on the task labels, the first dataset, the second dataset, and the third dataset, label loading is performed to obtain the target sample. An initial scene understanding model is determined. The initial scene understanding model is used to perform scene understanding processing based on the first dataset, the second dataset, and the third dataset to obtain an initial scene understanding result. The task moving average loss is obtained. The task progress is calculated based on the task moving average loss to obtain a task learning progress ratio. The task is classified based on the task learning progress ratio to obtain a task type. The initial task loss is obtained by calculating the loss based on the task type, the target sample, and the initial scene understanding result. An initial configuration is performed to obtain an optimized configuration. Based on the optimized configuration, the initial task loss is scaled using a dynamic weighting method or an uncertain weighting method. If the dynamic weighting method is used, a weight sensitivity coefficient is obtained. The task weight is calculated based on the task learning progress ratio and the weight sensitivity coefficient to obtain the target task weight. The initial task loss is then weighted and optimized based on the target task weight to obtain the target task loss. If the aforementioned uncertain weighting method is adopted, the logarithmic variance parameter is obtained, and the initial task loss is subjected to uncertain weighting processing based on the logarithmic variance parameter to obtain the target task loss; The shared parameters and task-specific branch parameters of the initial scene understanding model are optimized according to the target task loss to obtain a first scene understanding model. The first scene understanding model is iteratively trained to obtain a second scene understanding model. The second scene understanding model is structurally compressed to obtain a target scene understanding model. An image to be recognized is obtained, and the image to be recognized is recognized using the target scene understanding model to obtain the target scene understanding result.

[0093] The process of acquiring the UAV image dataset, preprocessing and classifying the image dataset to obtain a first dataset, a second dataset, and a third dataset, and acquiring task labels, and then performing label loading based on the task labels, the first dataset, the second dataset, and the third dataset to obtain target samples, specifically includes: The image dataset of the UAV is acquired, and the image dataset is subjected to size adjustment, tensor quantization and normalization to obtain the target dataset; The target dataset is classified to obtain a first dataset, a second dataset, and a third dataset. Obtain task labels, and perform sample generation processing based on the first dataset, the second dataset, the third dataset, and the task labels to obtain target samples.

[0094] The target samples include a first sample, a second sample, and a third sample; The task labels include image-level category labels, target category labels, bounding box labels, and pixel-level semantic labels; The step of obtaining task labels involves performing sample generation processing based on the first dataset, the second dataset, the third dataset, and the task labels to obtain target samples, specifically including: The first sample is obtained by constructing a first sample based on the image-level category label and the first dataset. The second sample is obtained by constructing a second sample based on the target category label, the bounding box label, and the second dataset. The third sample is obtained by constructing a third sample based on the pixel-level semantic labels and the third dataset; The target sample is obtained based on the first sample, the second sample, and the third sample.

[0095] The initial scene understanding results include scene category probability, target category, bounding box, and pixel-level semantic segmentation results; The task moving average loss includes the current task moving average loss and the historical task moving average loss; The process of determining an initial scene understanding model involves using this model to perform scene understanding processing based on the first, second, and third datasets to obtain an initial scene understanding result. A task moving average loss is then obtained. Based on the task moving average loss, task progress is calculated to obtain a task learning progress ratio. Based on the task learning progress ratio, task classification is performed to obtain the task type. Finally, based on the task type, the target sample, and the initial scene understanding result, a loss calculation is performed to obtain the initial task loss. Specifically, this includes: An initial scene understanding model is determined, and scene understanding processing is performed based on the first dataset, the second dataset, and the third dataset using the initial scene understanding model to obtain the scene category probability, the target category, the bounding box, and the pixel-level semantic segmentation result. Obtain the current task's moving average loss and the historical task's moving average loss, and perform task progress calculation based on the current task's moving average loss and the historical task's moving average loss to obtain the task learning progress ratio; Obtain the initial sampling probability, and update the initial sampling probability according to the task learning progress ratio to obtain the target sampling probability; The task type is obtained by performing random sampling based on the target sampling probability. The target samples are filtered according to the task type to obtain task samples, and loss calculation is performed based on the task samples and the initial scene understanding results to obtain the initial task loss.

[0096] The process includes initial configuration processing to obtain an optimized configuration. Based on this optimized configuration, the initial task loss is scaled using either a dynamic weighting method or an uncertain weighting method. If the dynamic weighting method is used, a weight sensitivity coefficient is obtained. Task weights are calculated based on the task learning progress ratio and the weight sensitivity coefficient to obtain the target task weight. Finally, the initial task loss is weighted and optimized based on the target task weight to obtain the target task loss. Specifically, this includes: Obtain user requirements, perform initial configuration processing based on the user requirements to obtain optimized configuration, and scale the initial task loss using a dynamic weighting method or an uncertain weighting method based on the optimized configuration; If the dynamic weighting method is adopted, the weight sensitivity coefficient is obtained, and the weight prototype generation process is performed based on the task learning progress ratio and the weight sensitivity coefficient to obtain the initial weight prototype. The initial weight prototype is normalized, pruned, and smoothed to obtain the target task weight. The initial task loss is then weighted and optimized based on the target task weight to obtain the target task loss.

[0097] Specifically, if the uncertainty weighting method is adopted, the logarithmic variance parameter is obtained, and the initial task loss is subjected to uncertainty weighting processing based on the logarithmic variance parameter to obtain the target task loss, which specifically includes: If the aforementioned uncertain weighting method is adopted, the internal loss balance coefficient is obtained, and the internal loss is calculated based on the internal loss balance coefficient and the initial task loss to obtain the total task loss. Obtain the logarithmic variance parameter, and perform uncertainty-weighted loss calculation based on the logarithmic variance parameter and the total task loss to obtain the target task loss.

[0098] The target scene understanding results include scene-level understanding results, target-level understanding results, and pixel-level understanding results; The process involves optimizing the shared parameters and task-specific branch parameters of the initial scene understanding model based on the target task loss to obtain a first scene understanding model; iteratively training the first scene understanding model to obtain a second scene understanding model; performing structural compression on the second scene understanding model to obtain a target scene understanding model; acquiring the image to be recognized; and performing recognition processing on the image to be recognized using the target scene understanding model to obtain the target scene understanding result. Specifically, this includes: The shared parameters of the initial scene understanding model are subjected to conflict determination and projection correction to obtain a corrected scene understanding model; The shared parameters and task-specific branch parameters of the modified scene understanding model are optimized based on the target task loss to obtain the first scene understanding model; The first scene understanding model is iteratively trained, and when the iterative training is completed, the second scene understanding model is obtained. Obtain the embedding dimension and the feedforward hidden layer dimension, and perform a first compression process on the feedforward network parameters of the second scene understanding model based on the embedding dimension and the feedforward hidden layer dimension to obtain a parameter compressed model; Obtain the number of encoding layers, and perform a second compression process on the cumulative feedforward parameters of the encoding layers of the parameter compression model based on the number of encoding layers, the embedding dimension, and the feedforward hidden layer dimension to obtain the target scene understanding model; An image to be identified is acquired, and the image is processed by the target scene understanding model to obtain the scene-level understanding result, the target-level understanding result, and the pixel-level understanding result.

[0099] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a UAV image recognition program based on multi-task unified learning, and the UAV image recognition program based on multi-task unified learning implements the steps of the UAV image recognition method based on multi-task unified learning as described above when executed by a processor.

[0100] In summary, this invention provides a UAV image recognition method, system, and terminal based on multi-task unified learning. The method includes: acquiring a UAV image dataset; preprocessing and classifying the image dataset to obtain a first dataset, a second dataset, and a third dataset; acquiring task labels; performing label loading processing based on the task labels, the first dataset, the second dataset, and the third dataset to obtain target samples; determining an initial scene understanding model; performing scene understanding processing based on the first dataset, the second dataset, and the third dataset using the initial scene understanding model to obtain an initial scene understanding result; acquiring a task moving average loss; performing task progress calculation processing based on the task moving average loss to obtain a task learning progress ratio; performing task classification processing based on the task learning progress ratio to obtain a task type; and performing loss calculation processing based on the task type, the target samples, and the initial scene understanding result to obtain an initial task loss; performing initial configuration processing to obtain an optimized configuration; and then... The optimization configuration employs either a dynamic weighting method or an uncertain weighting method to scale the initial task loss. If the dynamic weighting method is used, a weight sensitivity coefficient is obtained, and task weights are calculated based on the task learning progress ratio and the weight sensitivity coefficient to obtain the target task weight. The initial task loss is then weighted and optimized based on the target task weight to obtain the target task loss. If the uncertain weighting method is used, a logarithmic variance parameter is obtained, and the initial task loss is weighted uncertainly based on the logarithmic variance parameter to obtain the target task loss. The shared parameters and task-specific branch parameters of the initial scene understanding model are optimized based on the target task loss to obtain a first scene understanding model. The first scene understanding model is iteratively trained to obtain a second scene understanding model. The second scene understanding model undergoes structural compression to obtain the target scene understanding model. An image to be recognized is then acquired, and the target scene understanding model is used to recognize the image to obtain the target scene understanding result. This invention calculates the loss based on the task classification result and uses an adaptive loss adjustment mechanism to scale the loss. The scaled task loss is then used to optimize model training, improving the accuracy of UAV visual scene understanding.

[0101] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal. Unless otherwise specified, an element defined by the phrase "comprising one" does not exclude the presence of other identical elements in the process, method, article, or terminal that includes that element.

[0102] Of course, those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.). The program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The computer-readable storage medium can be a memory, magnetic disk, optical disk, etc.

[0103] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A method for UAV image recognition based on multi-task unified learning, characterized in that, The unmanned aerial vehicle image recognition method based on multi-task unified learning comprises: An image dataset of an unmanned aerial vehicle is acquired, pre-processing and classification processing are performed on the image dataset to obtain a first dataset, a second dataset and a third dataset, and a task label is acquired, label loading processing is performed according to the task label, the first dataset, the second dataset and the third dataset to obtain a target sample; An initial scene understanding model is determined, scene understanding processing is performed on the first dataset, the second dataset and the third dataset by the initial scene understanding model to obtain an initial scene understanding result, a task sliding average loss is acquired, task progress calculation processing is performed according to the task sliding average loss to obtain a task learning progress ratio, task classification processing is performed according to the task learning progress ratio to obtain a task type, and loss calculation processing is performed according to the task type, the target sample and the initial scene understanding result to obtain an initial task loss; Initial configuration processing is performed to obtain an optimized configuration, the initial task loss is scaled by using a dynamic weight method or an uncertain weighting method according to the optimized configuration, if the dynamic weight method is used, a weight sensitivity coefficient is acquired, task weight calculation processing is performed according to the task learning progress ratio and the weight sensitivity coefficient to obtain a target task weight, and the initial task loss is weighted and optimized according to the target task weight to obtain a target task loss; If the uncertain weighting method is used, a logarithmic variance parameter is acquired, and the initial task loss is weighted according to the uncertainty according to the logarithmic variance parameter to obtain the target task loss; The shared parameters and the task-specific branch parameters of the initial scene understanding model are respectively optimized according to the target task loss to obtain a first scene understanding model, iterative training processing is performed on the first scene understanding model to obtain a second scene understanding model, structure compression processing is performed on the second scene understanding model to obtain a target scene understanding model, and an image to be recognized is acquired, and the target scene understanding model is used to perform recognition processing on the image to be recognized to obtain a target scene understanding result. 2.The multi-task unified learning based UAV image recognition method of claim 1, wherein, The image dataset of the unmanned aerial vehicle is acquired, pre-processing and classification processing are performed on the image dataset to obtain a first dataset, a second dataset and a third dataset, and a task label is acquired, label loading processing is performed according to the task label, the first dataset, the second dataset and the third dataset to obtain a target sample, and specifically comprises: An image dataset of an unmanned aerial vehicle is acquired, size adjustment processing, tensorization processing and normalization processing are performed on the image dataset to obtain a target dataset; The target dataset is classified to obtain a first dataset, a second dataset and a third dataset; A task label is acquired, sample generation processing is performed according to the first dataset, the second dataset, the third dataset and the task label to obtain a target sample. 3.The multi-task unified learning based UAV image recognition method of claim 2, wherein, The target sample comprises a first sample, a second sample and a third sample; The task labels include image-level category labels, target category labels, bounding box labels, and pixel-level semantic labels; The step of obtaining task labels involves performing sample generation processing based on the first dataset, the second dataset, the third dataset, and the task labels to obtain target samples, specifically including: The first sample is obtained by constructing a first sample based on the image-level category label and the first dataset. The second sample is obtained by constructing a second sample based on the target category label, the bounding box label, and the second dataset. The third sample is obtained by constructing a third sample based on the pixel-level semantic labels and the third dataset; The target sample is obtained based on the first sample, the second sample, and the third sample. 4.The multi-task unified learning based UAV image recognition method of claim 1, wherein, The initial scene understanding results include scene category probability, target category, bounding box, and pixel-level semantic segmentation results; The task moving average loss includes the current task moving average loss and the historical task moving average loss; The process of determining an initial scene understanding model involves using this model to perform scene understanding processing based on the first, second, and third datasets to obtain an initial scene understanding result. A task moving average loss is then obtained. Based on the task moving average loss, task progress is calculated to obtain a task learning progress ratio. Based on the task learning progress ratio, task classification is performed to obtain the task type. Finally, based on the task type, the target sample, and the initial scene understanding result, a loss calculation is performed to obtain the initial task loss. Specifically, this includes: An initial scene understanding model is determined, and scene understanding processing is performed based on the first dataset, the second dataset, and the third dataset using the initial scene understanding model to obtain the scene category probability, the target category, the bounding box, and the pixel-level semantic segmentation result. Obtain the current task's moving average loss and the historical task's moving average loss, and perform task progress calculation based on the current task's moving average loss and the historical task's moving average loss to obtain the task learning progress ratio; Obtain the initial sampling probability, and update the initial sampling probability according to the task learning progress ratio to obtain the target sampling probability; The task type is obtained by performing random sampling based on the target sampling probability. The target samples are filtered according to the task type to obtain task samples, and loss calculation is performed based on the task samples and the initial scene understanding results to obtain the initial task loss. 5.The multi-task unified learning based unmanned aerial vehicle image recognition method of claim 1, wherein, The initial configuration process is performed to obtain an optimized configuration. Based on the optimized configuration, the initial task loss is scaled using either a dynamic weighting method or an uncertain weighting method. If the dynamic weighting method is used, a weight sensitivity coefficient is obtained. Task weights are calculated based on the task learning progress ratio and the weight sensitivity coefficient to obtain the target task weight. The initial task loss is then weighted and optimized based on the target task weight to obtain the target task loss. Specifically, this includes: Obtain user requirements, perform initial configuration processing based on the user requirements to obtain optimized configuration, and scale the initial task loss using a dynamic weighting method or an uncertain weighting method based on the optimized configuration; If the dynamic weighting method is adopted, the weight sensitivity coefficient is obtained, and the weight prototype generation process is performed based on the task learning progress ratio and the weight sensitivity coefficient to obtain the initial weight prototype. The initial weight prototype is normalized, pruned, and smoothed to obtain the target task weight. The initial task loss is then weighted and optimized based on the target task weight to obtain the target task loss. 6.The multi-task unified learning based UAV image recognition method of claim 1, wherein, If the uncertainty weighting method is adopted, the logarithmic variance parameter is obtained, and the initial task loss is subjected to uncertainty weighting processing based on the logarithmic variance parameter to obtain the target task loss, specifically including: If the aforementioned uncertain weighting method is adopted, the internal loss balance coefficient is obtained, and the internal loss is calculated based on the internal loss balance coefficient and the initial task loss to obtain the total task loss. Obtain the logarithmic variance parameter, and perform uncertainty-weighted loss calculation based on the logarithmic variance parameter and the total task loss to obtain the target task loss. 7.The multi-task unified learning based unmanned aerial vehicle image recognition method of claim 1, wherein, The target scene understanding results include scene-level understanding results, target-level understanding results, and pixel-level understanding results; The process involves optimizing the shared parameters and task-specific branch parameters of the initial scene understanding model based on the target task loss to obtain a first scene understanding model; iteratively training the first scene understanding model to obtain a second scene understanding model; performing structural compression on the second scene understanding model to obtain a target scene understanding model; acquiring the image to be recognized; and performing recognition processing on the image to be recognized using the target scene understanding model to obtain the target scene understanding result. Specifically, this includes: The shared parameters of the initial scene understanding model are subjected to conflict determination and projection correction to obtain a corrected scene understanding model; The shared parameters and task-specific branch parameters of the modified scene understanding model are optimized based on the target task loss to obtain the first scene understanding model; The first scene understanding model is iteratively trained, and when the iterative training is completed, the second scene understanding model is obtained. Obtain the embedding dimension and the feedforward hidden layer dimension, and perform a first compression process on the feedforward network parameters of the second scene understanding model based on the embedding dimension and the feedforward hidden layer dimension to obtain a parameter compressed model; Obtain the number of encoding layers, and perform a second compression process on the cumulative feedforward parameters of the encoding layers of the parameter compression model based on the number of encoding layers, the embedding dimension, and the feedforward hidden layer dimension to obtain the target scene understanding model; An image to be identified is acquired, and the image is processed by the target scene understanding model to obtain the scene-level understanding result, the target-level understanding result, and the pixel-level understanding result. 8.A UAV image recognition system based on multi-task unified learning, characterized in that, The UAV image recognition system based on multi-task unified learning includes: The sample construction module is used to acquire image datasets of UAVs, preprocess and classify the image datasets to obtain a first dataset, a second dataset, and a third dataset, and acquire task labels. Based on the task labels, the first dataset, the second dataset, and the third dataset, the module performs label loading processing to obtain target samples. The loss calculation module is used to determine an initial scene understanding model, perform scene understanding processing based on the first dataset, the second dataset, and the third dataset using the initial scene understanding model to obtain an initial scene understanding result, obtain a task moving average loss, perform task progress calculation processing based on the task moving average loss to obtain a task learning progress ratio, perform task classification processing based on the task learning progress ratio to obtain a task type, and perform loss calculation processing based on the task type, the target sample, and the initial scene understanding result to obtain an initial task loss. The dynamic weight calculation module is used to perform initial configuration processing to obtain an optimized configuration. Based on the optimized configuration, the initial task loss is scaled using a dynamic weighting method or an uncertain weighting method. If the dynamic weighting method is used, a weight sensitivity coefficient is obtained. Based on the task learning progress ratio and the weight sensitivity coefficient, the task weight is calculated to obtain the target task weight. Based on the target task weight, the initial task loss is weighted and optimized to obtain the target task loss. An uncertain weighting module is used to obtain a logarithmic variance parameter if the uncertain weighting method is adopted, and to perform uncertain weighting processing on the initial task loss based on the logarithmic variance parameter to obtain the target task loss; The scene understanding module is used to optimize the shared parameters and task-specific branch parameters of the initial scene understanding model according to the target task loss to obtain a first scene understanding model, perform iterative training on the first scene understanding model to obtain a second scene understanding model, perform structural compression on the second scene understanding model to obtain a target scene understanding model, acquire the image to be recognized, and perform recognition processing on the image to be recognized through the target scene understanding model to obtain the target scene understanding result.

9. A terminal, characterized by comprising: The terminal includes a memory, a processor, and a UAV image recognition program based on multi-task unified learning stored in the memory and executable on the processor. When the processor executes the UAV image recognition program based on multi-task unified learning, it implements the steps of the UAV image recognition method based on multi-task unified learning as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a UAV image recognition program based on multi-task unified learning. When the UAV image recognition program based on multi-task unified learning is executed by a processor, it implements the steps of the UAV image recognition method based on multi-task unified learning as described in any one of claims 1-7.

Images