An air-ground cooperative target detection and feature analysis system and method

Abstract

The application provides an air-ground cooperative target detection and feature analysis system and method, which is through collection, data cleaning, enhancement, labeling, feature extraction and classification, corresponding to obtain an air sample database and a ground sample database, processing and screening the air feature sample dataset and the ground feature sample dataset through a sample processing module, inputting the screened air sample dataset and the screened ground feature sample dataset into a model for training, obtaining a trained air target recognition model and a trained ground target analysis model; optimizing the two models respectively, and realizing target detection and feature analysis by combining multi-modal feature fusion. Compared with the prior art, by combining the air target recognition model and the ground target analysis model and multi-modal feature fusion, and based on application scene training optimization, the target detection and feature analysis are realized, and the target detection precision and feature analysis accuracy are improved.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, and in particular to an air-ground collaborative target detection and feature analysis system and method. Background Technology

[0002] Artificial intelligence (AI)-based methods have become the mainstream technology in the field of object detection, demonstrating outstanding performance in various application scenarios due to their high accuracy, real-time performance, and rapid iterative optimization. Traditional object detection methods struggle to adapt to changes in complex environments, while deep learning-based object detection technology can not only autonomously learn high-dimensional features but also combine multi-scale perception, contextual information fusion, and temporal modeling to achieve more accurate object recognition and behavior analysis.

[0003] Currently, there are two main technical approaches to mainstream target detection systems: UAV-borne target detection based on small AI models and small target detection based on large AI models. Both have their own advantages but limitations. When used independently, they are difficult to achieve efficient and accurate target detection and behavior analysis in practical applications.

[0004] Large models, due to their increased complexity and number of parameters, often struggle to focus on smaller targets when dealing with large-scale scenes. Such models are easily drawn to large-scale elements or prominent features in the environment when faced with complex backgrounds and diverse targets, thus neglecting smaller, subtle targets. Furthermore, the performance limitations of AI accelerator cards make it exceptionally difficult to deploy large models on drones.

[0005] While small models can achieve rapid response in specific application scenarios, they suffer from severe deficiencies in deep analysis capabilities, making it difficult to perform fine-grained feature analysis of targets. Even if a small model can detect the presence of a target, it cannot provide sufficient information regarding the target's behavioral patterns, feature descriptions, or contextual understanding in complex scenarios.

[0006] When performing missions, drones not only need real-time detection but also higher-level intelligent analysis capabilities to cope with complex situations in dynamic environments. Separating these two aspects for independent use will inevitably lead to the loss of some functions. This necessitates that system design consider combining the advantages of small and large models to further promote the development of drone technology and ensure its efficient operation in diverse application scenarios.

[0007] Therefore, providing an air-ground cooperative target detection and feature analysis system and method to solve the above problems is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0008] The purpose of this invention is to provide an air-to-ground cooperative target detection and feature analysis system. This system is simple in structure, safe, effective, reliable, and easy to operate. It can achieve high-resolution, efficient, real-time detection, action prediction, and appearance analysis of small targets from the perspective of UAVs. By establishing an airborne target detection algorithm dataset, corresponding small models are trained for application scenarios and deployed on the edge. According to task requirements, a training set database is established for practical application functions such as target intent, behavior patterns, appearance description, and threat analysis. The edge-side large language model is fine-tuned to achieve more accurate human-computer interaction.

[0009] Based on the above objectives, the technical solution provided by the present invention is as follows:

[0010] An air-ground cooperative target detection and feature analysis system includes:

[0011] An aerial sample database is used to collect multi-source heterogeneous aerial raw data and then perform data cleaning, enhancement, annotation, feature extraction and classification on the multi-source heterogeneous aerial raw data in sequence to obtain an aerial feature sample training set, an aerial feature sample validation set and an aerial feature sample test set.

[0012] The ground sample database is used to collect multi-source heterogeneous ground raw data and then perform data cleaning, enhancement, annotation, feature extraction and classification on the multi-source heterogeneous ground raw data in sequence to obtain a ground feature sample training set, a ground feature sample validation set and a ground feature sample test set.

[0013] The sample processing and filtering module is used to process and filter the air feature sample training set, air feature sample verification set, and air feature sample test set in the air sample database and the ground feature sample training set, ground feature sample verification set, and ground feature sample test set in the ground sample database according to the application scenario, so as to obtain the filtered air feature sample training set, filtered air feature sample verification set, filtered air feature sample test set, as well as the filtered ground feature sample training set, filtered ground feature sample verification set, and filtered ground feature sample test set;

[0014] The model training module is used to construct and train an air target recognition model based on the selected air feature sample training set, the selected air feature sample validation set, and the selected air feature sample test set, and to construct and train a ground target analysis model based on the selected ground feature sample training set, the selected ground feature sample validation set, and the selected ground feature sample test set, so as to obtain the trained air target recognition model and the trained ground target analysis model.

[0015] The collaborative detection and analysis module is used to optimize the trained air target recognition model by enhancing the target detection algorithm based on the frequency domain features of air targets; to optimize the trained ground target analysis model by adaptive fine-tuning based on the scene; and to realize target detection and feature analysis based on the optimized air target recognition model, the optimized ground target analysis model, and the multimodal feature fusion module.

[0016] Preferably, the aerial sample database includes:

[0017] The first data acquisition module is used to acquire raw aerial target data;

[0018] The first data cleaning and enhancement module is used to perform data enhancement on the original aerial target data after data cleaning to obtain the first enhanced aerial target data.

[0019] The first data annotation module is used to annotate the first enhanced aerial target data using annotation tools to obtain annotated aerial target data.

[0020] The first feature extraction module is used to extract features from the labeled aerial target data to obtain the first aerial features;

[0021] The first classification module is used to classify the first aerial feature into the aerial feature sample training set, the aerial feature sample validation set, and the aerial feature sample test set.

[0022] Preferably, the ground sample database includes:

[0023] The second data acquisition module is used to collect raw ground target data;

[0024] The second data cleaning and enhancement module is used to perform data enhancement on the original ground target data after data cleaning to obtain the second enhanced ground target data.

[0025] The second data annotation module is used to annotate the second enhanced aerial target data using annotation tools to obtain annotated ground target data.

[0026] The second feature extraction module is used to extract features from the labeled ground target data to obtain the second ground features;

[0027] The second classification module is used to classify the second ground feature into the ground feature sample training set, the ground feature sample validation set, and the ground feature sample test set.

[0028] Preferably, the sample processing and screening module includes: a sample acquisition module, a sample supplementation module, and a sample screening module;

[0029] The sample acquisition module is used to acquire additional aerial feature sample datasets and additional ground feature sample datasets according to the application scenario.

[0030] The sample supplementation module is used to supplement the air feature sample training set, air feature sample validation set, and air feature sample test set with additional air feature sample datasets, respectively, to obtain supplemented air feature sample training set, supplemented air feature sample validation set, and supplemented air feature sample test set;

[0031] The sample supplementation module is also used to supplement the ground feature sample training set, ground feature sample validation set and ground feature sample test set with additional ground feature sample datasets respectively, so as to obtain the supplemented ground feature sample training set, the supplemented ground feature sample validation set and the supplemented ground feature sample test set.

[0032] The sample screening module is used to screen the supplemented aerial feature sample training set, the supplemented aerial feature sample verification set, and the supplemented aerial feature sample test set respectively, so as to obtain the corresponding screened aerial feature sample training set, screened aerial feature sample verification set, and screened aerial feature sample test set.

[0033] The sample screening module is also used to screen the supplemented ground feature sample training set, the supplemented ground feature sample validation set, and the supplemented ground feature sample test set respectively, so as to obtain the corresponding screened ground feature sample training set, screened ground feature sample validation set, and screened ground feature sample test set.

[0034] Preferably, the model training module includes: an aerial target recognition model training module and a ground target analysis model training module;

[0035] The aerial target recognition model training module is used to construct an initial aerial target recognition model, input the selected aerial feature sample training set, the selected aerial feature sample verification set, and the selected aerial feature sample test set into the initial aerial target recognition model for training, so as to correct the initial aerial target recognition model, obtain the trained aerial target recognition model, and deploy the trained aerial target recognition model.

[0036] The ground target analysis model training module is used to construct an initial ground target analysis model, adjust training parameters to optimize the initial ground target analysis model, input the selected ground feature sample training set, the selected ground feature sample validation set, and the selected ground feature sample test set into the optimized initial ground target analysis model for training, and after multiple iterations, evaluate the performance of the optimized initial ground target analysis model according to preset indicators, and use the optimized initial ground target analysis model with the best performance as the trained ground target analysis model.

[0037] Preferably, the collaborative detection and analysis module includes: an airborne target frequency domain feature enhancement module, a scene adaptive fine-tuning module, and a multimodal feature fusion module;

[0038] The air target frequency domain feature enhancement module is used to enhance the multi-mode frequency domain features of the trained air target recognition model. It reshapes the multi-mode frequency domain features through content awareness, extracts different technical features, and couples and enhances the different technical features to optimize the trained air target recognition model.

[0039] The scene adaptive fine-tuning module is used to perform joint visual-language optimization of the trained ground target analysis model according to the scene adaptive adjustment parameters.

[0040] The multimodal feature fusion module is used to couple and enhance the optimized aerial target recognition model and the optimized ground target analysis model to achieve target detection and feature analysis.

[0041] A method for air-ground cooperative target detection and feature analysis includes the following steps:

[0042] After collecting multi-source heterogeneous aerial raw data, the multi-source heterogeneous aerial raw data is sequentially cleaned, enhanced, labeled, feature extracted and classified to obtain an aerial feature sample training set, an aerial feature sample validation set and an aerial feature sample test set;

[0043] After collecting multi-source heterogeneous ground raw data, the multi-source heterogeneous ground raw data are sequentially cleaned, enhanced, labeled, feature extracted and classified to obtain ground feature sample training set, ground feature sample validation set and ground feature sample test set;

[0044] According to the application scenario, the training set, validation set, and test set of air feature samples in the air sample database and the training set, validation set, and test set of ground feature samples in the ground sample database are processed and filtered respectively to obtain the filtered training set, validation set, and test set of air feature samples, as well as the filtered training set, validation set, and test set of ground feature samples.

[0045] An air target recognition model is constructed and trained based on the selected air feature sample training set, the selected air feature sample validation set, and the selected air feature sample test set. A ground target analysis model is constructed and trained based on the selected ground feature sample training set, the selected ground feature sample validation set, and the selected ground feature sample test set, resulting in the trained air target recognition model and the trained ground target analysis model.

[0046] The trained air target recognition model is optimized based on the air target frequency domain feature enhancement target detection algorithm; the trained ground target analysis model is optimized based on scene adaptive fine-tuning; and target detection and feature analysis are achieved based on the optimized air target recognition model, the optimized ground target analysis model, and the multimodal feature fusion module.

[0047] This invention provides an air-ground collaborative target detection and feature analysis system. It involves data acquisition, cleaning, enhancement, annotation, feature extraction, and classification to obtain corresponding air sample databases and ground sample databases. A sample processing module processes and filters the air and ground feature sample datasets. The filtered air and ground feature sample datasets are then input into the model for training, resulting in a trained air target recognition model and a trained ground target analysis model. Both models are then optimized, and multimodal feature fusion is used to achieve target detection and feature analysis.

[0048] Compared to existing technologies, this invention combines an aerial target recognition model and a ground target analysis model with multimodal feature fusion, and trains and optimizes based on application scenarios to achieve target detection and feature analysis, thereby improving the accuracy of target detection and feature analysis.

[0049] This invention also provides an air-ground cooperative target detection and feature analysis method, which, since it shares the same technical concept and solves the same technical problem as this system, should have the same beneficial effects, and will not be described again. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 This is a schematic diagram of the structure of an air-ground cooperative target detection and feature analysis system provided in an embodiment of the present invention;

[0052] Figure 2 A schematic diagram of the structure of the airborne sample database provided in an embodiment of the present invention;

[0053] Figure 3 This is a schematic diagram of the structure of the ground sample database provided in an embodiment of the present invention;

[0054] Figure 4 This is a schematic diagram of the sample processing and screening module provided in an embodiment of the present invention;

[0055] Figure 5 This is a schematic diagram of the structure of the model training module provided in an embodiment of the present invention;

[0056] Figure 6 This is a schematic diagram of the structure of the collaborative detection and analysis module provided in an embodiment of the present invention;

[0057] Figure 7 This is a flowchart of an air-ground cooperative target detection and feature analysis method provided in an embodiment of the present invention. Detailed Implementation

[0058] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0059] The embodiments of this invention are written in a progressive manner.

[0060] This invention provides an air-ground collaborative target detection and feature analysis system. It primarily addresses the technical problems in existing technologies where large models, due to their increased complexity and number of parameters, often struggle to focus on small targets when processing large scenes, and where small models, while capable of rapid response in specific application scenarios, suffer from severe deficiencies in deep analysis capabilities, making it difficult to achieve fine-grained feature analysis of targets.

[0061] like Figure 1 As shown, an air-ground cooperative target detection and feature analysis system includes:

[0062] The aerial sample database is used to collect multi-source heterogeneous aerial raw data and then perform data cleaning, enhancement, annotation, feature extraction and classification on the multi-source heterogeneous aerial raw data in sequence to obtain aerial feature sample training set, aerial feature sample validation set and aerial feature sample test set.

[0063] The ground sample database is used to collect multi-source heterogeneous ground raw data and then perform data cleaning, enhancement, annotation, feature extraction and classification on the multi-source heterogeneous ground raw data in sequence to obtain ground feature sample training set, ground feature sample validation set and ground feature sample test set.

[0064] The sample processing and filtering module is used to process and filter the training set, validation set, and test set of aerial feature samples in the aerial sample database and the training set, validation set, and test set of ground feature samples in the ground sample database according to the application scenario, so as to obtain the filtered training set, validation set, and test set of aerial feature samples, as well as the filtered training set, validation set, and test set of ground feature samples.

[0065] The model training module is used to construct and train an air target recognition model based on the selected air feature sample training set, the selected air feature sample validation set, and the selected air feature sample test set, and to construct and train a ground target analysis model based on the selected ground feature sample training set, the selected ground feature sample validation set, and the selected ground feature sample test set, so as to obtain the trained air target recognition model and the trained ground target analysis model.

[0066] The collaborative detection and analysis module is used to optimize the trained air target recognition model by enhancing the target detection algorithm based on the frequency domain features of air targets; to optimize the trained ground target analysis model by adaptive fine-tuning based on the scene; and to realize target detection and feature analysis based on the optimized air target recognition model, the optimized ground target analysis model, and the multimodal feature fusion module.

[0067] In practical applications, the air-ground collaborative target detection and feature analysis system includes an air sample database, a ground sample database, a sample processing and screening module, a model training module, and a collaborative detection and analysis module.

[0068] In this embodiment, raw data is collected (divided into multi-source sensor data such as visible light, infrared, hyperspectral, and radar according to the multiple sources of raw data), and then the data is cleaned, enhanced and labeled, features are extracted and classified. Based on the source of the raw data (such as drone data collection and ground data collection), an aerial sample dataset is formed and stored in the aerial sample database, and a corresponding ground sample dataset is formed and stored in the ground sample database.

[0069] Based on specific application scenarios, the aerial feature sample datasets (training set, validation set, and test set) and the ground feature sample datasets (training set, validation set, and test set) are processed and filtered respectively. The filtered aerial feature sample datasets and the filtered ground feature sample datasets are then input into the model for training to obtain the aerial target recognition model and the ground target analysis model. After optimizing the aerial target recognition model and the ground target analysis model respectively, multimodal feature fusion is combined to realize target detection and feature analysis.

[0070] like Figure 2 As shown, preferably, the aerial sample database includes:

[0071] The first data acquisition module is used to acquire raw aerial target data;

[0072] The first data cleaning and enhancement module is used to clean and enhance the original aerial target data to obtain the first enhanced aerial target data.

[0073] The first data annotation module is used to annotate the first enhanced aerial target data using annotation tools to obtain annotated aerial target data.

[0074] The first feature extraction module is used to extract features from the labeled aerial target data to obtain the first aerial features;

[0075] The first classification module is used to classify the first aerial features into an aerial feature sample training set, an aerial feature sample validation set, and an aerial feature sample test set.

[0076] In practical applications, the aerial sample database is equipped with a first data acquisition module, a first data cleaning and enhancement module, a first data annotation module, a first feature extraction module, and a first classification module. The first data acquisition module collects raw aerial target data, and then performs data cleaning, data enhancement, data annotation, feature extraction, and data classification in sequence to obtain an aerial feature sample training set, an aerial feature sample validation set, and an aerial feature sample test set.

[0077] In this embodiment, data is collected by multiple source sensors. The collected heterogeneous raw data is preprocessed by data acquisition, cleaning, and enhancement. The target of interest is labeled and classified. Finally, training set, validation set and test set are formed and stored to form an airborne sample database. The sample quality is improved and the sample is expanded throughout the entire life cycle of the airborne target recognition small model.

[0078] like Figure 3 As shown, preferably, the ground sample database includes:

[0079] The second data acquisition module is used to collect raw ground target data;

[0080] The second data cleaning and enhancement module is used to clean and enhance the original ground target data to obtain the second enhanced ground target data.

[0081] The second data annotation module is used to annotate the second enhanced aerial target data using annotation tools to obtain annotated ground target data.

[0082] The second feature extraction module is used to extract features from the labeled ground target data to obtain the second ground features;

[0083] The second classification module is used to classify the second ground features into a ground feature sample training set, a ground feature sample validation set, and a ground feature sample test set.

[0084] In practical applications, the ground sample database is equipped with a second data acquisition module, a second data cleaning and enhancement module, a second data annotation module, a second feature extraction module, and a second classification module. The second data acquisition module collects raw ground target data, and then performs data cleaning, data enhancement, data annotation, feature extraction, and data classification in sequence to obtain a ground feature sample training set, a ground feature sample validation set, and a ground feature sample test set.

[0085] In this embodiment, relevant training datasets are created or collected according to the format of the large model requiring fine-tuning, based on task requirements. Diverse data is collected to cover different situations and scenarios, avoiding model overfitting. The collected or compiled datasets undergo data cleaning, enhancement, format conversion, and segmentation preprocessing before being stored to form a ground sample database.

[0086] like Figure 4 As shown, preferably, the sample processing and screening module includes: a sample acquisition module, a sample supplementation module, and a sample screening module;

[0087] The sample acquisition module is used to acquire additional aerial feature sample datasets and additional ground feature sample datasets according to the application scenario.

[0088] The sample supplementation module is used to supplement the air feature sample training set, air feature sample validation set, and air feature sample test set with additional air feature sample datasets, respectively, to obtain the supplemented air feature sample training set, supplemented air feature sample validation set, and supplemented air feature sample test set;

[0089] The sample supplementation module is also used to supplement the ground feature sample training set, ground feature sample validation set and ground feature sample test set with additional ground feature sample datasets respectively, so as to obtain the supplemented ground feature sample training set, supplemented ground feature sample validation set and supplemented ground feature sample test set;

[0090] The sample filtering module is used to filter the supplemented aerial feature sample training set, the supplemented aerial feature sample validation set, and the supplemented aerial feature sample test set respectively, so as to obtain the corresponding filtered aerial feature sample training set, filtered aerial feature sample validation set, and filtered aerial feature sample test set.

[0091] The sample screening module is also used to screen the supplemented ground feature sample training set, the supplemented ground feature sample validation set, and the supplemented ground feature sample test set respectively, so as to obtain the corresponding screened ground feature sample training set, screened ground feature sample validation set, and screened ground feature sample test set.

[0092] In practical applications, the sample processing and filtering module includes a sample acquisition module, a sample supplementation module, and a sample filtering module. The sample acquisition module collects additional aerial feature sample datasets and additional ground feature sample datasets according to the application scenario. The sample supplementation module supplements the corresponding aerial feature sample datasets and ground feature sample datasets based on the additional aerial feature sample datasets and additional ground feature sample datasets, respectively. After the supplementation is completed, the sample filtering module filters the datasets to obtain the corresponding filtered aerial feature sample datasets (training set, validation set, and test set) and filtered ground feature sample datasets (training set, validation set, and test set).

[0093] In this embodiment, the focus on target types varies depending on the application scenario, requiring the addition and filtering of existing samples in the database based on the application scenario. Data is categorized into multi-source sensor data such as visible light, infrared, hyperspectral, and radar, and acquired on different working platforms and in different working environments to enrich the variety and quantity of training samples.

[0094] like Figure 5 As shown, preferably, the model training module includes: an aerial target recognition model training module and a ground target analysis model training module;

[0095] The aerial target recognition model training module is used to build an initial aerial target recognition model. The selected aerial feature sample training set, the selected aerial feature sample validation set, and the selected aerial feature sample test set are input into the initial aerial target recognition model for training, so as to correct the initial aerial target recognition model, obtain the trained aerial target recognition model, and deploy the trained aerial target recognition model.

[0096] The ground target analysis model training module is used to construct an initial ground target analysis model, adjust training parameters to optimize the initial ground target analysis model, and input the selected ground feature sample training set, the selected ground feature sample validation set, and the selected ground feature sample test set into the optimized initial ground target analysis model for training. After multiple iterations, the performance of the optimized initial ground target analysis model is evaluated according to preset indicators, and the optimized initial ground target analysis model with the best performance is used as the trained ground target analysis model.

[0097] In practical applications, the model training module includes an air target recognition model training module and a ground target analysis model training module. The air target recognition model training module constructs an initial air target recognition model. Selected air feature sample data is input into this initial model for training to refine it, resulting in a trained air target recognition model, which is then deployed. The ground target analysis model training module constructs an initial ground target analysis model, adjusts training parameters to optimize the model, and selects selected ground feature sample data for training. After multiple iterations, the performance of the optimized initial ground target analysis model is tested according to preset evaluation parameters. The model with the best performance is then used as the trained ground target analysis model.

[0098] In this embodiment, the establishment of the small model of the airborne target recognition algorithm specifically involves: training, testing, and optimizing the algorithm on a ground system with abundant computing and storage resources to form a small model. Finally, the trained and corrected small model is deployed to the airborne AI acceleration card.

[0099] The establishment of the large-scale ground-based target recognition algorithm model involves: determining the necessary computing resources, including memory and storage space, to handle large-scale data based on the fine-tuned pre-trained AI large-scale model algorithm; adjusting key parameters such as learning rate, batch size, and training cycle to optimize model performance; training the large-scale model using a ground sample database; evaluating the model through metrics such as accuracy, F1 score, and BLEU score; and forming the large-scale model after multiple iterations.

[0100] like Figure 6As shown, preferably, the collaborative detection and analysis module includes: an airborne target frequency domain feature enhancement module, a scene adaptive fine-tuning module, and a multimodal feature fusion module;

[0101] The air target frequency domain feature enhancement module is used to enhance the multi-mode frequency domain features of the trained air target recognition model. It reshapes the multi-mode frequency domain features through content awareness, extracts different technical features, and couples and enhances the different technical features to optimize the trained air target recognition model.

[0102] The scene-adaptive fine-tuning module is used to perform joint visual-language optimization of the trained ground target analysis model by adaptively adjusting parameters according to the scene.

[0103] The multimodal feature fusion module is used to couple and enhance the optimized aerial target recognition model and the optimized ground target analysis model to achieve target detection and feature analysis.

[0104] In practical applications, the collaborative detection and analysis module includes an airborne target frequency domain feature enhancement module, a scene adaptive fine-tuning module, and a multimodal feature fusion module. The airborne target frequency domain feature enhancement module enhances the multimodal frequency domain features in the trained airborne target recognition model, reshapes them through content awareness, extracts different technical features, and performs coupling enhancement to optimize the trained airborne target recognition model. The scene adaptive fine-tuning module adjusts parameters according to the scene to perform joint visual-language optimization of the trained ground target analysis model. The multimodal feature fusion module couples and enhances the two optimized models to achieve target detection and feature analysis.

[0105] In this embodiment, the Multimodal Feature Fusion Module (MFFM) has a dual function: first, it provides enhanced multimodal features for subsequent multimodal feature extraction networks; second, it provides multimodal input features for multi-scale feature fusion and includes a feedback mechanism. MFFM has three key modules: a feature map hybrid pooling compression module, a frequency domain multi-head aggregation enhancement module, and a residual-enhanced dual-modal fusion module, enabling high-precision image feature extraction and fusion.

[0106] The aerial target frequency domain feature enhancement module employs a lightweight small target detection algorithm based on YOLOv8, called FC-YOLO, which includes three key modules: a frequency domain dynamic core transform module, a content-aware dynamic kernel upsampling module, and a cross-layer high-resolution fine-grained detection layer. The frequency domain dynamic core transform module enhances the frequency domain features of multimodal sensor data, effectively improving the detection capability for small targets. The content-aware dynamic kernel upsampling module reshapes multimodal features through content awareness, extracting different features of small targets from multiple modalities, and then coupling and enhancing these features to improve the feature extraction capability and positioning accuracy of small targets. The cross-layer high-resolution fine-grained detection layer removes the large target detection layer from the algorithm and adds and enhances the small target detection layer, achieving resolution improvement based on multimodal feature enhancement, thereby optimizing the detection capability for small targets. FC-YOLO demonstrates good performance in high-resolution small target detection, improving the accuracy of small target detection under the wide field of view of the UAV.

[0107] The adaptive fine-tuning module specifically performs scene-specific adaptive fine-tuning based on the Qwen2-VL large model. Through systematic modifications to target domain features and enhanced retrieval generation, it significantly improves the model's task accuracy in vertical scenarios. Focusing on deep adaptation of Qwen2-VL's multimodal understanding capabilities, it designs a vision-language joint optimization mechanism. While maintaining Qwen2-VL's original generalization capabilities, it achieves a dual improvement in cross-scene transfer efficiency and accuracy in small target detection and fine-grained feature analysis tasks from an airborne perspective.

[0108] like Figure 7 As shown, an air-to-ground cooperative target detection and feature analysis method includes the following steps:

[0109] S1. After collecting multi-source heterogeneous raw aerial data, the multi-source heterogeneous raw aerial data is sequentially cleaned, enhanced, labeled, feature extracted and classified to obtain aerial feature sample training set, aerial feature sample validation set and aerial feature sample test set;

[0110] S2. After collecting multi-source heterogeneous ground raw data, the multi-source heterogeneous ground raw data are sequentially cleaned, enhanced, labeled, feature extracted and classified to obtain ground feature sample training set, ground feature sample validation set and ground feature sample test set;

[0111] S3. Based on the application scenario, the training set, validation set, and test set of air feature samples in the air sample database and the training set, validation set, and test set of ground feature samples in the ground sample database are processed and filtered respectively to obtain the filtered training set, validation set, and test set of air feature samples, as well as the filtered training set, validation set, and test set of ground feature samples;

[0112] S4. Construct and train an air target recognition model based on the selected air feature sample training set, the selected air feature sample validation set, and the selected air feature sample test set; and construct and train a ground target analysis model based on the selected ground feature sample training set, the selected ground feature sample validation set, and the selected ground feature sample test set, to obtain the trained air target recognition model and the trained ground target analysis model.

[0113] S5. Optimize the trained air target recognition model by enhancing the target detection algorithm based on the air target frequency domain features; optimize the trained ground target analysis model by adaptive fine-tuning based on the scene; realize target detection and feature analysis based on the optimized air target recognition model, the optimized ground target analysis model and the multimodal feature fusion module.

[0114] This invention also provides a method for air-ground cooperative target detection and feature analysis, which shares the same technical concept and has the same technical effect as this system.

[0115] In the embodiments provided in this application, it should be understood that the disclosed methods and systems can be implemented in other ways. The system embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple modules or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, and can be electrical, mechanical, or other forms.

[0116] Furthermore, in the various embodiments of the present invention, each functional module can be fully integrated into a processor, or each module can be a separate device, or two or more modules can be integrated into a device; each functional module in the various embodiments of the present invention can be implemented in hardware or in the form of hardware plus software functional units.

[0117] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by program instructions and related hardware. The aforementioned program instructions can be stored in a computer-readable storage medium. When the program instructions are executed, they perform the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0118] It should be understood that the use of terms such as "system," "device," "unit," and / or "module" in this application is merely one method of distinguishing different components, elements, parts, sections, or assemblies at different levels. However, if other terms can achieve the same purpose, they may be replaced by other expressions.

[0119] As indicated in this application and claims, unless the context clearly indicates otherwise, the words "a," "an," "a," and / or "the" are not specifically singular and may include the plural. Generally, the terms "comprising" and "including" only indicate the inclusion of expressly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements. An element defined by the phrase "comprising an..." does not exclude the presence of other identical elements in the process, method, product, or apparatus that includes the element.

[0120] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0121] If a flowchart is used in this application, it is used to illustrate the operations performed by the system according to embodiments of this application. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, the steps can be processed in reverse order or simultaneously. Furthermore, other operations can be added to these processes, or one or more steps can be removed from them.

[0122] The foregoing has provided a detailed description of an air-ground cooperative target detection and feature analysis system provided by the present invention. The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An air-ground cooperative target detection and feature analysis system, characterized in that, include: An aerial sample database is used to collect multi-source heterogeneous aerial raw data and then perform data cleaning, enhancement, annotation, feature extraction and classification on the multi-source heterogeneous aerial raw data in sequence to obtain an aerial feature sample training set, an aerial feature sample validation set and an aerial feature sample test set. The ground sample database is used to collect multi-source heterogeneous ground raw data and then perform data cleaning, enhancement, annotation, feature extraction and classification on the multi-source heterogeneous ground raw data in sequence to obtain a ground feature sample training set, a ground feature sample validation set and a ground feature sample test set. The sample processing and filtering module is used to process and filter the air feature sample training set, air feature sample verification set, and air feature sample test set in the air sample database and the ground feature sample training set, ground feature sample verification set, and ground feature sample test set in the ground sample database according to the application scenario, so as to obtain the filtered air feature sample training set, filtered air feature sample verification set, filtered air feature sample test set, as well as the filtered ground feature sample training set, filtered ground feature sample verification set, and filtered ground feature sample test set; The model training module is used to construct and train an air target recognition model based on the selected air feature sample training set, the selected air feature sample validation set, and the selected air feature sample test set, and to construct and train a ground target analysis model based on the selected ground feature sample training set, the selected ground feature sample validation set, and the selected ground feature sample test set, so as to obtain the trained air target recognition model and the trained ground target analysis model. The collaborative detection and analysis module is used to optimize the trained air target recognition model by enhancing the target detection algorithm based on the frequency domain features of air targets. The trained ground target analysis model is optimized by adaptive fine-tuning based on the scenario; target detection and feature analysis are achieved based on the optimized air target recognition model, the optimized ground target analysis model, and the multimodal feature fusion module. The collaborative detection and analysis module includes: an air target frequency domain feature enhancement module, a scene adaptive fine-tuning module, and a multimodal feature fusion module; The air target frequency domain feature enhancement module is used to enhance the multi-mode frequency domain features of the trained air target recognition model. It reshapes the multi-mode frequency domain features through content awareness, extracts different technical features, and couples and enhances the different technical features to optimize the trained air target recognition model. The scene adaptive fine-tuning module is used to perform joint visual-language optimization of the trained ground target analysis model according to the scene adaptive adjustment parameters. The multimodal feature fusion module is used to couple and enhance the optimized air target recognition model and the optimized ground target analysis model to achieve target detection and feature analysis.

2. The air-ground cooperative target detection and feature analysis system as described in claim 1, characterized in that, The aerial sample database includes: The first data acquisition module is used to acquire raw aerial target data; The first data cleaning and enhancement module is used to perform data enhancement on the original aerial target data after data cleaning to obtain the first enhanced aerial target data. The first data annotation module is used to annotate the first enhanced aerial target data using annotation tools to obtain annotated aerial target data. The first feature extraction module is used to extract features from the labeled aerial target data to obtain the first aerial features; The first classification module is used to classify the first aerial feature into the aerial feature sample training set, the aerial feature sample validation set, and the aerial feature sample test set.

3. The air-ground cooperative target detection and feature analysis system as described in claim 1, characterized in that, The ground sample database includes: The second data acquisition module is used to collect raw ground target data; The second data cleaning and enhancement module is used to perform data enhancement on the original ground target data after data cleaning to obtain the second enhanced ground target data. The second data annotation module is used to annotate the second enhanced aerial target data using annotation tools to obtain annotated ground target data. The second feature extraction module is used to extract features from the labeled ground target data to obtain the second ground features; The second classification module is used to classify the second ground feature into the ground feature sample training set, the ground feature sample validation set, and the ground feature sample test set.

4. The air-ground cooperative target detection and feature analysis system as described in claim 1, characterized in that, The sample processing and screening module includes: a sample acquisition module, a sample supplementation module, and a sample screening module; The sample acquisition module is used to acquire additional aerial feature sample datasets and additional ground feature sample datasets according to the application scenario. The sample supplementation module is used to supplement the air feature sample training set, air feature sample validation set, and air feature sample test set with additional air feature sample datasets, respectively, to obtain supplemented air feature sample training set, supplemented air feature sample validation set, and supplemented air feature sample test set; The sample supplementation module is also used to supplement the ground feature sample training set, ground feature sample validation set and ground feature sample test set with additional ground feature sample datasets respectively, so as to obtain the supplemented ground feature sample training set, the supplemented ground feature sample validation set and the supplemented ground feature sample test set. The sample selection module is used to select samples from the supplemented aerial feature sample training set, the supplemented aerial feature sample validation set, and the supplemented aerial feature sample test set, respectively, to obtain the corresponding selected aerial feature sample training set, selected aerial feature sample validation set, and selected aerial feature sample test set; the sample selection module is also used to select samples from the supplemented ground feature sample training set and the supplemented ground feature sample test set, respectively. The sample validation set and the supplemented ground feature sample test set are used to filter the samples, resulting in the corresponding filtered ground feature sample training set, filtered ground feature sample validation set, and filtered ground feature sample test set.

5. The air-ground cooperative target detection and feature analysis system as described in claim 1, characterized in that, The model training module includes: an aerial target recognition model training module and a ground target analysis model training module; The aerial target recognition model training module is used to construct an initial aerial target recognition model, input the selected aerial feature sample training set, the selected aerial feature sample verification set, and the selected aerial feature sample test set into the initial aerial target recognition model for training, so as to correct the initial aerial target recognition model, obtain the trained aerial target recognition model, and deploy the trained aerial target recognition model. The ground target analysis model training module is used to construct an initial ground target analysis model, adjust training parameters to optimize the initial ground target analysis model, input the selected ground feature sample training set, the selected ground feature sample validation set, and the selected ground feature sample test set into the optimized initial ground target analysis model for training, and after multiple iterations, evaluate the performance of the optimized initial ground target analysis model according to preset indicators, and use the optimized initial ground target analysis model with the best performance as the trained ground target analysis model.

6. A method for air-ground cooperative target detection and feature analysis, characterized in that, Includes the following steps: After collecting multi-source heterogeneous aerial raw data, the multi-source heterogeneous aerial raw data is sequentially cleaned, enhanced, labeled, feature extracted and classified to obtain aerial feature sample training set, aerial feature sample validation set and aerial feature sample test set, which are then summarized in the aerial sample database. After collecting multi-source heterogeneous ground raw data, the multi-source heterogeneous ground raw data are sequentially cleaned, enhanced, labeled, feature extracted and classified to obtain ground feature sample training set, ground feature sample validation set and ground feature sample test set, which are then summarized in the ground sample database. According to the application scenario, the training set, validation set, and test set of air feature samples in the air sample database and the training set, validation set, and test set of ground feature samples in the ground sample database are processed and filtered respectively to obtain the filtered training set, validation set, and test set of air feature samples, as well as the filtered training set, validation set, and test set of ground feature samples. An air target recognition model is constructed and trained based on the selected air feature sample training set, the selected air feature sample validation set, and the selected air feature sample test set. A ground target analysis model is constructed and trained based on the selected ground feature sample training set, the selected ground feature sample validation set, and the selected ground feature sample test set, resulting in the trained air target recognition model and the trained ground target analysis model. The trained aerial target recognition model is optimized based on the aerial target frequency domain feature enhancement target detection algorithm; The trained ground target analysis model is optimized by adaptive fine-tuning based on the scenario; target detection and feature analysis are achieved based on the optimized air target recognition model, the optimized ground target analysis model, and the multimodal feature fusion module. The trained aerial target recognition model is optimized by using an enhanced target detection algorithm based on the frequency domain features of aerial targets. The trained ground target analysis model is optimized through scene-adaptive fine-tuning. Based on the optimized aerial target recognition model, the optimized ground target analysis model, and the multimodal feature fusion module, target detection and feature analysis are achieved, including the following steps: Feature enhancement is performed on the multi-mode frequency domain features of the trained aerial target recognition model. The multi-mode frequency domain features are reshaped through content awareness, different technical features are extracted, and the different technical features are coupled and enhanced to optimize the trained aerial target recognition model. The trained ground target analysis model is jointly optimized using vision and language by adaptively adjusting parameters according to the scenario. The optimized aerial target recognition model and the optimized ground target analysis model are coupled and enhanced to achieve target detection and feature analysis.

Images