Air-ground cooperative target detection and feature analysis system and method

Abstract

The invention provides an air-ground cooperative target detection and feature analysis system and method, and the method comprises the steps: correspondingly obtaining an air sample database and a ground sample database through collection, data cleaning, enhancement, labeling, feature extraction and classification; the air feature sample data set and the ground feature sample data set are processed and screened through a sample processing module, the screened air sample data set and the screened ground feature sample data set are input into a model for training, and a trained air target recognition model and a trained ground target analysis model are obtained; and the two models are optimized respectively, and detection and feature analysis of the target are realized in combination with multi-modal feature fusion. Compared with the prior art, through mutual combination of the aerial target identification model and the ground target analysis model and multi-modal feature fusion, target detection and feature analysis are realized based on application scene training optimization, and target detection precision and feature analysis accuracy are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection, and particularly to an air-ground collaborative target detection and feature analysis system and method. Background Art

[0002] Methods based on Artificial Intelligence (AI) have become the mainstream technologies in the field of target detection. With their high accuracy, strong real-time performance, and rapid iterative optimization, they have demonstrated excellent performance in various application scenarios. Traditional target detection methods are difficult to adapt to the changes in complex environments. Target detection technologies based on deep learning networks can not only autonomously learn high-dimensional features but also combine multi-scale perception, context information fusion, and temporal modeling to achieve more accurate target recognition and behavior analysis.

[0003] Currently, there are mainly two technical paths for mainstream target detection systems: unmanned aerial vehicle (UAV)-borne target detection based on small AI models and small target detection based on large AI models. Both have their own advantages but also have their own limitations. When used independently, it is difficult to achieve efficient and accurate target detection and behavior analysis in practical applications.

[0004] When dealing with large scenes, due to their complexity and the increase in the number of parameters, large models often have difficulty concentrating on small targets. Such models are easily attracted by large-scale elements or significant features in the environment when facing complex backgrounds and diverse targets, thus ignoring those smaller and more subtle targets. At the same time, due to the limitations of the performance of AI acceleration cards, it will be extremely difficult to deploy large models on the UAV end side.

[0005] In specific application scenarios, although small models can achieve fast response, they have serious deficiencies in deep analysis capabilities and are difficult to achieve fine-grained feature analysis of targets. Even if small models can detect the presence of targets, they cannot provide sufficient information regarding the behavior patterns, feature descriptions, and context understanding of targets in complex scenarios.

[0006] When performing tasks, UAVs not only need real-time detection but also higher-level intelligent analysis capabilities to handle complex situations in dynamic environments. Using the two separately will inevitably lead to the lack of some functions. This requires that in system design, the advantages of small models and large models be considered to be combined to promote the further development of UAV technology and ensure its efficient operation in changing application scenarios.

[0007] Therefore, it is an urgent problem for those skilled in the art to provide an air-ground collaborative target detection and feature analysis system and method to solve the above problems. Summary of the Invention

[0008] The object of the present invention is to provide an air-ground collaborative target detection and feature analysis system, which has a simple structure, is safe, effective, reliable and easy to operate, and can realize functions such as high-resolution and efficient end-side real-time detection of small targets from the perspective of unmanned aerial vehicles, action prediction, appearance analysis, etc. By establishing an airborne end-side target detection algorithm dataset, training corresponding small models for application scenarios and deploying them on the end side; according to the task requirements, establishing a training set database for actual application functions such as target intention, behavior mode, appearance description, threat analysis, etc., and fine-tuning the edge large language model to achieve more accurate human-machine interaction.

[0009] Based on the above object, the technical solution provided by the present invention is as follows: An air-ground collaborative target detection and feature analysis system, comprising: An aerial sample database, which is used to collect multi-source heterogeneous aerial raw data, and then sequentially perform data cleaning, enhancement, annotation, feature extraction and classification on the multi-source heterogeneous aerial raw data to obtain an aerial feature sample training set, an aerial feature sample validation set and an aerial feature sample test set; A ground sample database, which is used to collect multi-source heterogeneous ground raw data, and then sequentially perform data cleaning, enhancement, annotation, feature extraction and classification on the multi-source heterogeneous ground raw data to obtain a ground feature sample training set, a ground feature sample validation set and a ground feature sample test set; A sample processing and screening module, which is used to process and screen the aerial feature sample training set, the aerial feature sample validation set and the aerial feature sample test set in the aerial sample database and the ground feature sample training set, the ground feature sample validation set and the ground feature sample test set in the ground sample database according to the application scenario, and obtain the screened aerial feature sample training set, the screened aerial feature sample validation set and the screened aerial feature sample test set, as well as the screened ground feature sample training set, the screened ground feature sample validation set and the screened ground feature sample test set; A model training module, which is used to construct and train an aerial target recognition model according to the screened aerial feature sample training set, the screened aerial feature sample validation set and the screened aerial feature sample test set, and construct and train a ground target analysis model according to the screened ground feature sample training set, the screened ground feature sample validation set and the screened ground feature sample test set, to obtain a trained aerial target recognition model and a trained ground target analysis model; The collaborative detection and analysis module is used to optimize the trained air target recognition model according to the frequency domain characteristics of air targets to enhance the target detection algorithm; optimize the trained ground target analysis model according to scene adaptive fine-tuning; and achieve target detection and feature analysis based on the optimized air target recognition model, the optimized ground target analysis model, and the multi-modal feature fusion module.

[0010] Preferably, the air sample database includes: The first data acquisition module is used to acquire raw air target data. The first data cleaning and enhancement module is used to perform data enhancement after data cleaning on the raw air target data to obtain first enhanced air target data. The first data annotation module is used to perform data annotation on the first enhanced air target data by using an annotation tool to obtain annotated air target data. The first feature extraction module is used to extract features from the annotated air target data to obtain first air features. The first classification module is used to classify the first air features into the air feature sample training set, the air feature sample validation set, and the air feature sample test set.

[0011] Preferably, the ground sample database includes: The second data acquisition module is used to acquire raw ground target data. The second data cleaning and enhancement module is used to perform data enhancement after data cleaning on the raw ground target data to obtain second enhanced ground target data. The second data annotation module is used to perform data annotation on the second enhanced air target data by using an annotation tool to obtain annotated ground target data. The second feature extraction module is used to extract features from the annotated ground target data to obtain second ground features. The second classification module is used to classify the second ground features into the ground feature sample training set, the ground feature sample validation set, and the ground feature sample test set.

[0012] Preferably, the sample processing and screening module includes: a sample acquisition module, a sample supplementation module, and a sample screening module. The sample acquisition module is used to correspondingly acquire additional air feature sample data sets and additional ground feature sample data sets according to the application scenario. The sample augmentation module is used to augment the aerial feature sample training set, the aerial feature sample validation set, and the aerial feature sample test set with an additional aerial feature sample data set respectively, to obtain an augmented aerial feature sample training set, an augmented aerial feature sample validation set, and an augmented aerial feature sample test set; The sample augmentation module is further used to augment the ground feature sample training set, the ground feature sample validation set, and the ground feature sample test set with an additional ground feature sample data set respectively, to obtain an augmented ground feature sample training set, an augmented ground feature sample validation set, and an augmented ground feature sample test set; The sample screening module is used to screen the samples in the augmented aerial feature sample training set, the augmented aerial feature sample validation set, and the augmented aerial feature sample test set respectively, to obtain the corresponding screened aerial feature sample training set, screened aerial feature sample validation set, and screened aerial feature sample test set; The sample screening module is further used to screen the samples in the augmented ground feature sample training set, the augmented ground feature sample validation set, and the augmented ground feature sample test set respectively, to obtain the corresponding screened ground feature sample training set, screened ground feature sample validation set, and screened ground feature sample test set.

[0013] Preferably, the model training module includes: an aerial target recognition model training module and a ground target analysis model training module; The aerial target recognition model training module is used to construct an initial aerial target recognition model, input the screened aerial feature sample training set, the screened aerial feature sample validation set, and the screened aerial feature sample test set into the initial aerial target recognition model for training, to correct the initial aerial target recognition model, obtain the trained aerial target recognition model, and deploy the trained aerial target recognition model; The ground target analysis model training module is used to construct an initial ground target analysis model, adjust the training parameters to optimize the initial ground target analysis model, input the screened ground feature sample training set, the screened ground feature sample validation set, and the screened ground feature sample test set into the optimized initial ground target analysis model for training. After multiple rounds of iteration, evaluate the performance of the optimized initial ground target analysis model with parameters tested according to a preset index, and use the optimized initial ground target analysis model with the optimal performance as the trained ground target analysis model.

[0014] Preferably, the collaborative detection and analysis module includes: an aerial target frequency domain feature enhancement module, a scene adaptive fine-tuning module, and a multi-modal feature fusion module; The aerial target frequency-domain feature enhancement module is used to enhance the multi-modal frequency-domain features of the trained aerial target recognition model. It reshapes the multi-modal frequency-domain features through content awareness, extracts different technical features, and couples and enhances the different technical features to optimize the trained aerial target recognition model. The scene adaptive fine-tuning module is used to perform visual-language joint optimization on the trained ground target analysis model by adaptively adjusting parameters according to the scene. The multi-modal feature fusion module is used to couple and enhance the optimized aerial target recognition model and the optimized ground target analysis model, delete the large target detection layer, and add and enhance the small target detection layer to achieve target detection and feature analysis.

[0015] An air-ground collaborative target detection and feature analysis method includes the following steps: After collecting multi-source heterogeneous aerial raw data, the multi-source heterogeneous aerial raw data is sequentially subjected to data cleaning, enhancement, annotation, feature extraction, and classification to obtain an aerial feature sample training set, an aerial feature sample validation set, and an aerial feature sample test set. After collecting multi-source heterogeneous ground raw data, the multi-source heterogeneous ground raw data is sequentially subjected to data cleaning, enhancement, annotation, feature extraction, and classification to obtain a ground feature sample training set, a ground feature sample validation set, and a ground feature sample test set. According to the application scenario, the aerial feature sample training set, aerial feature sample validation set, and aerial feature sample test set in the aerial sample database and the ground feature sample training set, ground feature sample validation set, and ground feature sample test set in the ground sample database are respectively processed and screened to obtain a screened aerial feature sample training set, a screened aerial feature sample validation set, a screened aerial feature sample test set, a screened ground feature sample training set, a screened ground feature sample validation set, and a screened ground feature sample test set. An aerial target recognition model is constructed and trained based on the screened aerial feature sample training set, screened aerial feature sample validation set, and screened aerial feature sample test set, and a ground target analysis model is constructed and trained based on the screened ground feature sample training set, screened ground feature sample validation set, and screened ground feature sample test set to obtain a trained aerial target recognition model and a trained ground target analysis model. The trained aerial target recognition model is optimized according to the aerial target frequency-domain feature enhancement target detection algorithm; the trained ground target analysis model is optimized according to scene adaptive fine-tuning; target detection and feature analysis are achieved based on the optimized aerial target recognition model, optimized ground target analysis model, and multi-modal feature fusion module.

[0016] The present invention provides an air-ground collaborative target detection and feature analysis system. By collecting, data cleaning, enhancing, annotating, feature extracting and classifying, an aerial sample database and a ground sample database are obtained correspondingly. The aerial feature sample data set and the ground feature sample data set are processed and screened by a sample processing module, and the screened aerial sample data set and the screened ground feature sample data set are input into a model for training to obtain a trained aerial target recognition model and a trained ground target analysis model; the above two models are optimized respectively, and the detection and feature analysis of the target are realized by combining multi-modal feature fusion.

[0017] Compared with the prior art, the present invention combines the aerial target recognition model and the ground target analysis model and multi-modal feature fusion, and realizes the detection and feature analysis of the target through training and optimization based on the application scenario, improving the target detection accuracy and feature analysis accuracy.

[0018] The present invention also provides an air-ground collaborative target detection and feature analysis method. Since it belongs to the same technical concept as the system and solves the same technical problems, it should have the same beneficial effects and will not be elaborated again. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0020] Figure 1 It is a schematic structural diagram of an air-ground collaborative target detection and feature analysis system provided by an embodiment of the present invention; Figure 2 It is a schematic structural diagram of the aerial sample database provided by an embodiment of the present invention; Figure 3 It is a schematic structural diagram of the ground sample database provided by an embodiment of the present invention; Figure 4 It is a schematic structural diagram of the sample processing and screening module provided by an embodiment of the present invention; Figure 5 It is a schematic structural diagram of the model training module provided by an embodiment of the present invention; Figure 6 It is a schematic structural diagram of the collaborative detection and analysis module provided by an embodiment of the present invention; Figure 7 It is a flowchart of an air-ground collaborative target detection and feature analysis method provided by an embodiment of the present invention. Detailed implementation manners

[0021] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0022] The embodiments of the present invention are written in a progressive manner.

[0023] The embodiments of the present invention provide an air-ground collaborative target detection and feature analysis system. It mainly solves the technical problems in the prior art that when a large model processes a large scene, due to its complexity and the increase in the number of parameters, it is often difficult to focus on small targets. Although a small model can achieve fast response in a specific application scenario, its depth analysis ability is seriously insufficient, and it is difficult to achieve fine-grained feature analysis of targets.

[0024] As Figure 1 shown, an air-ground collaborative target detection and feature analysis system includes: An air sample database, which is used to collect multi-source heterogeneous air raw data, and then sequentially perform data cleaning, enhancement, annotation, feature extraction and classification on the multi-source heterogeneous air raw data to obtain an air feature sample training set, an air feature sample validation set and an air feature sample test set; A ground sample database, which is used to collect multi-source heterogeneous ground raw data, and then sequentially perform data cleaning, enhancement, annotation, feature extraction and classification on the multi-source heterogeneous ground raw data to obtain a ground feature sample training set, a ground feature sample validation set and a ground feature sample test set; A sample processing and screening module, which is used to process and screen the air feature sample training set, the air feature sample validation set and the air feature sample test set in the air sample database and the ground feature sample training set, the ground feature sample validation set and the ground feature sample test set in the ground sample database according to the application scenario, to obtain a screened air feature sample training set, a screened air feature sample validation set and a screened air feature sample test set, as well as a screened ground feature sample training set, a screened ground feature sample validation set and a screened ground feature sample test set; The model training module is used to construct and train an aerial target recognition model based on the filtered aerial feature sample training set, the filtered aerial feature sample validation set, and the filtered aerial feature sample test set, and to construct and train a ground target analysis model based on the filtered ground feature sample training set, the filtered ground feature sample validation set, and the filtered ground feature sample test set, so as to obtain a trained aerial target recognition model and a trained ground target analysis model; The collaborative detection and analysis module is used to optimize the trained aerial target recognition model according to the aerial target frequency-domain feature enhanced target detection algorithm; optimize the trained ground target analysis model according to the scene adaptive fine-tuning; and realize target detection and feature analysis according to the optimized aerial target recognition model, the optimized ground target analysis model, and the multimodal feature fusion module.

[0025] In the actual application process, an air-ground collaborative target detection and feature analysis system is provided with an aerial sample database, a ground sample database, a sample processing and screening module, a model training module, and a collaborative detection and analysis module.

[0026] In this embodiment, by collecting raw data (classified into multi-source sensing data such as visible light, infrared, hyperspectral, radar, etc. according to the multi-source raw data sources), data cleaning, enhanced annotation, feature extraction, and classification are carried out, and an aerial sample data set is correspondingly formed and stored in the aerial sample database according to the raw data sources (such as drone collection and ground-end collection), and a ground sample data set is correspondingly formed and stored in the ground sample database; According to the specific application scenario, the aerial feature sample data set (training set, validation set, and test set) and the ground feature sample data set (training set, validation set, and test set) are respectively processed and screened, and the screened aerial feature sample data set and the screened ground feature sample data set are input into the model for training to obtain an aerial target recognition model and a ground target analysis model; the aerial target recognition model and the ground target analysis model are respectively optimized and then combined with multimodal feature fusion to realize target detection and feature analysis.

[0027] As Figure 2 shown, preferably, the aerial sample database includes: The first data acquisition module is used to acquire aerial raw target data; The first data cleaning and enhancement module is used to perform data enhancement after data cleaning on the aerial raw target data to obtain the first enhanced aerial target data; The first data annotation module is used to use an annotation tool to perform data annotation on the first enhanced aerial target data to obtain the annotated aerial target data; The first feature extraction module is used to extract features from the labeled aerial target data to obtain the first aerial features. The first classification module is used to classify the first aerial features into an aerial feature sample training set, an aerial feature sample validation set, and an aerial feature sample test set.

[0028] In the actual application process, a first data acquisition module, a first data cleaning and enhancement module, a first data annotation module, a first feature extraction module, and a first classification module are set in the aerial sample database. The first data acquisition module collects the original aerial target data, and then successively performs data cleaning, data enhancement, data annotation, feature extraction, and data classification to obtain an aerial feature sample training set, an aerial feature sample validation set, and an aerial feature sample test set.

[0029] In this embodiment, data is collected through multi-source sensors, and preprocessing such as data acquisition, cleaning, and enhancement is performed on the collected multi-source heterogeneous original data. Data annotation and classification are performed on the target of interest, and finally, a training set, a validation set, and a test set are formed and stored in the database to form an aerial sample database, achieving the improvement of sample quality and sample expansion throughout the entire life cycle of the airborne target recognition small model.

[0030] As Figure 3 shown, preferably, the ground sample database includes: A second data acquisition module for collecting the original ground target data; A second data cleaning and enhancement module for performing data enhancement after cleaning the original ground target data to obtain the second enhanced ground target data; A second data annotation module for using an annotation tool to perform data annotation on the second enhanced aerial target data to obtain the labeled ground target data; A second feature extraction module for extracting features from the labeled ground target data to obtain the second ground features; A second classification module for classifying the second ground features into a ground feature sample training set, a ground feature sample validation set, and a ground feature sample test set.

[0031] In the actual application process, a second data acquisition module, a second data cleaning and enhancement module, a second data annotation module, a second feature extraction module, and a second classification module are set in the ground sample database. The second data acquisition module collects the original ground target data, and then successively performs data cleaning, data enhancement, data annotation, feature extraction, and data classification to obtain a ground feature sample training set, a ground feature sample validation set, and a ground feature sample test set.

[0032] In this embodiment, according to the task requirements, relevant training data sets are created or collected according to the format of the large model to be fine-tuned. Diversified data is collected to cover different situations and scenarios to avoid overfitting of the model. Preprocessing such as data cleaning, enhancement, format conversion, and partitioning is performed on the collected or compiled data sets, and they are stored in a database to form a ground sample database.

[0033] As Figure 4 shown, preferably, the sample processing and screening module includes: a sample acquisition module, a sample augmentation module, and a sample screening module; The sample acquisition module is used to acquire additional aerial feature sample data sets and additional ground feature sample data sets corresponding to the application scenario; The sample augmentation module is used to augment the additional aerial feature sample data sets to the aerial feature sample training set, the aerial feature sample validation set, and the aerial feature sample test set respectively, to obtain the augmented aerial feature sample training set, the augmented aerial feature sample validation set, and the augmented aerial feature sample test set; The sample augmentation module is also used to augment the additional ground feature sample data sets to the ground feature sample training set, the ground feature sample validation set, and the ground feature sample test set respectively, to obtain the augmented ground feature sample training set, the augmented ground feature sample validation set, and the augmented ground feature sample test set; The sample screening module is used to screen the augmented aerial feature sample training set, the augmented aerial feature sample validation set, and the augmented aerial feature sample test set respectively, to obtain the corresponding screened aerial feature sample training set, screened aerial feature sample validation set, and screened aerial feature sample test set; The sample screening module is also used to screen the augmented ground feature sample training set, the augmented ground feature sample validation set, and the augmented ground feature sample test set respectively, to obtain the corresponding screened ground feature sample training set, screened ground feature sample validation set, and screened ground feature sample test set.

[0034] In the actual application process, the sample acquisition module, the sample augmentation module, and the sample screening module are set in the sample processing and screening module. The additional aerial feature sample data sets and additional ground feature sample data sets are acquired according to the application scenario through the sample acquisition module. The corresponding aerial feature sample data sets and ground feature sample data sets are augmented according to the additional aerial feature sample data sets and additional ground feature sample data sets respectively through the sample augmentation module. After the augmentation is completed, screening is performed through the sample screening module to obtain the corresponding screened aerial feature sample data sets (training set, validation set, and test set) and screened ground feature sample data sets (training set, validation set, and test set).

[0035] In this embodiment, for different application scenarios, the types of target to be focused on vary, and it is necessary to supplement and screen the existing samples in the database according to the application scenarios. The multi-source data is divided into multi-source sensing data such as visible light, infrared, hyperspectral, and radar, and data is acquired in different working platforms and different working environments to enrich the types and quantities of training samples.

[0036] As Figure 5 shown, preferably, the model training module includes: an air target recognition model training module and a ground target analysis model training module; The air target recognition model training module is used to construct an initial air target recognition model, input the screened air feature sample training set, the screened air feature sample validation set, and the screened air feature sample test set into the initial air target recognition model for training to correct the initial air target recognition model, obtain the trained air target recognition model, and deploy the trained air target recognition model; The ground target analysis model training module is used to construct an initial ground target analysis model, adjust the training parameters to optimize the initial ground target analysis model, input the screened ground feature sample training set, the screened ground feature sample validation set, and the screened ground feature sample test set into the optimized initial ground target analysis model for training. After multiple iterations, evaluate the performance of the optimized initial ground target analysis model with parameter testing according to the preset index, and use the optimized initial ground target analysis model with the best performance as the trained ground target analysis model.

[0037] In the actual application process, the model training module is provided with an air target recognition model training module and a ground target analysis model training module; through the air target recognition model training module, an initial air target recognition model is constructed, and the screened air feature sample data is input into the initial target recognition model for training to correct the initial target recognition model, obtain the trained air target recognition model, and deploy the trained air target recognition model; through the ground target analysis model training module, an initial ground target analysis model is constructed, the training parameters are adjusted to optimize the model, and the screened ground feature sample data is input into the optimized ground target analysis model for training. After multiple iterations, evaluate the performance of the optimized initial ground target analysis model with parameter testing according to the preset index, and use the model with the best performance as the trained ground target analysis model; In this embodiment, the establishment of the small model of the air-end target recognition algorithm is specifically as follows: train, test, and optimize in a ground system with rich computing and storage resources to form an algorithm small model. Finally, deploy the trained and corrected small model to the air-end AI acceleration card; The establishment of the large model for the ground-based target recognition algorithm is specifically as follows: Based on the fine-tuned pre-trained AI large model algorithm, determine the necessary computing resources, including memory and storage space, to process large-scale data. Adjust key parameters such as the learning rate, batch size, and training cycles to optimize the model performance. Use the ground sample database to train the large model, and evaluate the model through indicators such as accuracy, F1 score, and BLEU score for testing. After multiple rounds of iteration, a large model is formed.

[0038] As Figure 6 shown, preferably, the collaborative detection and analysis module includes: an air target frequency domain feature enhancement module, a scene adaptive fine-tuning module, and a multi-modal feature fusion module; The air target frequency domain feature enhancement module is used to enhance the multi-modal frequency domain features of the trained air target recognition model, reshape the multi-modal frequency domain features through content awareness, extract different technical features, and couple and enhance the different technical features to optimize the trained air target recognition model; The scene adaptive fine-tuning module is used to perform visual-language joint optimization on the trained ground target analysis model according to the scene adaptive adjustment parameters; The multi-modal feature fusion module is used to couple and enhance the optimized air target recognition model and the optimized ground target analysis model, delete the large target detection layer, and add and enhance the small target detection layer to achieve target detection and feature analysis.

[0039] In the actual application process, the collaborative detection and analysis module is provided with an air target frequency domain feature enhancement module, a scene adaptive fine-tuning module, and a multi-modal feature fusion module; the multi-modal frequency domain features in the trained air target recognition model are enhanced by the air target frequency domain feature enhancement module, reshaped through content awareness, different technical features are extracted, and coupled and enhanced to optimize the trained air target recognition model; through the scene adaptive fine-tuning module, the trained ground target analysis model is subjected to visual-language joint optimization according to the scene adaptive adjustment parameters; through multi-modal feature fusion, the two optimized models are coupled and enhanced, the large target detection layer is deleted, and the small target detection layer is added and enhanced to achieve target detection and feature analysis.

[0040] In this embodiment, there is a Multimodal Feature Fusion Module (MFFM), which has two functions: First, it provides multi-modal features enhanced for each modality for the subsequent multi-modal feature extraction network; Second, it provides multi-modal input features for multi-scale feature fusion and has a feedback mechanism. MFFM has three key modules: a feature map hybrid pooling and compression module, a frequency-domain multi-head aggregation and enhancement module, and a residual reinforcement dual-mode fusion module, which can achieve high-precision image feature extraction and fusion; In the collaborative detection and analysis module, a lightweight small target detection algorithm based on YOLOv8, called FC-YOLO, is adopted, which includes three key modules: a frequency-domain dynamic core transformation module, a content-aware dynamic kernel upsampling module, and a frequency-domain dynamic core transformation module, a content-aware dynamic kernel upsampling module, and a cross-layer high-resolution fine-grained detection layer. The frequency-domain dynamic core transformation module enhances the frequency-domain features of multi-modal sensor data, effectively improving the perception ability of small targets; The content-aware dynamic kernel upsampling module reshapes multi-modal features through content awareness, extracts different features of small targets in multi-modalities, and then couples and enhances the multi-features, enhancing the feature extraction ability and positioning accuracy of small targets; The cross-layer high-resolution fine-grained detection layer deletes the large target detection layer in the algorithm, adds and enhances the small target detection layer, and improves the resolution according to the enhancement of multi-modal features, thus optimizing the detection ability of small targets. FC-YOLO shows good performance in the detection of small targets with high resolution, improving the detection accuracy of small targets from a large perspective of UAVs.

[0041] The adaptive fine-tuning module is specifically as follows: It conducts attention scene adaptive fine-tuning based on the Qwen2-VL large model. Through systematic transformation of target domain features, retrieval enhancement generation is carried out, significantly improving the task accuracy of the model in vertical scenarios. Focusing on the in-depth adaptation of the multi-modal understanding ability of Qwen2-VL, a design including visual-language joint optimization is carried out. On the basis of maintaining the original generalization ability of Qwen2-VL, a double improvement in cross-scene transfer efficiency and the accuracy of small target detection and fine-grained feature analysis tasks from the airborne perspective is achieved.

[0042] As Figure 7 shown, a method for air-ground collaborative target detection and feature analysis includes the following steps: S1. After collecting multi-source heterogeneous aerial raw data, the multi-source heterogeneous aerial raw data is sequentially subjected to data cleaning, enhancement, annotation, feature extraction and classification to obtain an aerial feature sample training set, an aerial feature sample validation set, and an aerial feature sample test set; S2. After collecting multi-source heterogeneous ground raw data, perform data cleaning, enhancement, annotation, feature extraction, and classification on the multi-source heterogeneous ground raw data in sequence to obtain a ground feature sample training set, a ground feature sample validation set, and a ground feature sample test set; S3. Process and filter the aerial feature sample training set, aerial feature sample validation set, and aerial feature sample test set in the aerial sample database and the ground feature sample training set, ground feature sample validation set, and ground feature sample test set in the ground sample database according to the application scenario to obtain a filtered aerial feature sample training set, a filtered aerial feature sample validation set, a filtered aerial feature sample test set, a filtered ground feature sample training set, a filtered ground feature sample validation set, and a filtered ground feature sample test set; S4. Construct and train an aerial target recognition model based on the filtered aerial feature sample training set, filtered aerial feature sample validation set, and filtered aerial feature sample test set, and construct and train a ground target analysis model based on the filtered ground feature sample training set, filtered ground feature sample validation set, and filtered ground feature sample test set to obtain a trained aerial target recognition model and a trained ground target analysis model; S5. Optimize the trained aerial target recognition model according to the aerial target frequency domain feature enhanced target detection algorithm; optimize the trained ground target analysis model according to the scene adaptive fine-tuning; realize target detection and feature analysis according to the optimized aerial target recognition model, optimized ground target analysis model, and multi-modal feature fusion module.

[0043] The present invention also provides an air-ground collaborative target detection and feature analysis method, which belongs to the same technical concept as the system and has the same technical effect.

[0044] In the embodiments provided in this application, it should be understood that the disclosed method and system can be implemented in other ways. The system embodiments described above are only illustrative. For example, the division of modules is only a logical function division. In actual implementation, there may be other division methods, such as: multiple modules or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed may be through some interfaces, indirect coupling or communication connection of devices or modules, and may be electrical, mechanical, or other forms.

[0045] In addition, in each embodiment of the present invention, each functional module can be entirely integrated in one processor, or each module can be separately used as a device alone, or two or more modules can be integrated in one device; each functional module in each embodiment of the present invention can be implemented in the form of hardware, or in the form of a combination of hardware and software functional units.

[0046] Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by program instructions and related hardware. The foregoing program instructions can be stored in a computer-readable storage medium. When the program instructions are executed, they perform the steps including the above method embodiments; and the foregoing storage medium includes: various media that can store program codes such as removable storage devices, read-only memory (ROM), magnetic disks, or optical discs.

[0047] It should be understood that in the present application, if terms such as "system", "device", "unit", and / or "module" are used, they are only a method for distinguishing different components, elements, parts, portions, or assemblies at different levels. However, if other words can achieve the same purpose, that term can be replaced by other expressions.

[0048] As shown in the present application and the claims, unless the context clearly indicates an exceptional situation, words such as "a", "an", "one", and / or "the" are not specifically singular and may also include the plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements. An element defined by the statement "including one..." does not exclude the existence of another identical element in the process, method, commodity, or device including the element.

[0049] Hereinafter, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features.

[0050] If a flowchart is used in the present application, the flowchart is used to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the operations before or after do not necessarily need to be executed precisely in order. On the contrary, they can be executed in reverse order or simultaneously. At the same time, other operations can also be added to these processes, or one or several operations can be removed from these processes.

[0051] The above has introduced in detail a system for air-ground collaborative target detection and feature analysis provided by the present invention. The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown herein, but rather will be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An air-ground collaborative target detection and feature analysis system, characterized in that, Including: An aerial sample database, which is used to collect multi-source heterogeneous aerial raw data, and then sequentially perform data cleaning, enhancement, annotation, feature extraction and classification on the multi-source heterogeneous aerial raw data to obtain an aerial feature sample training set, an aerial feature sample validation set and an aerial feature sample test set; A ground sample database, which is used to collect multi-source heterogeneous ground raw data, and then sequentially perform data cleaning, enhancement, annotation, feature extraction and classification on the multi-source heterogeneous ground raw data to obtain a ground feature sample training set, a ground feature sample validation set and a ground feature sample test set; A sample processing and screening module, which is used to process and screen the aerial feature sample training set, the aerial feature sample validation set and the aerial feature sample test set of the aerial sample database and the ground feature sample training set, the ground feature sample validation set and the ground feature sample test set in the ground sample database according to the application scenario, and obtain a screened aerial feature sample training set, a screened aerial feature sample validation set and a screened aerial feature sample test set, as well as a screened ground feature sample training set, a screened ground feature sample validation set and a screened ground feature sample test set; A model training module, which is used to construct and train an aerial target recognition model according to the screened aerial feature sample training set, the screened aerial feature sample validation set and the screened aerial feature sample test set, and construct and train a ground target analysis model according to the screened ground feature sample training set, the screened ground feature sample validation set and the screened ground feature sample test set, and obtain a trained aerial target recognition model and a trained ground target analysis model; A collaborative detection and analysis module, which is used to optimize the trained aerial target recognition model according to the aerial target frequency domain feature enhanced target detection algorithm; Optimize the trained ground target analysis model according to the scene adaptive fine-tuning; realize target detection and feature analysis according to the optimized aerial target recognition model, the optimized ground target analysis model and the multi-modal feature fusion module.

2. The air-ground collaborative target detection and feature analysis system according to claim 1, characterized in that, The aerial sample database includes: A first data collection module, which is used to collect aerial raw target data; A first data cleaning and enhancement module, which is used to perform data enhancement after data cleaning on the aerial raw target data to obtain first enhanced aerial target data; A first data annotation module, which is used to use an annotation tool to perform data annotation on the first enhanced aerial target data to obtain annotated aerial target data; A first feature extraction module, which is used to perform feature extraction on the annotated aerial target data to obtain first aerial features; A first classification module, which is used to classify the first aerial features into the aerial feature sample training set, the aerial feature sample validation set and the aerial feature sample test set.

3. The air-to-ground collaborative target detection and feature analysis system according to claim 1, characterized in that: The ground sample database includes: A second data collection module, which is used to collect ground raw target data; A second data cleaning and enhancement module, which is used to perform data enhancement after data cleaning on the ground raw target data to obtain second enhanced ground target data; The second data annotation module is used to perform data annotation on the second enhanced aerial target data by using an annotation tool to obtain the annotated ground target data; The second feature extraction module is used to extract features from the annotated ground target data to obtain the second ground features; The second classification module is used to classify the second ground features into the ground feature sample training set, the ground feature sample validation set, and the ground feature sample test set.

4. The air-ground collaborative target detection and feature analysis system according to claim 1, wherein The sample processing and screening module includes: a sample acquisition module, a sample augmentation module, and a sample screening module; The sample acquisition module is used to correspondingly acquire additional aerial feature sample data sets and additional ground feature sample data sets according to the application scenario; The sample augmentation module is used to augment the additional aerial feature sample data sets to the aerial feature sample training set, the aerial feature sample validation set, and the aerial feature sample test set respectively to obtain the augmented aerial feature sample training set, the augmented aerial feature sample validation set, and the augmented aerial feature sample test set; The sample augmentation module is further used to augment the additional ground feature sample data sets to the ground feature sample training set, the ground feature sample validation set, and the ground feature sample test set respectively to obtain the augmented ground feature sample training set, the augmented ground feature sample validation set, and the augmented ground feature sample test set; The sample screening module is used to perform sample screening on the augmented aerial feature sample training set, the augmented aerial feature sample validation set, and the augmented aerial feature sample test set respectively to obtain the corresponding screened aerial feature sample training set, the screened aerial feature sample validation set, and the screened aerial feature sample test set; The sample screening module is further used to perform sample screening on the augmented ground feature sample training set, the augmented ground feature sample validation set, and the augmented ground feature sample test set respectively to obtain the corresponding screened ground feature sample training set, the screened ground feature sample validation set, and the screened ground feature sample test set.

5. The air-ground collaborative target detection and feature analysis system according to claim 1, characterized in that, The model training module includes: an aerial target recognition model training module and a ground target analysis model training module; The aerial target recognition model training module is used to construct an initial aerial target recognition model, input the screened aerial feature sample training set, the screened aerial feature sample validation set, and the screened aerial feature sample test set into the initial aerial target recognition model for training to correct the initial aerial target recognition model to obtain the trained aerial target recognition model, and deploy the trained aerial target recognition model; The ground target analysis model training module is used to construct an initial ground target analysis model, adjust training parameters to optimize the initial ground target analysis model, input the filtered ground feature sample training set, filtered ground feature sample validation set, and filtered ground feature sample test set into the optimized initial ground target analysis model for training. After multiple rounds of iteration, evaluate the performance of the parameter-tested and optimized initial ground target analysis model according to preset metrics, and use the initial ground target analysis model with the optimal performance as the trained ground target analysis model.

6. The air-ground collaborative target detection and feature analysis system according to claim 1, characterized in that The collaborative detection and analysis module includes: an airborne target frequency-domain feature enhancement module, a scene adaptive fine-tuning module, and a multimodal feature fusion module; The airborne target frequency-domain feature enhancement module is used to enhance the multimodal frequency-domain features of the trained airborne target recognition model, reshape the multimodal frequency-domain features through content awareness, extract different technical features, and couple and enhance the different technical features to optimize the trained airborne target recognition model; The scene adaptive fine-tuning module is used to perform visual-language joint optimization on the trained ground target analysis model by adaptively adjusting parameters according to the scene; The multimodal feature fusion module is used to couple and enhance the optimized airborne target recognition model and the optimized ground target analysis model, delete the large target detection layer, and add and enhance the small target detection layer to achieve target detection and feature analysis.

7. A method for air-ground collaborative target detection and feature analysis, characterized in that, It includes the following steps: After collecting multi-source heterogeneous airborne raw data, sequentially perform data cleaning, enhancement, annotation, feature extraction, and classification on the multi-source heterogeneous airborne raw data to obtain an airborne feature sample training set, an airborne feature sample validation set, and an airborne feature sample test set, and summarize them in an airborne sample database; After collecting multi-source heterogeneous ground raw data, sequentially perform data cleaning, enhancement, annotation, feature extraction, and classification on the multi-source heterogeneous ground raw data to obtain a ground feature sample training set, a ground feature sample validation set, and a ground feature sample test set, and summarize them in a ground sample database; Process and filter the airborne feature sample training set, airborne feature sample validation set, and airborne feature sample test set in the airborne sample database and the ground feature sample training set, ground feature sample validation set, and ground feature sample test set in the ground sample database according to the application scenario to obtain a filtered airborne feature sample training set, a filtered airborne feature sample validation set, a filtered airborne feature sample test set, and a filtered ground feature sample training set, a filtered ground feature sample validation set, and a filtered ground feature sample test set; Construct and train an airborne target recognition model based on the filtered airborne feature sample training set, the filtered airborne feature sample validation set, and the filtered airborne feature sample test set, and construct and train a ground target analysis model based on the filtered ground feature sample training set, the filtered ground feature sample validation set, and the filtered ground feature sample test set to obtain the trained airborne target recognition model and the trained ground target analysis model; Optimize the trained airborne target recognition model according to the airborne target frequency domain feature enhanced target detection algorithm; Optimize the trained ground target analysis model according to scene adaptive fine-tuning; realize target detection and feature analysis based on the optimized airborne target recognition model, the optimized ground target analysis model, and the multi-modal feature fusion module.

Images