A multi-modal unmanned aerial vehicle detection method and system based on cooperative multi-agent reasoning

Abstract

The application discloses a multi-modal unmanned aerial vehicle detection method and system based on collaborative multi-agent reasoning. The method includes inputting radio spectrum observation data into a spectrum agent to obtain a spectrum candidate target set and a spectrum candidate confidence; inputting radar observation data into a radar agent to obtain a radar candidate target set and a radar candidate confidence; under the condition that consistency analysis is performed on any spectrum candidate target and any radar candidate target to obtain a consistency analysis result, a cross-modal candidate pair set is determined, a fusion strategy is determined based on the spectrum candidate target set, the radar candidate target set, the cross-modal candidate pair set, the spectrum candidate confidence and the radar candidate confidence; a current fusion weight value is determined based on a historical fusion weight value and a preset weight adjustment amount; and an unmanned aerial vehicle detection result is obtained through a fusion agent, thereby improving the robustness and detection result accuracy of unmanned aerial vehicle target detection.

Description

Technical Field

[0001] This application relates to the field of multimodal drone detection technology based on cooperative multi-agent reasoning, and in particular to a method and system for multimodal drone detection based on cooperative multi-agent reasoning. Background Technology

[0002] With the widespread application of low-altitude airspace and the rapid development of UAV technology, real-time and accurate detection and identification of UAVs in complex urban environments or field scenarios has become a key aspect of low-altitude safety management. Existing UAV detection systems generally employ multiple sensors, such as radar and radio spectrum data, for collaborative sensing, aiming to enhance detection capabilities through the complementary characteristics of multimodal data. However, radar observations are susceptible to weather conditions and terrain obstruction, while radio spectrum data is prone to electromagnetic interference and signal attenuation, resulting in significant fluctuations in the observation quality of different modalities in dynamic scenarios. Current technologies largely rely on fixed rules, preset thresholds, or static fusion mechanisms for target selection and decision-making. These methods, in environments with dense multi-target flight or strong interference, cannot adaptively adjust based on real-time candidate target distribution characteristics, confidence distribution, and changes in environmental conditions. This leads to an abnormally high false alarm rate or frequent missed detections of real targets, severely limiting the accuracy of detection results and the robustness of the system. Summary of the Invention

[0003] This application aims to at least address the technical problems existing in the prior art. To this end, this application proposes a multimodal UAV detection method and system based on cooperative multi-agent reasoning, which can improve the accuracy of UAV detection.

[0004] The first aspect of this application provides a multimodal unmanned aerial vehicle (UAV) detection method based on cooperative multi-agent reasoning, comprising the following steps: Acquire radio spectrum observation data within the current preset time period, radar observation data within the current preset time period, and historical fusion weight values ​​within the previous preset time period; The radio spectrum observation data is input into a spectrum agent to obtain a spectrum candidate target set output by the spectrum agent and a spectrum candidate confidence level corresponding to each spectrum candidate target; the radar observation data is input into a radar agent to obtain a radar candidate target set output by the radar agent and a radar candidate confidence level corresponding to each radar candidate target. If a consistency analysis is performed on any of the aforementioned spectrum candidate targets and any of the aforementioned radar candidate targets, and a consistency analysis result is obtained, a cross-modal candidate pair set is determined based on the consistency analysis result, wherein each cross-modal candidate pair in the cross-modal candidate pair set includes one of the aforementioned spectrum candidate targets and one of the aforementioned radar candidate targets; Based on the spectrum candidate target set, the radar candidate target set, the cross-modal candidate pair set, the spectrum candidate confidence score, and the radar candidate confidence score, a fusion strategy is determined, specifically as follows: Based on a preset confidence threshold, the spectrum candidate target set, the radar candidate target set, the cross-modal candidate pair set, the spectrum candidate confidence level, and the radar candidate confidence level, a missed detection risk value is determined; based on the preset confidence threshold, the spectrum candidate confidence level, and the radar candidate confidence level, a false alarm risk value is determined. The risk direction value is obtained by subtracting the false alarm risk value from the missed detection risk value. If the risk direction value is greater than the first preset risk threshold, the recall protection strategy will be used as the fusion strategy. If the risk direction value is greater than or equal to the second preset risk threshold and less than or equal to the first preset risk threshold, the balancing strategy will be used as the fusion strategy. Based on the historical fusion weight values ​​and the preset weight adjustment amount, the current fusion weight value is determined as follows: If the risk direction value is less than the second preset risk threshold, the accuracy protection strategy is used as the fusion strategy; based on the spectrum candidate target set, the radar candidate target set, and the cross-modal candidate pair set, the spectrum mode reliability value and the radar mode reliability value are determined; Based on the historical fusion weight values ​​and the preset maximum weight change, the spectrum fusion weight range is determined; Based on the spectrum mode reliability value, the radar mode reliability value, and the spectrum fusion weight range, the current fusion weight value is determined; The spectrum candidate target set, the radar candidate target set, the cross-modal candidate pair set, the fusion strategy, and the current fusion weight value are input into the fusion agent to obtain the UAV detection result output by the fusion agent.

[0005] The multimodal UAV detection method based on cooperative multi-agent reasoning according to the embodiments of this application has at least the following beneficial effects: This application acquires radio spectrum observation data and radar observation data from the current period, as well as historical fusion weight values ​​from the previous period. It then outputs corresponding candidate target sets and their confidence levels through a spectrum agent and a radar agent, respectively. After conducting consistency analysis on the candidate targets, a cross-modal candidate pair set is obtained, which in turn determines the appropriate fusion strategy. The current fusion weight value is updated by combining historical fusion weights with preset weight adjustment amounts and then fed into the fusion agent to output the UAV detection results. This approach achieves complementary advantages between spectrum and radar cross-modal information, dynamically adjusts the fusion weights based on data from previous and subsequent periods, adapts to complex observation scenarios, effectively reduces the missed and false detection problems caused by single-modal detection, and improves the robustness and accuracy of UAV target detection.

[0006] A second aspect of this application provides a multimodal unmanned aerial vehicle (UAV) detection system based on cooperative multi-agent reasoning, the multimodal UAV detection system based on cooperative multi-agent reasoning comprising: The data acquisition module is used to acquire radio spectrum observation data, radar observation data and historical fusion weight values ​​within the current preset time period. The intelligent agent output module is used to input the radio spectrum observation data into the spectrum intelligent agent to obtain the spectrum candidate target set output by the spectrum intelligent agent and the spectrum candidate confidence level corresponding to each spectrum candidate target; and to input the radar observation data into the radar intelligent agent to obtain the radar candidate target set output by the radar intelligent agent and the radar candidate confidence level corresponding to each radar candidate target. A cross-modal candidate pair set determination module is used to determine a cross-modal candidate pair set based on the consistency analysis results obtained when performing consistency analysis on any of the spectrum candidate targets and any of the radar candidate targets, wherein each cross-modal candidate pair in the cross-modal candidate pair set includes one of the spectrum candidate targets and one of the radar candidate targets; The fusion strategy determination module is used to determine a fusion strategy based on the spectrum candidate target set, the radar candidate target set, the cross-modal candidate pair set, the spectrum candidate confidence score, and the radar candidate confidence score, specifically: Based on a preset confidence threshold, the spectrum candidate target set, the radar candidate target set, the cross-modal candidate pair set, the spectrum candidate confidence level, and the radar candidate confidence level, a missed detection risk value is determined; based on the preset confidence threshold, the spectrum candidate confidence level, and the radar candidate confidence level, a false alarm risk value is determined. The risk direction value is obtained by subtracting the false alarm risk value from the missed detection risk value. If the risk direction value is greater than the first preset risk threshold, the recall protection strategy will be used as the fusion strategy. If the risk direction value is greater than or equal to the second preset risk threshold and less than or equal to the first preset risk threshold, the balancing strategy will be used as the fusion strategy. If the risk direction value is less than the second preset risk threshold, the accuracy protection strategy will be used as the fusion strategy. The current fusion weight value determination module is used to determine the current fusion weight value based on the historical fusion weight values ​​and the preset weight adjustment amount, specifically as follows: Based on the spectrum candidate target set, the radar candidate target set, and the cross-modal candidate pair set, the spectrum mode reliability value and the radar mode reliability value are determined; Based on the historical fusion weight values ​​and the preset maximum weight change, the spectrum fusion weight range is determined; Based on the spectrum mode reliability value, the radar mode reliability value, and the spectrum fusion weight range, the current fusion weight value is determined; The UAV detection result output module is used to input the spectrum candidate target set, the radar candidate target set, the cross-modal candidate pair set, the fusion strategy, and the current fusion weight value into the fusion agent to obtain the UAV detection result output by the fusion agent.

[0007] This system acquires radio spectrum observation data and radar observation data for the current period, as well as historical fusion weight values ​​from the previous period. It then outputs corresponding candidate target sets and their confidence levels through a spectrum agent and a radar agent, respectively. After conducting consistency analysis on the candidate targets, a cross-modal candidate pair set is obtained, which in turn determines the appropriate fusion strategy. The system updates the current fusion weight value by combining historical fusion weights with preset weight adjustments, and then sends this value back to the fusion agent to output the UAV detection results. This system achieves complementary advantages between spectrum and radar cross-modal information, dynamically adjusts the fusion weights based on data from previous and subsequent periods, adapts to complex observation scenarios, effectively reduces the missed and false detection problems caused by single-modal detection, and improves the robustness and accuracy of UAV target detection.

[0008] A third aspect of this application provides an electronic device including at least one processor and a memory for communicatively connecting to the processor; the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to cause the at least one processor to perform a multimodal unmanned aerial vehicle detection method based on cooperative multi-agent reasoning as described in the first aspect of this application.

[0009] A fourth aspect of this application provides a computer-readable storage medium storing computer-executable instructions for causing a computer to perform a multimodal unmanned aerial vehicle (UAV) detection method based on cooperative multi-agent reasoning as described in the first aspect of this application.

[0010] It should be noted that the beneficial effects of the third and fourth aspects of this application with the prior art are the same as the beneficial effects of the above-mentioned multimodal UAV detection method based on collaborative multi-agent reasoning with the prior art, and will not be described in detail here.

[0011] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0012] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which: Figure 1 This is a flowchart illustrating an embodiment of the multimodal unmanned aerial vehicle (UAV) detection method based on collaborative multi-agent reasoning provided in this application; Figure 2 This is a schematic diagram of the overall process of an embodiment of the multimodal UAV detection method based on collaborative multi-agent reasoning provided in this application; Figure 3 This is a schematic diagram of the structure of an embodiment of the multimodal unmanned aerial vehicle (UAV) detection system based on collaborative multi-agent reasoning provided in this application; Figure 4 This is a schematic diagram of the structure of an embodiment of the electronic device provided in this application. Detailed Implementation

[0013] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0014] In the description of this application, the use of terms such as "first," "second," etc., is for the purpose of distinguishing technical features only and should not be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated or the order of the technical features indicated.

[0015] In the description of this application, it should be understood that the orientation descriptions, such as up, down, etc., are based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.

[0016] In the description of this application, it should be noted that, unless otherwise explicitly defined, terms such as "setup," "installation," and "connection" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this application in conjunction with the specific content of the technical solution.

[0017] With the widespread application of low-altitude airspace and the rapid development of UAV technology, real-time and accurate detection and identification of UAVs in complex urban environments or field scenarios has become a key aspect of low-altitude safety management. Existing UAV detection systems generally employ multiple sensors, such as radar and radio spectrum data, for collaborative sensing, aiming to enhance detection capabilities through the complementary characteristics of multimodal data. However, radar observations are susceptible to weather conditions and terrain obstruction, while radio spectrum data is prone to electromagnetic interference and signal attenuation, resulting in significant fluctuations in the observation quality of different modalities in dynamic scenarios. Current technologies largely rely on fixed rules, preset thresholds, or static fusion mechanisms for target selection and decision-making. These methods, in environments with dense multi-target flight or strong interference, cannot adaptively adjust based on real-time candidate target distribution characteristics, confidence distribution, and changes in environmental conditions. This leads to an abnormally high false alarm rate or frequent missed detections of real targets, severely limiting the accuracy of detection results and the robustness of the system.

[0018] To address the aforementioned technical deficiencies, this application provides a multimodal unmanned aerial vehicle (UAV) detection method and system based on collaborative multi-agent reasoning.

[0019] Please see Figure 1 This is a flowchart illustrating a multimodal UAV detection method based on collaborative multi-agent reasoning provided in an embodiment of this application. This method is applied to electronic devices, such as servers. Figure 1 As shown, the multimodal UAV detection method based on cooperative multi-agent reasoning includes: Step S101: Obtain radio spectrum observation data within the current preset time period, radar observation data within the current preset time period, and historical fusion weight values ​​within the previous preset time period; The preset time period can be a value set in advance according to actual needs, and can be one second.

[0020] Step S102: Input radio spectrum observation data into the spectrum agent to obtain the spectrum candidate target set output by the spectrum agent and the spectrum candidate confidence level corresponding to each spectrum candidate target; input radar observation data into the radar agent to obtain the radar candidate target set output by the radar agent and the radar candidate confidence level corresponding to each radar candidate target. The aforementioned spectrum agent is a functional entity that performs spectrum candidate target analysis on radio spectrum observation data to generate a set of spectrum candidate targets and the spectrum candidate confidence level corresponding to each spectrum candidate target. Qwen-7B can be used as the inference backbone.

[0021] The aforementioned radar agent is another functional entity that performs radar candidate target analysis on radar observation data to generate a set of radar candidate targets and the radar candidate confidence level corresponding to each radar candidate target. Qwen-7B can be used as the inference backbone.

[0022] The aforementioned spectrum candidate target set can be a list of candidate targets identified by the spectrum agent from radio spectrum observation data.

[0023] The aforementioned radar candidate target set can be a list of candidate targets identified by the radar agent from radar observation data.

[0024] The aforementioned spectrum candidate confidence level is used to characterize the probability assessment of a spectrum candidate target being a candidate target, and can be a value between 0 and 1.

[0025] The aforementioned radar candidate confidence level is used to characterize the probability assessment of a radar candidate target being a candidate target, and can be a value between 0 and 1.

[0026] In step S102, the above-mentioned inputting radio spectrum observation data into the spectrum agent to obtain the spectrum candidate target set output by the spectrum agent and the spectrum candidate confidence level corresponding to each spectrum candidate target; inputting radar observation data into the radar agent to obtain the radar candidate target set output by the radar agent and the radar candidate confidence level corresponding to each radar candidate target may include: Radio spectrum observation data is input into a spectrum agent to extract a first feature from the radio spectrum observation data. The spectrum agent then identifies a set of spectrum candidate targets from the first feature and outputs the set of spectrum candidate targets and the spectrum candidate confidence level corresponding to each spectrum candidate target. The first feature may include time-frequency features, signal azimuth features, signal strength features, and signal format features. Radar observation data is input into a radar agent to extract a second feature from the radar observation data. The radar agent then identifies a set of radar candidate targets from the second feature and outputs the set of radar candidate targets and the radar candidate confidence score corresponding to each radar candidate target. The second feature may include target spatial location features, motion state features, and radar target attribute features.

[0027] Step S103: After performing consistency analysis on any spectrum candidate target and any radar candidate target and obtaining the consistency analysis results, determine the cross-modal candidate pair set based on the consistency analysis results. Each cross-modal candidate pair in the cross-modal candidate pair set includes one spectrum candidate target and one radar candidate target. In step S103, the consistency analysis of any spectrum candidate target and any radar candidate target is performed. The consistency analysis result can be the extraction of the first timestamp, first azimuth angle, first position, and first target attribute corresponding to the spectrum candidate target output by the spectrum agent, and the extraction of the second timestamp, second azimuth angle, second position, and second target attribute corresponding to the radar candidate target output by the radar agent. If the first timestamp and second timestamp are the same, the difference between the first azimuth angle and the second azimuth angle is less than a preset azimuth threshold according to actual needs, the difference between the first position and the second position is less than a preset position threshold according to actual needs, and the first target attribute and the second target attribute are the same, the first preset analysis result is taken as the consistency analysis result between the corresponding spectrum candidate target and the radar candidate target. If the first timestamp and second timestamp are different, the difference between the first azimuth angle and the second azimuth angle is greater than or equal to a preset azimuth threshold according to actual needs, the difference between the first position and the second position is greater than or equal to a preset position threshold according to actual needs, and the first target attribute and the second target attribute are different, the second preset analysis result is taken as the consistency analysis result between the corresponding spectrum candidate target and the radar candidate target. The target attribute can include UAVs and birds.

[0028] Step S104: Based on the spectrum candidate target set, radar candidate target set, cross-modal candidate pair set, spectrum candidate confidence score, and radar candidate confidence score, determine the fusion strategy, specifically as follows: Step S1041: Based on the preset confidence threshold, the spectrum candidate target set, the radar candidate target set, the cross-modal candidate pair set, the spectrum candidate confidence, and the radar candidate confidence, determine the missed detection risk value; based on the preset confidence threshold, the spectrum candidate confidence, and the radar candidate confidence, determine the false alarm risk value. The aforementioned preset reliability threshold can be a value preset according to actual needs.

[0029] Step S1042: Subtract the false alarm risk value from the missed detection risk value to obtain the risk direction value; Step S1043: If the risk direction value is greater than the first preset risk threshold, the recall protection strategy is used as the fusion strategy. Step S1044: When the risk direction value is greater than or equal to the second preset risk threshold and less than or equal to the first preset risk threshold, the balancing strategy is used as the fusion strategy. Step S1045: If the risk direction value is less than the second preset risk threshold, the accuracy protection strategy is used as the fusion strategy. The recall protection strategy described above can be used to increase the retention tendency of potential valid candidates when the risk of missed detection is significantly higher than the risk of false alarm, so as to reduce the possibility of missing the real target; the accuracy protection strategy can be used to strengthen the suppression of low-reliability candidates when the risk of false alarm is significantly higher than the risk of missed detection, so as to reduce the output of erroneous targets; the balancing strategy can be used to balance candidate retention and false alarm suppression when the two types of risks are close.

[0030] Step S105: Determine the current fusion weight value based on historical fusion weight values ​​and preset weight adjustment amounts, specifically as follows: Step S1051: Based on the spectrum candidate target set, the radar candidate target set, and the cross-modal candidate pair set, determine the spectrum mode reliability value and the radar mode reliability value; The aforementioned second preset risk threshold can be a value pre-set according to actual needs.

[0031] Step S1052: Determine the spectrum fusion weight range based on historical fusion weight values ​​and the preset maximum weight change. The maximum change in the preset weight can be a value that is preset according to actual needs.

[0032] Step S1053: Determine the current fusion weight value based on the spectrum mode reliability value, radar mode reliability value, and spectrum fusion weight range; Step S106: Input the spectrum candidate target set, radar candidate target set, cross-modal candidate pair set, fusion strategy and current fusion weight value into the fusion agent to obtain the UAV detection results output by the fusion agent.

[0033] The aforementioned fusion agent is a functional entity that performs multimodal target collaborative reasoning and fusion confirmation based on the spectrum candidate target set, radar candidate target set, cross-modal candidate pair set, fusion strategy, and current fusion weight value to obtain UAV detection results. Qwen can be used as the reasoning backbone.

[0034] It should be noted that different intelligent agents process the input information independently according to their respective tasks.

[0035] In step S106, the aforementioned input of the spectrum candidate target set, radar candidate target set, cross-modal candidate pair set, fusion strategy, and current fusion weight value into the fusion agent yields the UAV detection result output by the fusion agent. This result can be obtained by receiving candidate target features, confidence levels, cross-modal matching information, and the currently determined fusion strategy and weights from the input layer of the fusion agent, and outputting the final UAV detection result based on the learned patterns. For example, the fusion agent receives the spectrum candidate target set, radar candidate target set, cross-modal candidate pair set, fusion strategy, and current fusion weight value. If the current fusion strategy is a "balanced strategy," the fusion agent will perform a weighted average of the candidate targets from the two modalities based on the current fusion weight value, and combine it with the cross-modal candidate pair information to output the final UAV target list, including the precise location and identification category of each target.

[0036] This application acquires radio spectrum observation data and radar observation data from the current period, as well as historical fusion weight values ​​from the previous period. It then outputs corresponding candidate target sets and their confidence levels through a spectrum agent and a radar agent, respectively. After conducting consistency analysis on the candidate targets, a cross-modal candidate pair set is obtained, which in turn determines the appropriate fusion strategy. The current fusion weight value is updated by combining historical fusion weights with preset weight adjustment amounts and then fed into the fusion agent to output the UAV detection results. This approach achieves complementary advantages between spectrum and radar cross-modal information, dynamically adjusts the fusion weights based on data from previous and subsequent periods, adapts to complex observation scenarios, effectively reduces the missed and false detection problems caused by single-modal detection, and improves the robustness and accuracy of UAV target detection.

[0037] In some embodiments, step S101 may include steps S201 to S203: Step S201: Obtain initial radio spectrum observation data and initial radar observation data within the current preset time period; In step S201, the acquisition of initial radio spectrum observation data and initial radar observation data within the current preset time period can be achieved by measuring the initial radio spectrum observation data within the current preset time period using a radio spectrum sensor and by measuring the initial radar observation data within the current preset time period using a radar sensor.

[0038] Step S202: Preprocess the initial radio spectrum observation data and initial radar observation data to obtain radio spectrum observation data and radar observation data; In step S202, the preprocessing of the initial radio spectrum observation data and initial radar observation data to obtain the radio spectrum observation data and radar observation data may include: The initial radar observation data is analyzed to extract the timestamp, target position, range, speed, heading, and signal-to-noise ratio, which serve as the first analysis data. The initial radio spectrum observation data is also analyzed to extract the timestamp, azimuth, frequency, bandwidth, confidence level, signal strength, and modulation type, which serve as the second analysis data. Sort the first parsed data according to timestamps to obtain the sorted first parsed data; sort the second parsed data according to timestamps to obtain the sorted second parsed data. The first set of parsed data after sorting is manually processed to remove abnormal data, resulting in the first set of parsed data after filtering. The second set of parsed data after sorting is also manually processed to remove abnormal data, resulting in the second set of parsed data after filtering. Abnormal data may include data with missing key fields. The filtered first analytical data is organized into a preset data format according to actual needs to obtain radar observation data; the filtered second analytical data is organized into a preset data format according to actual needs to obtain radio spectrum observation data.

[0039] Step S203: Extract the pre-stored fusion weight values ​​from the previous preset time period as historical fusion weight values.

[0040] In step S203, the above-mentioned extraction of the fusion weight value within the previous preset time period as the historical fusion weight value can be the extraction of the fusion weight value within the previous preset time period from the preset storage area preset according to actual needs as the historical fusion weight value.

[0041] This application obtains initial radio spectrum observation data and initial radar observation data for the current period, performs preprocessing on the two types of raw observation data to obtain radio spectrum observation data and radar observation data that can be used as model input, and retrieves the fusion weight values ​​pre-stored in the previous preset time period as historical fusion weight values. This can eliminate noise and abnormal interference in the raw observation data, improve the quality of input data, and reuse the weight information of the historical period to provide a benchmark for weight iteration updates, ensuring the continuity of cross-modal fusion weight time-series updates, and providing a regular and reliable input data source for subsequent model inference, target matching, and UAV fusion detection.

[0042] In some embodiments, step S103 may include steps S301 to S303: Step S301: Filter out all consistency analysis results that are the first preset analysis results, and use them as the set of consistency analysis results to be filtered; The first preset analysis result mentioned above can be a value preset according to actual needs.

[0043] Step S302: Take the spectrum candidate target and the radar candidate target corresponding to each consistency analysis result in the set of consistency analysis results to be screened as the corresponding cross-modal candidate pair; Step S303: Combine all cross-modal candidate pairs to obtain a cross-modal candidate pair set.

[0044] This application selects consistency analysis results that meet preset analysis results to form a set of consistency analysis results to be screened. It extracts the spectral candidate targets and radar candidate targets corresponding to each result to form cross-modal candidate pairs, and summarizes them to obtain a set of cross-modal candidate pairs. This can effectively match the output targets of different modes, filter out target pairings that do not meet the matching degree, reduce the interference caused by incorrect pairing, and provide an accurate and reliable pairing sample basis for subsequent modal reliability assessment, weight calculation and target fusion detection. This is conducive to reducing the false alarm risk of UAV cross-modal target detection and improving the overall accuracy of fusion detection.

[0045] In some embodiments, step S1041 may include steps S401 to S407: Step S401: Count the total number of spectrum candidate targets in the spectrum candidate target set, and use it as the first target number; Step S402: Count the total number of radar candidate targets in the radar candidate target set, and use it as the second target number; Step S403: Count the total number of cross-modal candidate pairs in the cross-modal candidate pair set, and use it as the number of third targets; Step S404: Determine the proportion of single-modal candidates based on the number of first targets, the number of second targets, and the number of third targets; In step S404, determining the single-modal candidate proportion based on the number of first targets, the number of second targets, and the number of third targets may include: All spectral candidate targets and all radar candidate targets in the cross-modal candidate pair set are classified as dual-modal candidate types; all spectral candidate targets that do not belong to the cross-modal candidate pair set are classified as spectral single-modal candidate types; and all radar candidate targets that do not belong to the cross-modal candidate pair set are classified as radar single-modal candidate types. The total number of candidate targets is obtained by subtracting the number of third targets from the sum of the number of first targets and the number of second targets. Based on the total number of candidate targets and the number of third targets, the proportion of single-modal candidates is calculated using the following formula: ; ; in, This represents the proportion of dual-modal candidates. The number of the third target, The total number of candidate targets. This represents the proportion of single-modal candidates.

[0046] Step S405: Based on the preset confidence threshold, spectrum candidate confidence, and radar candidate confidence, determine the proportion of medium confidence candidates, low confidence candidates, and invalid confidence candidates. The aforementioned preset reliability threshold can be a value preset according to actual needs. The preset reliability threshold may include a first preset reliability threshold, a second preset reliability threshold, and a third preset reliability threshold preset according to actual needs. The first preset reliability threshold is greater than the second preset reliability threshold, and the second preset reliability threshold is greater than the third preset reliability threshold.

[0047] In step S405, determining the proportion of candidates with medium confidence, low confidence, and invalid confidence based on a preset confidence threshold, spectrum candidate confidence, and radar candidate confidence may include: All spectrum candidate confidence scores and all radar candidate confidence scores that are less than or equal to the first preset confidence threshold and greater than the second preset confidence threshold are classified as medium confidence scores; all spectrum candidate confidence scores and all radar candidate confidence scores that are less than or equal to the second preset confidence threshold and greater than the third preset confidence threshold are classified as low confidence scores; and all spectrum candidate confidence scores and all radar candidate confidence scores that are less than or equal to the third preset confidence threshold are classified as invalid confidence scores. Add the number of the first target and the number of the second target to get the first sum; The total number of spectrum candidate targets and radar candidate targets corresponding to medium confidence level is counted as the total number of medium confidence candidate targets; the total number of spectrum candidate targets and radar candidate targets corresponding to low confidence level is counted as the total number of low confidence candidate targets; the total number of spectrum candidate targets and radar candidate targets corresponding to invalid confidence level is counted as the total number of invalid confidence candidate targets. Divide the total number of medium-confidence candidate targets by the first sum to obtain the proportion of medium-confidence candidates; Divide the total number of low-confidence candidate targets by the first sum to obtain the proportion of low-confidence candidates; Divide the total number of invalid confidence candidate targets by the first sum to obtain the invalid confidence candidate ratio.

[0048] Step S406: Determine the missed detection risk value based on the single-modality candidate proportion and the medium-confidence candidate proportion; In step S406, the determination of the missed detection risk value based on the single-modality candidate proportion and the medium-confidence candidate proportion may include: The third product value is obtained by multiplying the single-modal candidate proportion value with the third contribution coefficient value pre-set according to actual needs; The fourth product is obtained by multiplying the proportion of candidates with medium confidence by the fourth contribution coefficient value pre-set according to actual needs; Add the third product value to the fourth product value to obtain the false negative risk value.

[0049] Step S407: Determine the false alarm risk value based on the proportion of low-confidence candidates and the proportion of invalid-confidence candidates.

[0050] In step S407, determining the false alarm risk value based on the proportion of low-confidence candidates and the proportion of invalid-confidence candidates may include: The first product value is obtained by multiplying the proportion of low-confidence candidates by the first contribution coefficient value pre-set according to actual needs; The invalid confidence candidate ratio is multiplied by the second contribution coefficient value pre-set according to actual needs to obtain the second product value; Add the first product value to the second product value to obtain the false alarm risk value.

[0051] This application obtains the number of first targets, second targets, and third targets through statistics, calculates the proportion of single-modal candidates based on these numbers, and further solves for the false alarm risk value and the missed detection risk value by combining a pre-set confidence threshold and the confidence value corresponding to each candidate target. Based on the two types of risk values, the corresponding fusion strategy is determined. This approach can quantitatively assess the level of missed detection and false alarm risks in the current observation scenario, and adaptively match the appropriate fusion processing strategy according to the actual risk situation. This avoids the problem of poor adaptability caused by using a fixed fusion mode, which is conducive to balancing the contradiction between missed detection and false alarm and improving the overall detection performance of UAV cross-modal target detection in complex scenarios.

[0052] In some embodiments, step S1051 may include steps S501 to S503: Step S501: Based on the first corresponding ID of the spectrum candidate target output by the spectrum agent and the second corresponding ID of the radar candidate target output by the radar agent, extract the candidate types (including dual-mode candidate type, spectrum single-mode candidate type, and radar single-mode candidate type) corresponding to the spectrum candidate target and radar candidate target within the preset iteration historical time period. Mark the spectrum candidate target and radar candidate target in the cross-modal candidate pair set as strong positive feedback; mark the spectrum candidate target and radar candidate target that appear K consecutively in the preset iteration historical time period (i.e., the corresponding ID also exists in the most recent multiple historical preset time periods) and whose candidate type is spectrum single-mode candidate type or radar single-mode candidate type as weak positive feedback; mark the spectrum candidate target and radar candidate target that appear only once in the preset iteration historical time period as negative feedback. The preset iteration historical time period includes the current preset time period and the most recent multiple historical preset time periods, and K is a value preset according to actual needs. Step S502: Historically set feedback tags and feedback strengths for each spectrum candidate target and radar candidate target. The feedback tag value for spectrum candidate targets or radar candidate targets marked as weak positive feedback or strong positive feedback is one; the feedback tag value for spectrum candidate targets or radar candidate targets marked as negative feedback is negative one; the feedback strength value for spectrum candidate targets or radar candidate targets marked as strong positive feedback is one; the feedback strength value for spectrum candidate targets or radar candidate targets marked as weak positive feedback is a first preset feedback strength set according to actual needs; the feedback strength value for spectrum candidate targets or radar candidate targets marked as negative feedback is a second preset feedback strength set according to actual needs, wherein the second preset feedback strength is less than the first preset feedback strength. Step S503: Calculate the spectral mode reliability value and the radar mode reliability value using the following formulas: ; ; ; in, For the first The modal type within the preset time period is The positive feedback evidence value, For the first The modal type within the preset time period is The negative feedback evidence value, For the pre-acquired first The modal type within the preset time period is The positive feedback evidence value, For the pre-acquired first The modal type within the preset time period is The negative feedback evidence value, This indicates the candidate type (spectral mode or radar mode) corresponding to the candidate target. For the first The set of candidate targets within a preset time period. Indicates the first The candidate target pairs have the following modal types: The preset feedback allocation coefficient is set to 1 when the candidate type is a spectrum single-mode candidate type or a radar single-mode candidate type, and to half when the candidate type is a dual-mode candidate type, and not belonging to the mode. When the value is 0); Indicates the first The feedback strength of each candidate target, Indicates an indicator function, This represents the coefficient value of the time decay factor, which is preset according to actual needs. This indicates the time interval between the generation of feedback and the current update. For the first The modal type within the preset time period is Reliability value, The first prior smoothing parameter is set in advance according to actual needs. This is a second prior smoothing parameter pre-set according to actual needs.

[0053] This application calculates the spectral modal reliability value and the radar modal reliability value based on the spectral candidate target set, the radar candidate target set, and the cross-modal candidate pair set, providing a more accurate data basis for the subsequent calculation of the current fusion weight value, thereby improving the accuracy of the UAV cross-modal target detection results.

[0054] In some embodiments, step S1052 may include step S601: Step S601: Based on the historical fusion weight values ​​and the preset maximum weight change, calculate the spectrum fusion weight range using the following formula: ; ; in, For the first The minimum value of the spectrum fusion weight range within the preset time period. For the first The maximum value of the spectrum fusion weight range within the preset time period. The minimum weight is pre-set according to actual needs. The maximum weight is preset according to actual needs. For historical fusion weight values, This represents the maximum change in the preset weight.

[0055] This application calculates the spectrum fusion weight range based on historical fusion weight values ​​and the maximum change in preset weights, providing more accurate data for subsequent calculation of the current fusion weight value, thereby improving the accuracy of UAV cross-modal target detection results.

[0056] In some embodiments, step S1053 may include step S701: Step S701: Based on the spectrum mode reliability value, radar mode reliability value, and spectrum fusion weight range, calculate the current fusion weight value using the following formula: ; ; in, This represents the current fusion weight value for the spectral modes. This represents the current fusion weight value for the radar modes. This represents the reliability value of the spectral modes. This represents the radar modal reliability value. A positive number pre-set according to actual needs. This refers to the pre-set weight adjustment amount based on actual needs.

[0057] This application limits the spectral fusion weight range by combining historical fusion weight values ​​and preset maximum weight change, and then calculates the current fusion weight value within this weight range based on the reliability values ​​of the two modes. This allows for the evaluation of the reliability of each mode in conjunction with the current actual detection results. At the same time, by constraining the weight adjustment range through the maximum weight change, it avoids sudden changes in weight that could cause oscillations in the detection results. This enables the fusion weight to be updated adaptively and reasonably with the reliability of the modes, giving full play to the detection advantages of high-reliability modes and improving the stability and accuracy of UAV cross-modal target detection results.

[0058] To facilitate understanding by those skilled in the art, a set of experimental examples are provided below: Reference Figure 2 In a real outdoor environment, a multimodal perception platform was built, which includes radar equipment, radio spectrum equipment, and protocol decoding equipment. The radar equipment is used to independently acquire target motion and spatial state information, the radio spectrum equipment is used to acquire the radio signal characteristics of UAVs, and the protocol decoding equipment is used to acquire UAV identity and geographical location information.

[0059] Data collection was conducted during multiple outdoor flights in open areas and areas surrounded by buildings, covering natural variations such as weather, visibility, background clutter, and electromagnetic interference. In single-target scenarios, each flight involved one drone; in multi-target scenarios, three heterogeneous drones were used. The experiments involved DJI Mavic 4 Pro, Air 3, Mavic 3 Pro, and other commercial or custom drones. Each flight lasted approximately 10 minutes and covered typical motion states such as cruising, hovering, circling, approaching, gathering, dispersing, and returning to home.

[0060] Each sensor observation is transmitted in the form of TCP data frames and synchronized based on millisecond-level timestamps and WGS84 coordinates. After time synchronization, data quality filtering, and removal of incomplete records, a total of 2046 effective time windows of 1 second in length are obtained, corresponding to 4114 offline reference UAV targets; among them, 995 are single-target time windows and 1051 are multi-target time windows, with a total of 3119 reference targets in the multi-target time windows. To ensure that the agent inference process in this embodiment has a clear implementation form, the high-performance Qwen-7B can be used as the inference backbone of the spectrum agent, radar agent, and fusion agent.

[0061] In the Within a 1-second time window, receive TCP data frames output by radio spectrum equipment and radar equipment.

[0062] First, the data frames are parsed: fields such as target position, range, velocity, heading, and signal-to-noise ratio are extracted from radar data; fields such as azimuth, frequency, bandwidth, device output confidence, signal strength, and modulation type are extracted from spectrum data. Then, the two types of data are sorted and synchronized according to millisecond-level timestamps, and observations falling within the same 1-second time window are organized into multimodal observations at the same time.

[0063] The analyzed observations undergo integrity checks and quality screening to remove records with analytical anomalies, missing key fields, or those unusable for subsequent target analysis. The retained data is organized according to a unified field format. The spectral structured representation includes time-frequency characteristics, signal azimuth, signal strength, and signal format or equipment-related information; the radar structured representation includes target range, azimuth, altitude, motion state, trajectory continuity, and radar target attributes. The preprocessed multimodal structured data is represented as follows: ; ; in, This indicates the raw spectrum observation data (initial radio spectrum observation data) acquired by radio spectrum equipment within this time window. This represents the raw radar observation data (initial radar observation data). This refers to the data frame parsing, time synchronization, abnormal record removal, data quality filtering, and field structuring processing described above. and These represent the spectrum structured representation (radio spectrum observation data) and the radar structured representation (radar observation data), respectively. This represents the complete multimodal structured data for the current time window.

[0064] Will and Inputting data into both a spectrum agent and a radar agent allows the spectrum agent to read time-frequency patterns, signal azimuth, signal strength, and signal format or device-related characteristics, enabling joint analysis of spectrum observations within the same time window. When a single observation or a group of observations provides valid support for a potential UAV target, a spectrum candidate target is generated, and its candidate identifier, target attributes, candidate confidence level, and corresponding spectrum evidence are recorded. The radar agent reads the target's spatial location, motion characteristics, trajectory continuity, and radar target attributes, performing target correlation analysis on radar observations. For observations that provide valid support for a UAV target, radar candidate targets are generated, and corresponding spatial, motion, and trajectory evidence is retained.

[0065] The two modal candidate target sets are represented as follows: ; in, This represents the candidate target generation process performed by two modal agents; This represents the set of candidate targets for the spectrum in the current time window; This represents the set of radar candidate targets for the current time window; This represents a complete set of candidates consisting of two categories. Each candidate includes at least a candidate identifier, target attributes, candidate confidence level, and sensor observation evidence supporting the candidate.

[0066] right and Consistency analysis is performed between any spectrum candidate target and any radar candidate target. The consistency analysis result can be the first timestamp, first azimuth, first position, and first target attribute corresponding to the spectrum candidate target output by the spectrum agent, and the second timestamp, second azimuth, second position, and second target attribute corresponding to the radar candidate target output by the radar agent. If the first timestamp and second timestamp are the same, the difference between the first azimuth and second azimuth is less than a preset azimuth threshold according to actual needs, the difference between the first position and second position is less than a preset position threshold according to actual needs, and the first target attribute and second target attribute are the same, the first preset analysis result is taken as the consistency analysis result between the corresponding spectrum candidate target and radar candidate target. If the first timestamp and second timestamp are different, the difference between the first azimuth and second azimuth is greater than or equal to a preset azimuth threshold according to actual needs, the difference between the first position and second position is greater than or equal to a preset position threshold according to actual needs, and the first target attribute and second target attribute are different, the second preset analysis result is taken as the consistency analysis result between the corresponding spectrum candidate target and radar candidate target. The target attribute can include UAVs and birds.

[0067] Select all consistency analysis results that are the first preset analysis results, and use them as the set of consistency analysis results to be screened; In the set of consistency analysis results to be screened, the spectrum candidate target and the radar candidate target corresponding to each consistency analysis result to be screened are taken as the corresponding cross-modal candidate pair; Combining all cross-modal candidate pairs yields a cross-modal candidate pair set; the cross-modal association process can be represented as: ; in, This represents the cross-modal candidate association process; Indicates the first Within a time window, a one-to-one spectrum-radar candidate association set (cross-modal candidate pair set) is established. Candidates that have been successfully associated are denoted as bimodal support candidates (bimodal candidate type). Spectrum candidates and radar candidates that have not been associated are retained as spectrum single-modal candidates (spectrum single-modal candidate type) and radar single-modal candidates (radar single-modal candidate type), respectively.

[0068] The total number of spectrum candidate targets in the statistical spectrum candidate target set is taken as the number of first targets; The total number of radar candidate targets in the radar candidate target set is used as the number of second targets; The total number of cross-modal candidate pairs in the cross-modal candidate pair set is used as the number of third objectives; All spectral candidate targets and all radar candidate targets in the cross-modal candidate pair set are classified as dual-modal candidate types; all spectral candidate targets that do not belong to the cross-modal candidate pair set are classified as spectral single-modal candidate types; and all radar candidate targets that do not belong to the cross-modal candidate pair set are classified as radar single-modal candidate types. The total number of candidate targets is obtained by subtracting the number of third targets from the sum of the number of first targets and the number of second targets. Based on the total number of candidate targets and the number of third targets, the proportion of single-modal candidates is calculated using the following formula: ; ; in, This represents the proportion of dual-modal candidates. The number of the third target, The total number of candidate targets. This represents the proportion of single-modal candidates.

[0069] All spectrum candidate confidence scores and all radar candidate confidence scores that are less than or equal to the second preset confidence threshold and greater than the third preset confidence threshold are classified as low confidence scores; all spectrum candidate confidence scores and all radar candidate confidence scores that are less than or equal to the third preset confidence threshold are classified as invalid confidence scores. Add the number of the first target and the number of the second target to get the first sum; The total number of spectrum candidate targets and radar candidate targets corresponding to low confidence is counted as the total number of low confidence candidate targets; the total number of spectrum candidate targets and radar candidate targets corresponding to invalid confidence is counted as the total number of invalid confidence candidate targets. Divide the total number of low-confidence candidate targets by the first sum to obtain the proportion of low-confidence candidates; divide the total number of invalid confidence candidate targets by the first sum to obtain the proportion of invalid confidence candidates. The first product value is obtained by multiplying the proportion of low-confidence candidates by the first contribution coefficient value pre-set according to actual needs; The invalid confidence candidate ratio is multiplied by the second contribution coefficient value pre-set according to actual needs to obtain the second product value; Add the first product value to the second product value to obtain the false alarm risk value.

[0070] Specifically, the formula for calculating the false alarm risk value can be expressed as: ; in, This is the risk value for false alarms. The first contribution coefficient value, The proportion of candidates with low confidence. This is the second contribution coefficient value. This represents the proportion of invalid confidence candidates.

[0071] All spectrum candidate confidence levels and all radar candidate confidence levels that are less than or equal to the first preset confidence threshold and greater than the second preset confidence threshold are classified as medium confidence levels. The total number of spectrum candidate targets and radar candidate targets corresponding to the medium confidence level is counted as the total number of medium confidence candidate targets; Divide the total number of medium-confidence candidate targets by the first sum to obtain the proportion of medium-confidence candidates.

[0072] The third product value is obtained by multiplying the single-modal candidate proportion value with the third contribution coefficient value pre-set according to actual needs; The fourth product is obtained by multiplying the proportion of candidates with medium confidence by the fourth contribution coefficient value pre-set according to actual needs; Add the third product value to the fourth product value to obtain the false negative risk value.

[0073] Specifically, the formula for calculating the risk value of missed detection can be expressed as: ; in, This represents the risk value for missed detection. This is the third contribution coefficient value. This represents the proportion of single-modal candidates. This is the fourth contribution coefficient value. The proportion of candidates with medium confidence; The risk difference between the two types of risks is calculated using the following formula, and is used as the risk direction value: ; in, This represents the risk direction value. Use strategy to switch thresholds ,when When a recall protection strategy is adopted, priority is given to retaining potentially real targets; when A precision protection strategy is employed to enhance the suppression of low-quality candidates; when A balancing strategy is adopted. This leads to the fusion strategy for the current time window.

[0074] Based on the first corresponding ID of the spectrum candidate target output by the spectrum agent and the second corresponding ID of the radar candidate target output by the radar agent, the candidate types (including dual-mode candidate type, spectrum single-mode candidate type, and radar single-mode candidate type) corresponding to the spectrum candidate target and radar candidate target are extracted within a preset iteration historical time period. The spectrum candidate targets and radar candidate targets in the cross-modal candidate pair set are marked as strong positive feedback. The spectrum candidate targets and radar candidate targets that appear K consecutively in the preset iteration historical time period (i.e., the corresponding ID also exists in the most recent multiple historical preset time periods) and whose candidate type is spectrum single-mode candidate type or radar single-mode candidate type are marked as weak positive feedback. The spectrum candidate targets and radar candidate targets that appear only once in the preset iteration historical time period are marked as negative feedback. The preset iteration historical time period includes the current preset time period and the most recent multiple historical preset time periods, and K is a value preset according to actual needs. For each spectrum candidate target and radar candidate target, a feedback label and feedback intensity are historically set. The feedback label for spectrum candidate targets or radar candidate targets marked as weak positive feedback or strong positive feedback is set to one; the feedback label for spectrum candidate targets or radar candidate targets marked as negative feedback is set to negative one; the feedback intensity for spectrum candidate targets or radar candidate targets marked as strong positive feedback is set to one; the feedback intensity for spectrum candidate targets or radar candidate targets marked as weak positive feedback is set to a first preset feedback intensity based on actual needs; and the feedback intensity for spectrum candidate targets or radar candidate targets marked as negative feedback is set to a second preset feedback intensity based on actual needs, wherein the second preset feedback intensity is less than the first preset feedback intensity. Step S503: Calculate the spectral mode reliability value and the radar mode reliability value using the following formulas: ; ; ; in, For the first The modal type within the preset time period is The positive feedback evidence value, For the first The modal type within the preset time period is The negative feedback evidence value, For the pre-acquired first The modal type within the preset time period is The positive feedback evidence value, For the pre-acquired first The modal type within the preset time period is The negative feedback evidence value, This indicates the candidate type (spectral mode or radar mode) corresponding to the candidate target. For the first The set of candidate targets within a preset time period. Indicates the first The candidate target pairs have the following modal types: The preset feedback allocation coefficient is set to 1 when the candidate type is a spectrum single-mode candidate type or a radar single-mode candidate type, and to half when the candidate type is a dual-mode candidate type, and not belonging to the mode. When the value is 0); Indicates the first The feedback strength of each candidate target, Indicates an indicator function, This represents the coefficient value of the time decay factor, which is preset according to actual needs. This indicates the time interval between the generation of feedback and the current update. For the first The modal type within the preset time period is Reliability value, The first prior smoothing parameter is set in advance according to actual needs. This is a second prior smoothing parameter pre-set according to actual needs; Based on historical fusion weight values ​​and the maximum preset weight change, the spectrum fusion weight range is calculated using the following formula: ; ; in, For the first The minimum value of the spectrum fusion weight range within the preset time period. For the first The maximum value of the spectrum fusion weight range within the preset time period. The minimum weight is pre-set according to actual needs. The maximum weight is preset according to actual needs. For historical fusion weight values, This represents the maximum change in the preset weight.

[0075] Based on the spectrum modal reliability value, radar modal reliability value, and spectrum fusion weight range, the current fusion weight value is calculated using the following formula: ; ; in, This represents the current fusion weight value for the spectral modes. This represents the current fusion weight value for the radar modes. This represents the reliability value of the spectral modes. This represents the radar modal reliability value. A positive number pre-set according to actual needs. The weights are pre-set based on actual needs. The updated reliability, weights, historical decisions, and validation results are written into the fusion memory. This is for use in the next time window.

[0076] The fusion agent is input with the spectrum candidate target set, radar candidate target set, cross-modal candidate pair set, fusion strategy, and current fusion weight value. The agent first categorizes candidates into three types based on their modal support type: spectrum-only support, radar-only support, and dual-modal support. Then, considering the candidate's own confidence, current spectrum / radar reliability, fusion strategy, and current scene state, it performs retention, weight reduction, or elimination of candidates. This ensures that for dual-modal support candidates, the fusion agent utilizes both spectrum and radar evidence for target confirmation; for single-modal candidates, it decides whether to retain them based on the current reliability of the corresponding modality and the fusion strategy selected in step S104. The final fusion process can be represented as: ; ; in, This represents the process of target confirmation and evidence trajectory generation performed by the fusion intelligent agent; This represents the modal weight vector (current fusion weight value) composed of spectral weights and radar weights. This indicates the fused output of the current time window; This indicates that the set of drone targets has been confirmed; This represents the corresponding set of evidence trajectories. Each evidence trajectory records the spectrum candidates, radar candidates, cross-modal correlations, underlying sensor observations, modal support types, and current fusion strategies referenced by the confirmed target.

[0077] After completing all the above steps, this application calls the reference UAV identity and geographic location provided by the protocol decoding device during the offline evaluation stage. A cost matrix is ​​constructed between the predicted target and the reference target at the same timestamp based on azimuth error, and a one-to-one matching is used to obtain the correspondence; when the circumferential azimuth error after matching does not exceed 10 degrees, it is considered a valid match. This 10-degree threshold and the offline one-to-one matching are only used for performance evaluation and are not part of the cross-modal candidate association rules within step S103.

[0078] True cases are counted based on valid matching relationships. False positives and false negatives And calculate precision, recall and F1 score: ; ; ; in, This indicates the number of predicted targets that form a valid match with the reference target. This indicates the number of predicted targets that did not form a valid match with the reference target. This represents the number of reference targets that were not effectively matched by any prediction result. Indicates the accuracy of the test results. Indicates recall rate, It represents the harmonic mean of precision and recall.

[0079] To verify the effectiveness of the multimodal UAV detection results of this application, a comparison was made with several representative target screening and detection methods in the prior art. Referring to Table 1, Table 1 shows the performance comparison results of UAV detection between this application and different baseline methods. In Table 1, Precision represents the precision of the UAV detection results; Recall represents the recall rate; F1-score represents the harmonic mean of precision and recall; Rule-Based in Table 1 is a candidate target screening method based on fixed rules in the prior art, which achieves target candidate screening and confirmation by setting preset thresholds for candidate confidence, motion features, and cross-modal consistency; Table 1... XGBoost is a machine learning method based on gradient boosting decision trees in the prior art. It achieves the identification and screening of UAV targets by jointly modeling the features of different types of candidate targets. MLP in Table 1 is a neural network method based on multilayer perceptron in the prior art. It learns the nonlinear relationship between the features of candidate targets by using a two-layer feedforward neural network and completes the target determination based on the network output. Single-Agent in Table 1 is a single agent multimodal reasoning method in the prior art. It inputs structured radio spectrum observation data and radar observation data into a single agent at the same time. The agent performs joint analysis and directly outputs the UAV target prediction result.

[0080] Table 1

[0081] As shown in Table 1, this application achieves the highest F1-score in both single-target and multi-target scenarios. In the single-target scenario, the Precision, Recall, and F1-score of this application are 91.62%, 92.09%, and 91.86%, respectively, which is 4.51 percentage points higher than the F1-score of Single-Agent. In the multi-target scenario, the Precision, Recall, and F1-score of this application are 70.57%, 81.79%, and 75.76%, respectively, which is 15.09 percentage points higher than XGBoost, which has the highest F1-score among the horizontal baselines. The results indicate that in multi-target scenarios with increased candidate density and association ambiguity, this application, through modal-independent candidate generation, cross-modal association, and adaptive reliability fusion, can better balance target recall and false alarm suppression, thereby improving the robustness and accuracy of UAV target detection.

[0082] Referring to Table 2, which compares the UAV detection performance of the agents (radar agent, spectrum agent, and fusion agent) using different large language model backbones, it can be seen that target candidate analysis and fusion reasoning can be completed under different large language model backbones. Furthermore, the overall performance improves further as the model's structured reasoning capability is enhanced. Specifically, Qwen-7B achieves the best results in both single-target and multi-target scenarios; Qwen-3B and DeepSeek-R1-7B also maintain high F1-scores. This indicates that the modal agent division of labor, evidence association, and dynamic reliability fusion process of this application do not depend on a single large language model and can maintain high target detection capabilities under different reasoning backbones.

[0083] Table 2

[0084] Additionally, refer to Figure 3 One embodiment of this application provides a multimodal unmanned aerial vehicle (UAV) detection system based on collaborative multi-agent reasoning, including a data acquisition module 100, an agent output module 200, a cross-modal candidate pair set determination module 300, a fusion strategy determination module 400, a current fusion weight value determination module 500, and a UAV detection result output module 600, wherein: The data acquisition module 100 is used to acquire radio spectrum observation data within the current preset time period, radar observation data within the current preset time period, and historical fusion weight values ​​within the previous preset time period; The intelligent agent output module 200 is used to input radio spectrum observation data into the spectrum intelligent agent to obtain the spectrum candidate target set output by the spectrum intelligent agent and the spectrum candidate confidence level corresponding to each spectrum candidate target; and to input radar observation data into the radar intelligent agent to obtain the radar candidate target set output by the radar intelligent agent and the radar candidate confidence level corresponding to each radar candidate target. The cross-modal candidate pair set determination module 300 is used to determine the cross-modal candidate pair set based on the consistency analysis results obtained after performing consistency analysis on any spectrum candidate target and any radar candidate target, wherein each cross-modal candidate pair in the cross-modal candidate pair set includes one spectrum candidate target and one radar candidate target; The fusion strategy determination module 400 is used to determine the fusion strategy based on the spectrum candidate target set, the radar candidate target set, the cross-modal candidate pair set, the spectrum candidate confidence score, and the radar candidate confidence score. Specifically: Based on a pre-set confidence threshold, a set of spectrum candidate targets, a set of radar candidate targets, a set of cross-modal candidate pairs, a confidence level of spectrum candidates, and a confidence level of radar candidates, a false alarm risk value is determined; based on a pre-set confidence threshold, a confidence level of spectrum candidates, and a confidence level of radar candidates, a false alarm risk value is determined. The risk direction value is obtained by subtracting the false alarm risk value from the missed detection risk value. If the risk direction value is greater than the first preset risk threshold, the recall protection strategy will be used as the fusion strategy. If the risk direction value is greater than or equal to the second preset risk threshold and less than or equal to the first preset risk threshold, the balancing strategy will be used as the fusion strategy. If the risk direction value is less than the second preset risk threshold, the accuracy protection strategy will be used as the fusion strategy. The current fusion weight value determination module 500 is used to determine the current fusion weight value based on historical fusion weight values ​​and preset weight adjustment amounts, specifically as follows: Based on the spectrum candidate target set, the radar candidate target set, and the cross-modal candidate pair set, the spectrum mode reliability value and the radar mode reliability value are determined; The spectrum fusion weight range is determined based on historical fusion weight values ​​and the maximum preset weight change. The current fusion weight value is determined based on the spectrum mode reliability value, radar mode reliability value, and spectrum fusion weight range. The UAV detection result output module 600 is used to input the spectrum candidate target set, radar candidate target set, cross-modal candidate pair set, fusion strategy and current fusion weight value into the fusion agent to obtain the UAV detection result output by the fusion agent.

[0085] This system acquires radio spectrum observation data and radar observation data for the current period, as well as historical fusion weight values ​​from the previous period. It then outputs corresponding candidate target sets and their confidence levels through a spectrum agent and a radar agent, respectively. After conducting consistency analysis on the candidate targets, a cross-modal candidate pair set is obtained, which in turn determines the appropriate fusion strategy. The system updates the current fusion weight value by combining historical fusion weights with preset weight adjustments, and then sends this value back to the fusion agent to output the UAV detection results. This system achieves complementary advantages between spectrum and radar cross-modal information, dynamically adjusts the fusion weights based on data from previous and subsequent periods, adapts to complex observation scenarios, effectively reduces the missed and false detection problems caused by single-modal detection, and improves the robustness and accuracy of UAV target detection.

[0086] It should be noted that the system embodiments described above are based on the same inventive concept as the method embodiments described above. Therefore, the relevant content of the method embodiments described above is also applicable to the system embodiments described above, and will not be repeated here.

[0087] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with relevant regulations. The acquisition, storage, use and processing of data in the technical solution of this application all comply with the relevant provisions of national laws and regulations.

[0088] like Figure 4 One embodiment of this application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the aforementioned multimodal UAV detection method based on cooperative multi-agent reasoning. The electronic device includes: At least one memory; At least one processor; At least one program; The program is stored in memory, and the processor executes at least one program to implement a multimodal unmanned aerial vehicle (UAV) detection method based on cooperative multi-agent reasoning according to the above embodiments of this application.

[0089] Electronic devices can be any smart terminal, including mobile phones, tablets, personal digital assistants (PDAs), and in-vehicle computers.

[0090] The electronic devices according to embodiments of this application will now be described in detail.

[0091] The processor 1600 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 1700 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 1700 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 1700 and is called and executed by the processor 1600 to execute the multimodal UAV detection method based on cooperative multi-agent reasoning of the embodiments of this application.

[0092] The input / output interface 1800 is used to implement information input and output. The communication interface 1900 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 2000 transmits information between various components of the device (e.g., processor 1600, memory 1700, input / output interface 1800, and communication interface 1900); The processor 1600, memory 1700, input / output interface 1800 and communication interface 1900 are connected to each other within the device via bus 2000.

[0093] This application also provides a storage medium, which is a computer-readable storage medium storing computer-executable instructions for causing a computer to execute the above-described multimodal UAV detection method based on cooperative multi-agent reasoning.

[0094] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0095] The above is a detailed description of the preferred embodiments of this application. However, the embodiments of this application are not limited to the above-described implementation methods. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the embodiments of this application. All such equivalent modifications or substitutions are included within the scope defined by the claims of the embodiments of this application.

Claims

1. A multimodal unmanned aerial vehicle (UAV) detection method based on cooperative multi-agent reasoning, characterized in that, The multimodal UAV detection method based on cooperative multi-agent reasoning includes: Acquire radio spectrum observation data within the current preset time period, radar observation data within the current preset time period, and historical fusion weight values ​​within the previous preset time period; The radio spectrum observation data is input into a spectrum agent to obtain a spectrum candidate target set output by the spectrum agent and a spectrum candidate confidence level corresponding to each spectrum candidate target; the radar observation data is input into a radar agent to obtain a radar candidate target set output by the radar agent and a radar candidate confidence level corresponding to each radar candidate target. If a consistency analysis is performed on any of the aforementioned spectrum candidate targets and any of the aforementioned radar candidate targets, and a consistency analysis result is obtained, a cross-modal candidate pair set is determined based on the consistency analysis result, wherein each cross-modal candidate pair in the cross-modal candidate pair set includes one of the aforementioned spectrum candidate targets and one of the aforementioned radar candidate targets; Based on the spectrum candidate target set, the radar candidate target set, the cross-modal candidate pair set, the spectrum candidate confidence score, and the radar candidate confidence score, a fusion strategy is determined, specifically as follows: Based on a preset confidence threshold, the spectrum candidate target set, the radar candidate target set, the cross-modal candidate pair set, the spectrum candidate confidence level, and the radar candidate confidence level, a missed detection risk value is determined; based on the preset confidence threshold, the spectrum candidate confidence level, and the radar candidate confidence level, a false alarm risk value is determined. The risk direction value is obtained by subtracting the false alarm risk value from the missed detection risk value. If the risk direction value is greater than the first preset risk threshold, the recall protection strategy will be used as the fusion strategy. If the risk direction value is greater than or equal to the second preset risk threshold and less than or equal to the first preset risk threshold, the balancing strategy will be used as the fusion strategy. If the risk direction value is less than the second preset risk threshold, the accuracy protection strategy will be used as the fusion strategy. Based on the historical fusion weight values ​​and the preset weight adjustment amount, the current fusion weight value is determined as follows: Based on the spectrum candidate target set, the radar candidate target set, and the cross-modal candidate pair set, the spectrum mode reliability value and the radar mode reliability value are determined; Based on the historical fusion weight values ​​and the preset maximum weight change, the spectrum fusion weight range is determined; Based on the spectrum mode reliability value, the radar mode reliability value, and the spectrum fusion weight range, the current fusion weight value is determined; The spectrum candidate target set, the radar candidate target set, the cross-modal candidate pair set, the fusion strategy, and the current fusion weight value are input into the fusion agent to obtain the UAV detection result output by the fusion agent.

2. The multimodal UAV detection method based on cooperative multi-agent reasoning according to claim 1, characterized in that, The acquisition of radio spectrum observation data within the current preset time period, radar observation data within the current preset time period, and historical fusion weight values ​​within the previous preset time period includes: Acquire initial radio spectrum observation data and initial radar observation data within the current preset time period; The initial radio spectrum observation data and the initial radar observation data are preprocessed to obtain the radio spectrum observation data and the radar observation data; Extract the pre-stored fusion weight values ​​from the previous preset time period and use them as the historical fusion weight values.

3. The multimodal UAV detection method based on cooperative multi-agent reasoning according to claim 1, characterized in that, The determination of the cross-modal candidate pair set based on the consistency analysis results includes: All the consistency analysis results that are the first preset analysis results are selected as the set of consistency analysis results to be selected. In the set of consistency analysis results to be screened, the spectrum candidate target and the radar candidate target corresponding to each consistency analysis result to be screened are taken as the corresponding cross-modal candidate pair; All the cross-modal candidate pairs are combined to obtain the cross-modal candidate pair set.

4. The multimodal UAV detection method based on cooperative multi-agent reasoning according to claim 1, characterized in that, The missed detection risk value is determined based on a preset confidence threshold, the spectrum candidate target set, the radar candidate target set, the cross-modal candidate pair set, the spectrum candidate confidence, and the radar candidate confidence. Based on the preset confidence threshold, the candidate spectrum confidence, and the candidate radar confidence, a false alarm risk value is determined, including: The total number of spectrum candidate targets in the spectrum candidate target set is counted and used as the first target number; The total number of radar candidate targets in the radar candidate target set is counted and used as the second target number; The total number of cross-modal candidate pairs in the cross-modal candidate pair set is taken as the third target number; Based on the number of the first target, the number of the second target, and the number of the third target, determine the proportion of single-modal candidates; Based on the preset confidence threshold, the spectrum candidate confidence, and the radar candidate confidence, the proportion of medium confidence candidates, the proportion of low confidence candidates, and the proportion of invalid confidence candidates are determined. The missed detection risk value is determined based on the single-modality candidate proportion and the medium-confidence candidate proportion. The false alarm risk value is determined based on the proportion of low-confidence candidates and the proportion of invalid-confidence candidates.

5. The multimodal UAV detection method based on cooperative multi-agent reasoning according to claim 4, characterized in that, The step of determining the spectral modal reliability value and the radar modal reliability value based on the spectral candidate target set, the radar candidate target set, and the cross-modal candidate pair set includes: Based on the first corresponding ID of the spectrum candidate target output by the spectrum agent and the second corresponding ID of the radar candidate target output by the radar agent, the candidate types corresponding to the spectrum candidate target and the radar candidate target within the preset iteration history time period are extracted. The spectrum candidate targets and radar candidate targets in the cross-modal candidate pair set are marked as strong positive feedback; the spectrum candidate targets and radar candidate targets that appear K consecutively in the preset iteration history time period and whose candidate type is either spectrum single-modal candidate type or radar single-modal candidate type are marked as weak positive feedback; the spectrum candidate targets and radar candidate targets that appear only once in the preset iteration history time period are marked as negative feedback. The preset iteration history time period includes the current preset time period and the most recent multiple historical preset time periods, and K is a value preset according to actual needs. For each spectrum candidate target and radar candidate target, a feedback label and feedback intensity are historically set. The feedback label for spectrum candidate targets or radar candidate targets marked as weak positive feedback or strong positive feedback is set to one; the feedback label for spectrum candidate targets or radar candidate targets marked as negative feedback is set to negative one; the feedback intensity for spectrum candidate targets or radar candidate targets marked as strong positive feedback is set to one; the feedback intensity for spectrum candidate targets or radar candidate targets marked as weak positive feedback is set to a first preset feedback intensity based on actual needs; and the feedback intensity for spectrum candidate targets or radar candidate targets marked as negative feedback is set to a second preset feedback intensity based on actual needs. The spectral modal reliability value and the radar modal reliability value are calculated using the following formulas: ； ； ； in, For the first The modal type within the preset time period is The positive feedback evidence value, For the first The modal type within the preset time period is The negative feedback evidence value, For the pre-acquired first The modal type within the preset time period is The positive feedback evidence value, For the pre-acquired first The modal type within the preset time period is The negative feedback evidence value, Indicates the candidate type corresponding to the candidate target. For the first The set of candidate targets within a preset time period. Indicates the first The candidate target pairs have the following modal types: Preset feedback allocation coefficient; Indicates the first The feedback strength of each candidate target, Indicates an indicator function, This represents the coefficient value of the time decay factor, which is preset according to actual needs. This indicates the time interval between the generation of feedback and the current update. For the first The modal type within the preset time period is Reliability value, The first prior smoothing parameter is set in advance according to actual needs. This is a second prior smoothing parameter pre-set according to actual needs.

6. The multimodal UAV detection method based on cooperative multi-agent reasoning according to claim 5, characterized in that, The step of determining the spectrum fusion weight range based on the historical fusion weight values ​​and the preset maximum weight change includes: Based on the historical fusion weight values ​​and the maximum change in the preset weights, the spectrum fusion weight range is calculated using the following formula: ； ； in, For the first The minimum value of the spectrum fusion weight range within the preset time period. For the first The maximum value of the spectrum fusion weight range within the preset time period. The minimum weight is pre-set according to actual needs. The maximum weight is preset according to actual needs. For historical fusion weight values, This represents the maximum change in the preset weight.

7. The multimodal UAV detection method based on cooperative multi-agent reasoning according to claim 6, characterized in that, The step of determining the current fusion weight value based on the spectrum mode reliability value, the radar mode reliability value, and the spectrum fusion weight range includes: Based on the spectrum modal reliability value, radar modal reliability value, and spectrum fusion weight range, the current fusion weight value is calculated using the following formula: ； ； in, This represents the current fusion weight value for the spectral modes. This represents the current fusion weight value for the radar modes. This represents the reliability value of the spectral modes. This represents the radar modal reliability value. A positive number pre-set according to actual needs. This refers to the pre-set weight adjustment amount based on actual needs.

8. A multimodal unmanned aerial vehicle (UAV) detection system based on cooperative multi-agent reasoning, characterized in that, The multimodal UAV detection system based on collaborative multi-agent reasoning includes: The data acquisition module is used to acquire radio spectrum observation data, radar observation data and historical fusion weight values ​​within the current preset time period. The intelligent agent output module is used to input the radio spectrum observation data into the spectrum intelligent agent to obtain the spectrum candidate target set output by the spectrum intelligent agent and the spectrum candidate confidence level corresponding to each spectrum candidate target; and to input the radar observation data into the radar intelligent agent to obtain the radar candidate target set output by the radar intelligent agent and the radar candidate confidence level corresponding to each radar candidate target. A cross-modal candidate pair set determination module is used to determine a cross-modal candidate pair set based on the consistency analysis results obtained when performing consistency analysis on any of the spectrum candidate targets and any of the radar candidate targets, wherein each cross-modal candidate pair in the cross-modal candidate pair set includes one of the spectrum candidate targets and one of the radar candidate targets; The fusion strategy determination module is used to determine a fusion strategy based on the spectrum candidate target set, the radar candidate target set, the cross-modal candidate pair set, the spectrum candidate confidence score, and the radar candidate confidence score, specifically: Based on a preset confidence threshold, the spectrum candidate target set, the radar candidate target set, the cross-modal candidate pair set, the spectrum candidate confidence level, and the radar candidate confidence level, a missed detection risk value is determined; based on the preset confidence threshold, the spectrum candidate confidence level, and the radar candidate confidence level, a false alarm risk value is determined. The risk direction value is obtained by subtracting the false alarm risk value from the missed detection risk value. If the risk direction value is greater than the first preset risk threshold, the recall protection strategy will be used as the fusion strategy. If the risk direction value is greater than or equal to the second preset risk threshold and less than or equal to the first preset risk threshold, the balancing strategy will be used as the fusion strategy. If the risk direction value is less than the second preset risk threshold, the accuracy protection strategy will be used as the fusion strategy. The current fusion weight value determination module is used to determine the current fusion weight value based on the historical fusion weight values ​​and the preset weight adjustment amount, specifically as follows: Based on the spectrum candidate target set, the radar candidate target set, and the cross-modal candidate pair set, the spectrum mode reliability value and the radar mode reliability value are determined; Based on the historical fusion weight values ​​and the preset maximum weight change, the spectrum fusion weight range is determined; Based on the spectrum mode reliability value, the radar mode reliability value, and the spectrum fusion weight range, the current fusion weight value is determined; The UAV detection result output module is used to input the spectrum candidate target set, the radar candidate target set, the cross-modal candidate pair set, the fusion strategy, and the current fusion weight value into the fusion agent to obtain the UAV detection result output by the fusion agent.

9. An electronic device, characterized in that, It includes at least one processor and a memory for communicatively connecting with the processor; the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to cause the at least one processor to perform a multimodal unmanned aerial vehicle detection method based on cooperative multi-agent reasoning as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions for causing a computer to perform a multimodal unmanned aerial vehicle (UAV) detection method based on cooperative multi-agent reasoning as described in any one of claims 1 to 7.

Images