An acoustic-optical-electromagnetic collaborative low-altitude target detection and identification method based on acoustic leader constraint

Abstract

The present application relates to the technical field of multi-modal collaborative detection, and particularly relates to a method for acoustic-optical-electromagnetic collaborative low-altitude target detection and confirmation based on acoustic precursor constraint, comprising: continuously monitoring a target area to identify acoustic events corresponding to suspected low-altitude flying targets; determining acoustic direction information according to the acoustic events, generating a directional search sector and an associated time window as restricted search conditions; controlling an optical device to collect images within the sector, and performing target detection, entity confirmation and trajectory extraction; scanning, identifying and direction-finding electromagnetic signals within the sector to obtain electromagnetic characteristics; performing spatio-temporal alignment and cross-modal correlation of acoustic-optical-electromagnetic information, and outputting a confirmation result based on multi-dimensional consistency; continuously tracking the target, and feeding back the confirmation result to the acoustic identification and correlation process to form a closed-loop collaborative detection mechanism. The present application reduces false alarm rate and improves the reliability of low-altitude target detection through acoustic precursor triggering and multi-modal cross-confirmation.

Description

Technical Field

[0001] This invention relates to the field of multi-mode cooperative detection technology, and in particular to a method for detecting and confirming low-altitude targets using a combination of acoustic, optical, and electromagnetic signals based on acoustic leader constraints. Background Technology

[0002] With the rapid development of the low-altitude economy and the increasing popularity of UAV technology, low-altitude target detection technology is showing a significant trend towards multi-sensor fusion. Traditional technologies typically employ continuous scanning with a single sensor or simple multi-sensor parallel detection schemes, mainly including techniques such as all-time acoustic monitoring, all-time radar scanning, all-time optoelectronic monitoring, and independent combination of multiple sensors.

[0003] All-time acoustic monitoring uses microphone arrays to continuously collect and analyze acoustic signals. While it has the advantages of omnidirectional perception and low cost, it is limited by factors such as multipath propagation in complex environments, wind noise interference, and long-distance attenuation, making it difficult to obtain precise target location and category information. All-time radar scanning can obtain target distance, speed, and angle information, but the equipment is expensive and consumes a lot of power, and its ability to detect low, slow, and small targets is limited. All-time photoelectric monitoring uses visible light or infrared imaging for image acquisition and search, which can provide intuitive visual confirmation information, but photoelectric sensors have a limited field of view, low wide-area search efficiency, and insufficient target acquisition probability.

[0004] Simple multi-sensor fusion schemes deploy heterogeneous sensors such as acoustic, photoelectric, and radar sensors in parallel, allowing them to operate independently before target association and confirmation via backend fusion algorithms. However, due to the lack of an effective collaborative scheduling mechanism among the sensors, the system needs to maintain high-load operation of all sensors in all times and all spaces, resulting in wasted computing resources and data redundancy. More critically, the independent triggering and asynchronous detection mode of each sensor makes it difficult to align multimodal data in the spatiotemporal dimension, hindering the establishment of reliable cross-modal correlations and easily leading to false alarms and identity confusion. Furthermore, the existing fusion system architecture is fixed and cannot be dynamically adjusted according to the environment. It is difficult to use the confirmation results of high-precision modalities to optimize triggering conditions, making it difficult to continuously reduce the false trigger rate, and it has shortcomings in terms of environmental adaptability and intelligence. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing technologies, such as high wide-area search load, difficulty in cross-modal spatiotemporal correlation, and difficulty in continuously reducing false triggering rates in existing low-altitude target detection. This invention provides a method for low-altitude target detection and confirmation based on acoustic leader constraints and a combination of acoustic, optical, and electromagnetic methods, comprising the following steps: S1. Perform continuous acoustic monitoring of the target area, and preprocess and extract features from the collected acoustic signals to identify acoustic events corresponding to suspected low-altitude flying targets. S2. Determine the acoustic direction and time information of the suspected low-altitude flying target based on the acoustic event, and generate a directional search sector and associated time window based on the acoustic direction and time information as a restricted search condition for subsequent optical and electromagnetic detection. S3. Control the optical detection device to collect target image data within the directional search sector and the associated time window, and perform target detection, entity confirmation and trajectory extraction on the target image data to obtain target visual information; S4. Under the constraints of the directional search sector and the associated time window, the electromagnetic signals in the corresponding direction and / or within the preset frequency band range of the directional search sector are scanned, identified, and the direction of arrival is estimated to obtain the target electromagnetic characteristic information. S5. Perform spatiotemporal alignment, spatial mapping, and cross-modal association on the acoustic direction information, the target visual information, and the target electromagnetic feature information, and output the target confirmation result based on direction consistency, temporal consistency, and feature consistency; S6. Based on the target confirmation result, continuously track the target and feed back the optical confirmation result and / or electromagnetic confirmation result to the acoustic event recognition process and / or cross-modal association process to correct the subsequent target search range, association conditions or triggering conditions, forming a closed-loop cooperative detection mechanism based on acoustic leader constraints.

[0006] Preferably, step S1 includes: A microphone array is used to continuously collect multi-channel acoustic data from the target area. The acquired raw acoustic signals are subjected to bandpass filtering, background noise reduction, and frame segmentation to suppress environmental noise interference. Time-frequency analysis is performed on the processed acoustic signal to extract acoustic features including short-time energy characteristics, spectral peak characteristics, and harmonic structure characteristics; Based on the acoustic characteristics, it is determined whether the acoustic event should be generated, and the acoustic event is used as a precursor event to trigger subsequent optical and electromagnetic detection.

[0007] Preferably, step S2 includes: The inter-channel time delay difference is calculated for the multi-channel acoustic signals corresponding to the acoustic event, and the arrival direction of the suspected low-altitude flying target is estimated by combining the geometric relationship of the microphone array. The inter-channel time delay difference is determined by the peak position of the generalized cross-correlation function, which is: , in, Let be the cross-power spectral density of the signals received by the i-th array element and the reference array element. It is a frequency domain weighting function; Based on the direction of arrival, the spatial azimuth and pitch angles are calculated, and the horizontal angle and pitch angle width are expanded with the direction of arrival as the central direction to generate the directional search sector; Based on the start time, duration, target propagation characteristics, and response hysteresis of each sensor, an associated time window is established.

[0008] Preferably, step S3 includes: The gimbal or photoelectric turntable is adjusted according to the directional search sector so that the visible light imaging device and the infrared imaging device can preferentially cover the directional search sector. Continuous image frames are acquired within the associated time window, and candidate target detection is performed within a limited field of view; For candidate target areas, physical confirmation is performed by combining optical features, including shape contour, scale changes, and motion continuity, in order to distinguish between real low-altitude flying targets and interference targets. Once the target is confirmed, it is associated with consecutive frames to extract the target's motion trajectory and output the target's visual category information, target position change information, and target trajectory information.

[0009] Preferably, entity confirmation and consecutive frame association include: A temporal visual observation sequence is established based on the displacement changes, area changes, contour stability, and orientation changes of candidate targets in adjacent image frames; Cross-validate edge contour information in visible light images with thermal distribution information in infrared images; Cross-frame matching of the same target is performed based on the target's velocity and orientation constraints in consecutive image frames to maintain the continuity and consistency of the target's identity; After obtaining a stable trajectory, the target category label, trajectory curve, and visual confirmation confidence level are output.

[0010] Preferably, step S4 includes: Under the constraints of the directional search sector and the associated time window, electromagnetic signals within a preset frequency band are directionally scanned and acquired to obtain radio frequency observation data; The radio frequency observation data is subjected to energy detection, spectral structure analysis, and modulation feature analysis to identify suspected target electromagnetic signals; Electromagnetic features, including carrier frequency, bandwidth, and frequency hopping characteristics, are extracted from the electromagnetic signals of identified suspected targets. By combining a multi-channel receiving structure, the direction of arrival of the suspected target electromagnetic signal is estimated to obtain the target's electromagnetic characteristics and corresponding direction information.

[0011] Preferably, the target electromagnetic characteristic information, used as supplementary communication verification information for target confirmation, includes: The radio frequency observation data is segmented in the time domain and decomposed in the frequency domain to distinguish between continuous transmission signals, intermittent transmission signals and frequency hopping signals; The spectrum occupancy range, power variation law, duration characteristics and periodic variation characteristics of different types of signals are calculated respectively, and matched with the preset target communication template; When the matching result meets the preset conditions, the direction of incoming wave is calculated based on the phase difference, amplitude difference and / or spatial spectrum peak value of the multi-channel received signal; The obtained direction of arrival is verified to correspond with the directional search sector and the direction of optical trajectory. When the verification meets the consistency condition, the corresponding electromagnetic signal is determined as the communication supplementary verification information for target confirmation.

[0012] Preferably, step S5 includes: Acoustic, optical, and electromagnetic detection results are each assigned a time signature under a unified clock reference to establish cross-modal time synchronization relationships; Based on the installation position, attitude parameters, field of view parameters, array geometric parameters and coordinate transformation relationships of each sensor, a unified coordinate mapping is performed on the acoustic orientation, optical target position and electromagnetic wave direction; The mapped multimodal observation results are projected onto a unified reference coordinate system, and a set of candidate targets is established according to adjacent time windows and adjacent spatial ranges; The acoustic, visual, and communication attributes in the candidate target set are pre-associated and organized to form target candidate association data for joint determination.

[0013] Preferably, in step S5, cross-modal association and target confirmation include: The acoustic orientation, optical trajectory direction and electromagnetic wave direction in the target candidate association data are compared for consistency to obtain the direction association results. Match the correspondence between the occurrence times of acoustic events, visual targets, and electromagnetic signals to obtain temporal correlation results; The feature association results are calculated based on the correspondence between the target's visual category features, acoustic features, and electromagnetic features, and the behavior association results are obtained by combining the target trajectory continuity. A weighted scoring fusion method is used to jointly determine the directional association results, temporal association results, feature association results, and behavioral association results, wherein the comprehensive confidence score is used. Calculate using the following formula: , in, Scoring is given for consistency of direction. For time-series matching scores, For feature matching scores, For trajectory continuity score, , , , For the preset weighting coefficients, satisfy ;when When, output the target confirmation result; when When, output the result to be confirmed and continue observation; when At that time, output the false alarm suppression result.

[0014] Preferably, step S6 includes: Establish a target state model for the confirmed target, including target location, target velocity, target direction, target category, and target confidence level; The target state model is recursively updated using acoustic, optical, and electromagnetic observations received at subsequent time points to achieve continuous fusion tracking of the target. The recursive update employs a Kalman filter framework, and the state prediction equation and state update equation are as follows: , in, Let k be the predicted state value. Here is the state transition matrix. This is the state estimate at time k-1. To control the input matrix, For control vectors; This is the state update value at time k. Here is the Kalman gain matrix. Let k be the multimodal observation vector at time k. The observation matrix; When optical observation results obtain higher precision position or trajectory information, the position or trajectory information is fed back to the electromagnetic scanning process and / or acoustic event recognition process; When the electromagnetic observation results obtain stable communication characteristics or stable incoming wave direction, the stable communication characteristics or stable incoming wave direction are fed back to the optical target re-acquisition process and / or cross-modal correlation process; When the joint determination result indicates that the previous acoustic event was falsely triggered, the false trigger result is fed back to the acoustic event identification process to update the trigger threshold, optimize the acoustic discrimination conditions, or correct the associated time window.

[0015] Compared with the prior art, the beneficial effects of the present invention are: This invention uses a microphone array to continuously monitor the target area acoustically. Based on the collected acoustic signals, it identifies acoustic events corresponding to suspected low-altitude flying targets and determines acoustic direction and time information based on the acoustic events to generate directional search sectors and associated time windows. This transforms wide-area indiscriminate search into a restricted search under acoustic guidance constraints, enabling optical and electromagnetic detection devices to operate only within the spatiotemporal range triggered by acoustic events. This significantly reduces the system's computational load and power consumption, and improves real-time response. This invention performs time alignment, spatial mapping, and cross-modal correlation on acoustic information obtained from acoustic events, target visual information obtained from optical detection devices, and target electromagnetic feature information obtained from electromagnetic detection devices. Based on directional consistency, temporal consistency, and feature consistency, it outputs target confirmation results, realizing redundancy complementarity and cross-verification of three heterogeneous sensing modalities (acoustic, optical, and electromagnetic) under a unified reference coordinate system. This effectively suppresses false alarms and missed detections of a single modality, and improves the accuracy and reliability of low-altitude target confirmation. This invention achieves a closed-loop collaborative detection mechanism that optimizes the lead triggering conditions based on the confirmation results by continuously fusing and tracking the target according to the target confirmation results, and feeding back the optical confirmation results and / or electromagnetic confirmation results to the acoustic event recognition process and / or cross-modal association process. This enables parameters such as acoustic triggering threshold, search range, and association time window to be dynamically corrected according to the confirmation results of subsequent high-precision modes, thereby continuously reducing the false triggering rate and improving the system's adaptability to complex low-altitude environments. Attached Figure Description

[0016] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention.

[0017] Figure 1 This is a flowchart of a method for detecting and confirming low-altitude targets based on acoustic leader constraints using a combination of acoustic, optical, and electromagnetic methods, according to the present invention. Figure 2 This is another flowchart of the method for detecting and confirming low-altitude targets based on acoustic guidance constraints and coordinating acoustic-optical-electromagnetic signals according to the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. Obviously, the described embodiments are only some, not all, of the embodiments described in this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without creative effort are within the scope of protection of this application.

[0019] Those skilled in the art will understand that, unless otherwise stated, the singular forms “a” and “an” used herein, and “the”, may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0020] First Embodiment Please see Figure 1 and Figure 2 As shown in the figure, this embodiment provides a method for low-altitude target detection and confirmation based on acoustic precursor constraints and coordinated acoustic-optical-electromagnetic methods, applied to urban low-altitude security scenarios. This method uses acoustic monitoring results as precursor triggering information, applies time and spatial domain constraints to optical and electromagnetic detection, and achieves low-altitude target confirmation and continuous tracking through multimodal correlation. The method includes the following steps: S1. Perform continuous acoustic monitoring of the target area, and preprocess and extract features from the collected acoustic signals to identify acoustic events corresponding to suspected low-altitude flying targets.

[0021] Preferably, step S1 includes: A microphone array is used to continuously acquire acoustic signals from the target area in multiple channels. Specifically, in this embodiment, a ring microphone array consisting of 8 MEMS microphones is deployed around the target area. The array radius is 0.3m, and continuous multi-channel acoustic signal acquisition is performed at a sampling rate of 48kHz.

[0022] The acquired raw acoustic signal is subjected to bandpass filtering, background noise reduction, and framing to suppress environmental noise interference. Specifically, in this embodiment, a fourth-order Butterworth bandpass filter is used to filter the raw signal to retain the main energy frequency band of the rotor noise of low-altitude flying targets and to filter out low-frequency wind noise and high-frequency random noise. Subsequently, background noise reduction is performed using spectral subtraction based on short-time Fourier transform to suppress wind noise, ground traffic noise, and environmental random noise. After that, the denoised signal is framed and windowed, and endpoint detection is performed using a dual-threshold method based on short-time energy and zero-crossing rate to determine whether there is a valid sound source.

[0023] Among them, the short-time energy of the nth frame signal It can be represented as: ,in, Let w(m) be the amplitude of the m-th sampling point within the n-th frame, w(m) be the window function, and N be the frame length. Zero crossing rate of the nth frame signal It can be represented as: ,in, For symbolic functions, by and By comparing the current frame with the preset energy threshold and the zero-crossing rate threshold, it can be preliminarily determined whether the current frame contains a valid sound source.

[0024] The processed acoustic signal is subjected to time-frequency analysis to extract acoustic features including short-time energy features, spectral peak features, and harmonic structure features. Specifically, in this embodiment, a rapid initial screening is triggered based on the short-time energy threshold and the number of spectral peaks. When the short-time energy exceeds the preset energy threshold and the number of spectral peaks in the 1 kHz to 8 kHz frequency band exceeds the preset number threshold, a suspected valid sound source is determined to exist, and the second-level identification process is triggered.

[0025] In the second-level recognition process, the corresponding spectrogram is input into a pre-trained convolutional recurrent hybrid network classification model. This model consists of three layers of convolutional neural networks and two layers of long short-term memory networks connected in series. The convolutional neural network is used to extract local spectral texture features, and the long short-term memory network is used to capture the periodic variation features caused by the drone's rotor speed. The model outputs the probabilities of four categories—drone, bird, vehicle noise, and environmental noise—after passing through fully connected layers and a softmax function.

[0026] The softmax output can be represented as: ,in, The output score is for the i-th class. This represents the probability of the corresponding category.

[0027] Based on acoustic characteristics, it is determined whether an acoustic event is generated, and the acoustic event is used as a precursor event to trigger subsequent optical and electromagnetic detection. Specifically, in this embodiment, when the output confidence of the UAV category exceeds 0.75, an acoustic event trigger signal is generated to determine the existence of a suspected low-altitude UAV target.

[0028] S2. Determine the acoustic direction and time information of suspected low-altitude flying targets based on acoustic events, and generate directional search sectors and associated time windows based on the acoustic direction and time information as limited search conditions for subsequent optical and electromagnetic detection.

[0029] Preferably, step S2 includes: The inter-channel time delay difference is calculated for the multi-channel acoustic signals corresponding to the acoustic event, and the arrival direction of the suspected low-altitude flying target is estimated by combining the geometric relationship of the microphone array. The inter-channel time delay difference is determined by the peak position of the generalized cross-correlation function, which is: ,in, Let be the cross-power spectral density of the signals received by the i-th array element and the reference array element. Let be the frequency domain weighting function, and τ be the time delay variable. This is achieved by calculating... By determining the peak position, the estimated delay difference of the corresponding channel pair can be obtained.

[0030] Solve the spatial azimuth angle based on the direction of arrival. and pitch angle And expand the preset angle width with the arrival direction as the center direction. and pitch angle width A directional search sector is generated. Specifically, in this embodiment, based on the obtained multi-channel delay difference and array geometry, the SRP-PHAT spatial spectrum estimation algorithm is used for azimuth estimation. By performing phase alignment and energy accumulation on the theoretical delays corresponding to each candidate azimuth in the search space, the spatial spectrum function is obtained: Where M is the number of array elements. This is the output of the generalized cross-correlation between the i-th and j-th channels. Candidate location The theoretical time delay is determined by searching for spatial spectrum peaks within the range of azimuth 0° to 360° and elevation 0° to 90°, thus identifying the azimuth and elevation angles corresponding to the target's maximum peak value.

[0031] Based on the start time, duration, target propagation characteristics, and response hysteresis of each sensor, an associated time window is established.

[0032] S3. Control the optical detection equipment to collect target image data within the directional search sector and in the associated time window, and perform target detection, entity confirmation and trajectory extraction on the target image data to obtain target visual information.

[0033] Preferably, step S3 includes: The gimbal or photoelectric turntable is adjusted according to the directional search sector to ensure that visible light imaging equipment and infrared imaging equipment prioritize coverage of the directional search sector.

[0034] Within the associated time window, continuous image frames are acquired, and candidate target detection is performed within a limited field of view. Specifically, in this embodiment, a deep learning-based target detection network (such as the YOLO series network) is used to perform candidate target detection within a limited field of view. The input is a visible light image and an infrared image, and the output is the detection bounding box and the target confidence score.

[0035] For candidate target regions, entity confirmation is performed by combining optical features, including shape contour, scale variation, and motion continuity, to distinguish real low-altitude flying targets from interference targets. Specifically, the orientation gradient histogram features and local binary pattern features of the candidate regions are extracted as shape contour descriptors; the cross-union ratio and center displacement of candidate boxes between adjacent frames are calculated to evaluate motion continuity; the frequency flicker characteristics of the candidate regions are analyzed to identify the periodic flicker patterns of rotor targets; and the edge contours in the visible light image are cross-validated with the thermal distribution information in the infrared image.

[0036] The intersection-union ratio (IUU) of adjacent candidate frames can be expressed as: ,in, and These represent the bounding boxes of candidate targets in two adjacent frames. Based on the above features, combined with shape contour, scale variation, and motion continuity, entity confirmation is performed on candidate targets to distinguish between real low-altitude flying targets and interference targets.

[0037] Once the target is confirmed, it is associated with consecutive frames to extract the target's motion trajectory and output the target's visual category information, target position change information, and target trajectory information.

[0038] Preferably, entity confirmation and consecutive frame association include: A temporal visual observation sequence is established based on the displacement changes, area changes, contour stability, and orientation changes of candidate targets in adjacent image frames; Cross-validate edge contour information in visible light images with thermal distribution information in infrared images; Cross-frame matching of the same target is performed based on the target's velocity and orientation constraints in consecutive image frames to maintain the continuity and consistency of the target's identity; After obtaining a stable trajectory, the target category label, trajectory curve, and visual confirmation confidence level are output.

[0039] In this embodiment, after the above confirmation process, the system eliminates interference from a bird and a glass curtain wall reflection. The bird exhibits an unstable outline and does not have rotor flickering characteristics, while the glass curtain wall reflection is only visible in the visible light band and has no corresponding heat source in the infrared band. Finally, a real quadcopter drone target is confirmed, and an online multi-target tracking algorithm (such as a DeepSORT-like algorithm) is initiated on the target. The output includes the target category label "quadcopter drone," the trajectory curve, and the visual confirmation confidence level.

[0040] S4. Under the constraints of directional search sector and associated time window, the electromagnetic signals in the online multi-target tracking algorithm (such as DeepSORT-like algorithms) are scanned, identified and the direction of arrival is estimated to obtain the electromagnetic characteristic information of the target.

[0041] Preferably, step S4 includes: Under the constraints of directional search sectors and associated time windows, electromagnetic signals within a preset frequency band are directionally scanned and acquired to obtain radio frequency observation data.

[0042] The radio frequency observation data is subjected to energy detection, spectral structure analysis, and modulation feature analysis to identify suspected target electromagnetic signals. Specifically, in this embodiment, effective signal segments are first screened by an energy detector; then, short-time Fourier transform is performed on the effective signal segments to obtain a time-spectrum diagram, and the suspected target communication signals are identified by combining the spectral structure and modulation feature analysis.

[0043] The short-time Fourier transform of a radio frequency signal can be expressed as: Where x(m) is a discrete radio frequency signal, For window functions.

[0044] Electromagnetic features, including carrier frequency, bandwidth, and frequency hopping characteristics, are extracted from the identified suspected target electromagnetic signals. Specifically, in this embodiment, carrier frequency, bandwidth, fixed frequency characteristics, power intensity, and OFDM baseband characteristics are extracted from the target signal, and the signal is determined to be a non-frequency hopping fixed frequency communication signal. By combining a multi-channel receiving structure, the direction of arrival of the electromagnetic signal of the suspected target is estimated, so as to obtain the electromagnetic characteristic information of the target and the corresponding direction information.

[0045] Preferably, the target electromagnetic characteristic information, used as supplementary communication verification information for target confirmation, includes: Radio frequency observation data is segmented in the time domain and decomposed in the frequency domain to distinguish between continuous transmission signals, intermittent transmission signals and frequency hopping signals.

[0046] The spectrum occupancy range, power variation law, duration characteristics and periodic variation characteristics of different types of signals are calculated respectively, and matched with the preset target communication template.

[0047] When the matching result meets the preset conditions, the incoming wave direction is calculated based on the phase difference, amplitude difference and / or spatial spectrum peak value of the multi-channel received signal.

[0048] For example, the angle of arrival estimation based on phase difference can be approximated as: ,in, Let d be the phase difference between adjacent receiving channels, d be the element spacing, and λ be the signal wavelength. The direction of the incoming electromagnetic signal wave.

[0049] The obtained direction of arrival is verified against the directional search sector and the direction of the optical trajectory. When the verification meets the consistency condition, the corresponding electromagnetic signal is determined as the communication supplementary information for target confirmation.

[0050] S5. Perform spatiotemporal alignment, spatial mapping, and cross-modal association on acoustic direction information, target visual information, and target electromagnetic feature information, and output target confirmation results based on direction consistency, temporal consistency, and feature consistency.

[0051] Preferably, step S5 includes: Acoustic, optical, and electromagnetic detection results are each assigned a time signature under a unified clock reference to establish cross-modal time synchronization relationships.

[0052] Based on the installation location, attitude parameters, field of view parameters, array geometric parameters, and coordinate transformation relationships of each sensor, a unified coordinate mapping is performed for acoustic orientation, optical target position, and electromagnetic wave direction.

[0053] The mapped multimodal observation results are projected into a unified reference coordinate system, and a set of candidate targets is established according to adjacent time windows and adjacent spatial ranges.

[0054] The acoustic, visual, and communication attributes in the candidate target set are pre-correlated and organized to form target candidate association data for joint judgment.

[0055] Preferably, in step S5, cross-modal association and target confirmation include: The acoustic orientation, optical trajectory direction and electromagnetic wave direction in the target candidate association data are compared for consistency to obtain the direction association results. Match the correspondence between the occurrence times of acoustic events, visual targets, and electromagnetic signals to obtain temporal correlation results; The feature association results are calculated based on the correspondence between the target's visual category features, acoustic features, and electromagnetic features, and the behavior association results are obtained by combining the target trajectory continuity. A weighted scoring fusion method is used to jointly determine the directional association results, temporal association results, feature association results, and behavioral association results, among which the comprehensive confidence score is used. Calculate using the following formula: ,in, Scoring is given for consistency of direction. For time-series matching scores, For feature matching scores, For trajectory continuity score, , , , For the preset weighting coefficients, satisfy ;when When, output the target confirmation result; when When, output the result to be confirmed and continue observation; when At that time, output the false alarm suppression result.

[0056] S6. Based on the target confirmation results, continuously track the target and feed back the optical confirmation results and / or electromagnetic confirmation results to the acoustic event recognition process and / or cross-modal association process to correct the subsequent target search range, association conditions or triggering conditions, forming a closed-loop cooperative detection mechanism based on acoustic leader constraints.

[0057] Preferably, step S6 includes: Establish a target state model for the confirmed target, including target location, target velocity, target direction, target category, and target confidence level; The target state model is recursively updated using acoustic, optical, and electromagnetic observations received at subsequent time steps to achieve continuous fusion tracking of the target. The recursive update employs a Kalman filter framework, and the state prediction equation and state update equation are as follows: ,in, Let k be the predicted state value. Here is the state transition matrix. This is the state estimate at time k-1. To control the input matrix, For control vectors; This is the state update value at time k. Here is the Kalman gain matrix. Let k be the multimodal observation vector at time k. The observation matrix; When optical observation results obtain higher precision position or trajectory information, the position or trajectory information is fed back to the electromagnetic scanning process and / or acoustic event recognition process; When the electromagnetic observation results obtain stable communication characteristics or stable incoming wave direction, the stable communication characteristics or stable incoming wave direction are fed back to the optical target re-acquisition process and / or cross-modal correlation process; When the joint determination result indicates that the previous acoustic event was falsely triggered, the false trigger result is fed back to the acoustic event identification process to update the trigger threshold, optimize the acoustic discrimination conditions, or correct the associated time window.

[0058] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A method for detecting and confirming low-altitude targets using a combination of acoustic, optical, and electromagnetic signals based on acoustic guidance constraints, characterized in that... Includes the following steps: S1. Perform continuous acoustic monitoring of the target area, and preprocess and extract features from the collected acoustic signals to identify acoustic events corresponding to suspected low-altitude flying targets. S2. Determine the acoustic direction and time information of the suspected low-altitude flying target based on the acoustic event, and generate a directional search sector and associated time window based on the acoustic direction and time information as a restricted search condition for subsequent optical and electromagnetic detection. S3. Control the optical detection device to collect target image data within the directional search sector and the associated time window, and perform target detection, entity confirmation and trajectory extraction on the target image data to obtain target visual information; S4. Under the constraints of the directional search sector and the associated time window, the electromagnetic signals in the corresponding direction and / or within the preset frequency band range of the directional search sector are scanned, identified, and the direction of arrival is estimated to obtain the target electromagnetic characteristic information. S5. Perform spatiotemporal alignment, spatial mapping, and cross-modal association on the acoustic direction information, the target visual information, and the target electromagnetic feature information, and output the target confirmation result based on direction consistency, temporal consistency, and feature consistency; S6. Based on the target confirmation result, continuously track the target and feed back the optical confirmation result and / or electromagnetic confirmation result to the acoustic event recognition process and / or cross-modal association process to correct the subsequent target search range, association conditions or triggering conditions, forming a closed-loop cooperative detection mechanism based on acoustic leader constraints.

2. The method for low-altitude target detection and confirmation based on acoustic guidance constraints using acousto-optic-electromagnetic coordination according to claim 1, characterized in that, Step S1 includes: A microphone array is used to continuously collect multi-channel acoustic data from the target area. The acquired raw acoustic signals are subjected to bandpass filtering, background noise reduction, and frame segmentation to suppress environmental noise interference. Time-frequency analysis is performed on the processed acoustic signal to extract acoustic features including short-time energy characteristics, spectral peak characteristics, and harmonic structure characteristics; Based on the acoustic characteristics, it is determined whether the acoustic event should be generated, and the acoustic event is used as a precursor event to trigger subsequent optical and electromagnetic detection.

3. The method for low-altitude target detection and confirmation based on acoustic guidance constraints using acousto-optic-electromagnetic coordination according to claim 1, characterized in that, Step S2 includes: The inter-channel time delay difference is calculated for the multi-channel acoustic signals corresponding to the acoustic event, and the arrival direction of the suspected low-altitude flying target is estimated by combining the geometric relationship of the microphone array. The inter-channel time delay difference is determined by the peak position of the generalized cross-correlation function, which is: ,in, Let be the cross-power spectral density of the signals received by the i-th array element and the reference array element. It is a frequency domain weighting function; Based on the direction of arrival, the spatial azimuth and pitch angles are calculated, and the horizontal angle and pitch angle width are expanded with the direction of arrival as the central direction to generate the directional search sector; Based on the start time, duration, target propagation characteristics, and response hysteresis of each sensor, an associated time window is established.

4. The method for low-altitude target detection and confirmation based on acoustic guidance constraints using acousto-optic-electromagnetic coordination according to claim 1, characterized in that, Step S3 includes: The gimbal or photoelectric turntable is adjusted according to the directional search sector so that the visible light imaging device and the infrared imaging device can preferentially cover the directional search sector. Continuous image frames are acquired within the associated time window, and candidate target detection is performed within a limited field of view; For candidate target areas, physical confirmation is performed by combining optical features, including shape contour, scale changes, and motion continuity, in order to distinguish between real low-altitude flying targets and interference targets. Once the target is confirmed, it is associated with consecutive frames to extract the target's motion trajectory and output the target's visual category information, target position change information, and target trajectory information.

5. The method for low-altitude target detection and confirmation based on acoustic guidance constraints using acousto-optic-electromagnetic coordination according to claim 4, characterized in that, Entity confirmation and consecutive frame association include: A temporal visual observation sequence is established based on the displacement changes, area changes, contour stability, and orientation changes of candidate targets in adjacent image frames; Cross-validate edge contour information in visible light images with thermal distribution information in infrared images; Cross-frame matching of the same target is performed based on the target's velocity and orientation constraints in consecutive image frames to maintain the continuity and consistency of the target's identity; After obtaining a stable trajectory, the target category label, trajectory curve, and visual confirmation confidence level are output.

6. The method for low-altitude target detection and confirmation based on acoustic guidance constraints using acousto-optic-electromagnetic coordination according to claim 1, characterized in that, Step S4 includes: Under the constraints of the directional search sector and the associated time window, electromagnetic signals within a preset frequency band are directionally scanned and acquired to obtain radio frequency observation data; The radio frequency observation data is subjected to energy detection, spectral structure analysis, and modulation feature analysis to identify suspected target electromagnetic signals; Electromagnetic features, including carrier frequency, bandwidth, and frequency hopping characteristics, are extracted from the electromagnetic signals of identified suspected targets. By combining a multi-channel receiving structure, the direction of arrival of the suspected target electromagnetic signal is estimated to obtain the target's electromagnetic characteristics and corresponding direction information.

7. The method for low-altitude target detection and confirmation based on acoustic guidance constraints using acousto-optic-electromagnetic coordination according to claim 6, characterized in that, The target's electromagnetic signature information serves as supplementary communication verification information for target confirmation, including: The radio frequency observation data is segmented in the time domain and decomposed in the frequency domain to distinguish between continuous transmission signals, intermittent transmission signals and frequency hopping signals; The spectrum occupancy range, power variation law, duration characteristics and periodic variation characteristics of different types of signals are calculated respectively, and matched with the preset target communication template; When the matching result meets the preset conditions, the direction of incoming wave is calculated based on the phase difference, amplitude difference and / or spatial spectrum peak value of the multi-channel received signal; The obtained direction of arrival is verified to correspond with the directional search sector and the direction of optical trajectory. When the verification meets the consistency condition, the corresponding electromagnetic signal is determined as the communication supplementary verification information for target confirmation.

8. The method for low-altitude target detection and confirmation based on acoustic guidance constraints using acousto-optic-electromagnetic coordination according to claim 1, characterized in that, Step S5 includes: Acoustic, optical, and electromagnetic detection results are each assigned a time signature under a unified clock reference to establish cross-modal time synchronization relationships; Based on the installation position, attitude parameters, field of view parameters, array geometric parameters and coordinate transformation relationships of each sensor, a unified coordinate mapping is performed on the acoustic orientation, optical target position and electromagnetic wave direction; The mapped multimodal observation results are projected onto a unified reference coordinate system, and a set of candidate targets is established according to adjacent time windows and adjacent spatial ranges; The acoustic, visual, and communication attributes in the candidate target set are pre-associated and organized to form target candidate association data for joint determination.

9. The method for low-altitude target detection and confirmation based on acoustic guidance constraints using acousto-optic-electromagnetic coordination according to claim 1, characterized in that, In step S5, cross-modal association and target confirmation include: The acoustic orientation, optical trajectory direction and electromagnetic wave direction in the target candidate association data are compared for consistency to obtain the direction association results. Match the correspondence between the occurrence times of acoustic events, visual targets, and electromagnetic signals to obtain temporal correlation results; The feature association results are calculated based on the correspondence between the target's visual category features, acoustic features, and electromagnetic features, and the behavior association results are obtained by combining the target trajectory continuity. A weighted scoring fusion method is used to jointly determine the directional association results, temporal association results, feature association results, and behavioral association results, wherein the comprehensive confidence score is used. Calculate using the following formula: ,in, Scoring is given for consistency of direction. For time-series matching scores, For feature matching scores, For trajectory continuity score, , , , For the preset weighting coefficients, satisfy ;when When, output the target confirmation result; when When, output the result to be confirmed and continue observation; when At that time, output the false alarm suppression result.

10. The method for low-altitude target detection and confirmation based on acoustic guidance constraints using acousto-optic-electromagnetic coordination according to claim 1, characterized in that, Step S6 includes: Establish a target state model for the confirmed target, including target location, target velocity, target direction, target category, and target confidence level; The target state model is recursively updated using acoustic, optical, and electromagnetic observations received at subsequent time points to achieve continuous fusion tracking of the target. The recursive update employs a Kalman filter framework, and the state prediction equation and state update equation are as follows: ,in, Let k be the predicted state value. Here is the state transition matrix. This is the state estimate at time k-1. To control the input matrix, For control vectors; This is the state update value at time k. Here is the Kalman gain matrix. Let k be the multimodal observation vector at time k. The observation matrix; When optical observation results obtain higher precision position or trajectory information, the position or trajectory information is fed back to the electromagnetic scanning process and / or acoustic event recognition process; When the electromagnetic observation results obtain stable communication characteristics or stable incoming wave direction, the stable communication characteristics or stable incoming wave direction are fed back to the optical target re-acquisition process and / or cross-modal correlation process; When the joint determination result indicates that the previous acoustic event was falsely triggered, the false trigger result is fed back to the acoustic event identification process to update the trigger threshold, optimize the acoustic discrimination conditions, or correct the associated time window.

Images