Unmanned aerial vehicle multi-source fusion target detection method based on support vector machine

Abstract

The invention relates to an unmanned aerial vehicle multi-source fusion target detection method based on a support vector machine, and belongs to the technical field of image processing, and the method comprises the steps: firstly designing an improved constraint energy minimization CEM filter which is a finite impulse response linear filter, enabling the preprocessed data to pass through the linear filter, and obtaining a target detection result; hyperspectral personnel target information is obtained by adopting an adaptive threshold segmentation method; secondly, for RGB images and infrared data, more local information is obtained based on an improved YOLOv8 model, the detection precision of small targets of unmanned aerial vehicle personnel is improved, and then a convolution and attention fusion module is added to provide more comprehensive and effective feature extraction capability for an unmanned aerial vehicle multi-source fusion target detection model. A sliding window is utilized to improve the detection capability of personnel for small targets; and finally, performing intelligent inspection based on a support vector machine classifier, fusing the target information identified by the hyperspectral data, RGB and infrared data, and outputting the target of the drowning person of interest after multi-source fusion.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a multi-source fusion target detection method for unmanned aerial vehicles based on a support vector machine. Background Art

[0002] According to the World Health Organization (WHO), approximately 300,000 people die from drowning each year worldwide. In my country, the number of drowning deaths is approximately 59,000 per year, with minors accounting for over 95%. Rapid and accurate identification of drowning victims is crucial for rescue operations. Studies have shown that, without rescue measures, drowning victims begin to show signs of brain death approximately five minutes after exposure. Therefore, timely detection of drowning victims within five minutes is crucial. Advances in drone technology, with their low cost, remote operation capabilities, and high maneuverability, have been applied in numerous fields and are playing an increasingly important role in routine inspection missions.

[0003] When using drones for water surface inspections, intelligent recognition technology can be used to monitor the water surface in real time and effectively capture the movements of people in the surrounding area. This helps detect and prevent potentially dangerous behavior and reduce the incidence of people falling into the water. Currently, most drone inspections use RGB and infrared images to detect people falling into the water. However, these two methods rely on the shape characteristics of people, which can easily lead to false detections and missed detections.

[0004] Advances in remote sensing technology have garnered significant attention in various fields, including spectral unmixing, segmentation, classification, and target detection. Hyperspectral images are rich in spectral information, and subtle differences can be exploited to identify small targets. Hyperspectral target detection relies on spectral differences between the target and the background to identify the target. Various classic algorithms have been proposed and applied, such as matched filtering (MF), which enhances the target spectrum and attenuates the background. Spectral matched filtering (SMF), based on the Gaussian assumption, is suitable for simple backgrounds, but requires a known signal abundance for sub-pixel targets, limiting its practicality. Robey et al. proposed adaptive matched filtering (AMF), based on SMF. These basic algorithms suffer from poor detection accuracy in scenarios with complex spectral correlations, such as those involving spectral confusion between water and clothing, interference from algae / plankton, and dynamic water surface reflections, as well as target-background variability, such as dynamic wave interference, interference from underwater debris, and sudden changes in lighting conditions.

[0005] In the existing technology, although there are multiple sensor fusions at the feature level, the existing fusion strategies are all at the feature level. The fusion algorithm has high computational complexity, is difficult, and has high requirements for computer equipment. It is difficult to achieve real-time detection and cannot be directly applied to drones for real-time inspections of people who fall into the water. Summary of the Invention

[0006] In view of the above problems, the present invention provides a multi-source fusion target detection method for unmanned aerial vehicles (UAVs) based on support vector machines, realizing the complementary advantages of multi-source sensors. The present invention first designs an improved constrained energy minimization (CEM) filter, which is a finite impulse response linear filter. The preprocessed data is passed through the linear filter and an adaptive threshold segmentation method is used to obtain hyperspectral personnel target information. Secondly, for RGB images and infrared data, an improved YOLOv8 model is used to obtain more local information, improving the detection accuracy of small UAV personnel targets. Then, a convolution and attention fusion module (CAFM) is added to provide a more comprehensive and effective feature extraction capability for the UAV multi-source fusion target detection model, and a sliding window is used to improve the detection capability of small personnel targets. Finally, intelligent inspection is performed based on a support vector machine (SVM) classifier, and the target information identified by hyperspectral data, RGB and infrared data is fused, and the multi-source fusion output is the target of interest falling into the water.

[0007] The present invention provides a multi-source fusion target detection method for unmanned aerial vehicles based on support vector machines, comprising:

[0008] S1. Establish a multi-source fusion target detection model for UAVs;

[0009] Using multi-source sensors carried by drones, we collected images of people falling into the water under different lighting and temperature scenarios, and obtained hyperspectral image datasets, RGB image datasets, and infrared image datasets.

[0010] S2. Let t = 1. When t = 1, it represents the initial time;

[0011] S3. Input the hyperspectral image data at time t into the preprocessing module A t Perform normalization and noise reduction processing to obtain clean hyperspectral image data at time t; extract the target of interest from the clean hyperspectral image data at time t;

[0012] S4. Preprocess the RGB image data at time t and the infrared image data at time t, respectively, to obtain the processed RGB image data at time t and the processed infrared image data at time t;

[0013] S5. Input the target of interest of the hyperspectral image data at time t into the improved constrained energy minimization CEM filter B t , output the detection response graph at time t;

[0014] Perform square calculation on the detection response map output at time t to obtain the square enhancement response map at time t;

[0015] An unsupervised threshold calculation strategy is used to generate an adaptive threshold. Based on the adaptive threshold, the background residual speckle noise of the square enhancement response image at time t is eliminated, the person target of interest is enhanced, and the person target is segmented to obtain the final hyperspectral image of the person of interest at time t.

[0016] S6. Input the RGB image data at time t and the infrared image data at time t into the improved YOLOv8 model C t , output the target of interest in the RGB image at time t and the target of interest in the infrared image at time t respectively;

[0017] S7. Combining the target of interest in the RGB image at time t, the target of interest in the infrared image at time t, and the target of interest in the final hyperspectral image at time t to obtain a combined feature at time t;

[0018] Input the combined features at time t into the support vector machine SVM D t , based on the decision function, make a decision,

[0019] Obtain the final bounding box coordinates of the target of interest at time t, which represents the position of the target person falling into the water at time t;

[0020] S8. Determine whether t is greater than or equal to T. If so, obtain the position of the target person falling into the water at time t and the final UAV multi-source fusion target detection model. If not, set t = t + 1 and return to step S3;

[0021] S9. Use the final UAV multi-source fusion target detection model to perform UAV multi-source fusion target detection.

[0022] Optionally, the specific steps of extracting the target of interest from the clean hyperspectral image data at time t include:

[0023] S301. Normalizing the hyperspectral image dataset;

[0024] S302. Using wavelet transform to perform noise reduction on the normalized hyperspectral image dataset to obtain clean hyperspectral image data;

[0025] S303. Performing man-overboard detection on the clean hyperspectral image data to obtain corresponding man-overboard pixel regions;

[0026] Based on the random sampling technology strategy, multiple points of interest are extracted in the corresponding pixel area of the person falling into the water, and the average value of the multiple points of interest is taken as the target of interest in the hyperspectral image data.

[0027] Optionally, the specific steps of respectively obtaining the processed RGB image data at time t and the processed infrared image data at time t include:

[0028] S401. The RGB image dataset is size normalized, cropped, color corrected, pixel value normalized, data enhanced, and denoised to obtain a processed RGB image;

[0029] S402. Perform temperature normalization, histogram equalization, pseudo color enhancement, deblurring, background suppression, and data enhancement processing on the infrared image data to obtain a preprocessed infrared image dataset.

[0030] Optionally, the specific steps of establishing an improved constrained energy minimization CEM filter include:

[0031] Identify constraints;

[0032] The design of FIR linear filtering algorithm is regarded as a constrained energy minimization problem;

[0033] Based on FIR linear filter and constrained energy minimization CEM problem, an improved constrained energy minimization CEM filter is obtained.

[0034] Optionally, the constraint condition is expressed as:

[0035]

[0036] Among them, d is the personnel prior spectrum vector, which refers to the typical spectrum feature vector of the person falling into the water obtained in advance through experiments, w is the filter weight coefficient, c is an arbitrary scalar, usually 1, d k is the vector of the kth band of the personnel prior spectrum, w k is the weight coefficient of the filter in the kth band, k = 1, 2, 3…l.

[0037] Optionally, constrain the energy minimization problem, expressed as:

[0038]

[0039] Among them, w is the filter weight coefficient, y i is the output of the linear filter for the i-th pixel, i = 1, 2, 3…N, N is the total number of pixels in the hyperspectral image, X l×l is the autocorrelation matrix of the band, w k is the weight coefficient of the filter in the kth band, d is the personnel prior spectrum vector, d k is the vector of the kth band of the person's prior spectrum, k = 1, 2, 3…l, and l represents the total number of bands.

[0040] Optionally, a drone inspection target detection head, a convolution and attention fusion module CAFM, and a sliding window module are added to the YOLOv8 model to obtain an improved YOLOv8 model.

[0041] Optionally, the attention fusion module CAFM includes a local branch and a global branch;

[0042] The local branch is a neural network, including channel dimension adaptation, channel reorganization and local feature capture;

[0043] The global branch is a Transformer architecture, including query, key and value generation, attention graph construction and global output calculation.

[0044] Optionally, the decision function is expressed as:

[0045]

[0046] Among them, SV is the set of support vectors, sign is the sign function that returns +1 or -1, b is the bias term used to adjust the position of the classification hyperplane, and x is the combined feature; α n is the Lagrange multiplier of the nth support vector, y n is the category label of the nth support vector; K(.) is the kernel function.

[0047] Optionally, the constraint condition of the support vector machine SVM is expressed as:

[0048]

[0049] 0≤α n ≤C

[0050] Among them, C is the penalty parameter, α n is the Lagrange multiplier of the nth support vector, y n is the category label of the nth support vector.

[0051] Small targets in this invention include: (1) the head or arm of a drowning person: when photographed from a distance, the target may only appear as a dot-like area of a few pixels, which cannot be effectively captured by traditional shape detection. (2) Partially submerged targets: such as palms or pieces of clothing that are only exposed above the water surface, which have irregular shapes and spectral characteristics similar to the background.

[0052] The present invention uses the YOLO algorithm for visible light and infrared detection. The YOLO algorithm can recognize dozens of frames of images per second, and also uses a supervised algorithm with a second-level processing speed for hyperspectral images. Combined with the drone's inspection speed of 10-20m / s and the algorithm processing delay (which can reach the second level after optimization), the closed loop from data collection to alarm can be completed within 10 seconds to 2 minutes, meeting the 5-minute rescue window.

[0053] The present invention realizes the complementary advantages of spectral and spatial resolution, dynamic environmental adaptability, and data fusion to enhance detection confidence. Among them, the present invention provides continuous spectral information through a hyperspectral sensor, which can identify target substances (such as the clothing material or skin spectral characteristics of a person who falls into the water) through subtle spectral differences. Visible light RGB sensors provide high spatial resolution images to capture the shape, texture and motion trajectory of the target, and assist in locating the specific position of the person who falls into the water. Infrared sensor: Detects targets based on thermal radiation differences, can work at night or in low light conditions, and quickly locates the target through the temperature difference between the human body and the water body (such as the human body temperature is about 36°C, and the water temperature may be lower than 20°C).

[0054] The present invention uses an SVM classifier to integrate multi-source detection results and reduce the misjudgment of a single sensor through weighted voting. Among them, when the hyperspectral spectrum determines that there is a human spectrum in a certain area but the infrared does not detect temperature anomalies, a secondary verification can be triggered to improve the detection confidence.

[0055] Compared with the prior art, the present invention has at least the following beneficial effects:

[0056] (1) The present invention achieves complementary advantages in spectral and spatial resolution, dynamic environmental adaptability, and data fusion to enhance detection confidence;

[0057] (2) The present invention uses an SVM classifier to integrate multi-source detection results and reduce misjudgments of a single sensor through weighted voting;

[0058] (3) The real-time detection time interval of the multi-source fusion target detection model of the UAV of the present invention is in the order of seconds;

[0059] (4) The present invention obtains more local information of RGB images and infrared data based on the improved YOLOv8 model, thereby improving the detection accuracy of small targets such as drone personnel;

[0060] (5) The convolution and attention fusion module (CAFM) is added to the improved YOLOv8 model to provide a more comprehensive and effective feature extraction capability for the multi-source fusion target detection model of drones, and the sliding window is used to improve the detection capability of small human targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The drawings are only for purposes of illustrating particular embodiments and are not to be considered limiting of the invention.

[0062] Figure 1 Schematic diagram of multi-source fusion target detection for UAVs based on support vector machines in an embodiment of the present invention;

[0063] Figure 2 A schematic diagram of the process of acquiring an object of interest from hyperspectral data in an embodiment of the present invention;

[0064] Figure 3 Schematic diagram of the process of obtaining an object of interest using RGB and infrared data in an embodiment of the present invention;

[0065] Figure 4 A schematic diagram showing the structure of a fine detection head is added to an embodiment of the present invention;

[0066] Figure 5 This is a schematic diagram of the CAFM structure with the convolution and attention fusion module added in an embodiment of the present invention. DETAILED DESCRIPTION

[0067] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. In addition, the present invention can also be implemented in other ways different from those described herein. Therefore, the scope of protection of the present invention is not limited by the specific embodiments disclosed below.

[0068] A specific embodiment of the present invention, as Figure 1-5 , discloses a multi-source fusion target detection method for UAVs based on support vector machines. The specific implementation steps are as follows:

[0069] S1. Build a multi-source fusion target detection model for UAVs, including a preprocessing module, an improved constrained energy minimization (CEM) filter, an improved YOLOv8 model, and a support vector machine (SVM).

[0070] Using multi-source sensors carried by drones, we collected images of people falling into the water under different lighting and temperature scenarios, and obtained hyperspectral image datasets, RGB image datasets, and infrared image datasets.

[0071] Optionally, the multi-source sensor data includes: a hyperspectral sensor, an RGB sensor, and an infrared sensor;

[0072] The different light and temperature scenes include morning, noon and evening light and temperature scenes;

[0073] S2. Let t = 1. When t = 1, it represents the initial time;

[0074] S3. Input the hyperspectral image data at time t into the preprocessing module A t , perform normalization and noise reduction processing to obtain clean hyperspectral image data at time t; extract the target of interest from the clean hyperspectral image data at time t;

[0075] Optionally, the specific steps of extracting the target of interest from the clean hyperspectral image data at time t include:

[0076] S301. Normalize the hyperspectral image dataset. The expression is:

[0077]

[0078] Among them, X0 is the original data, and X is the normalized data.

[0079] S302. Using wavelet transform to perform noise reduction on the normalized hyperspectral image dataset to obtain clean hyperspectral image data;

[0080] Furthermore, the specific steps to obtain clean hyperspectral image data include:

[0081] Decomposing the hyperspectral data into multiple bands of data, and decomposing the data of each band into approximate coefficients and detail coefficients through wavelet transform; the approximate coefficients are the low-frequency part, and the detail coefficients are the high-frequency part;

[0082] The approximate coefficient and detail coefficient of each band data are de-noised by using the threshold parameter to obtain the denoised hyperspectral data, which is characterized as clean hyperspectral image data. The expression is:

[0083]

[0084] Among them, α is the scale factor and is greater than 0, τ is the displacement, t is the duration, WT x (α,τ) is the wavelet coefficient, and its value represents the high and low frequency signals respectively. x(t) is the spectral signal at time t, that is, the spectral data of the hyperspectral image. is the mother wavelet function.

[0085] It can be understood that the approximate coefficients contain the main information of the spectral signal, which is the background or overall trend of the image; in the denoising process, the approximate coefficients are used to reconstruct the denoised image to maintain the original structure and spectral characteristics of the image.

[0086] The detail coefficient contains the details and noise of the image and reflects the local changes of the spectral signal. In the noise reduction process, threshold processing or other filtering methods can be used to reduce the influence of noise while retaining useful detail information as much as possible.

[0087] S303. Performing man-overboard detection on the clean hyperspectral image data to obtain corresponding man-overboard pixel regions;

[0088] Based on the random sampling technology strategy, multiple points of interest are extracted in the corresponding pixel area of the person falling into the water, and the average value of the multiple points of interest is taken as the target of interest in the hyperspectral image data.

[0089] Exemplarily, the feature corresponding to the point of interest includes spectral information of the pixel, that is, reflectivity information of the pixel at various wavelengths, and the reflectivity forms a spectral reflectivity curve in continuous wavelength bands.

[0090] S4. Preprocess the RGB image data at time t and the infrared image data at time t, respectively, to obtain the processed RGB image data at time t and the processed infrared image data at time t;

[0091] Optionally, the specific steps of respectively obtaining the processed RGB image data at time t and the processed infrared image data at time t include:

[0092] S401. The RGB image dataset is size normalized, cropped, color corrected, pixel value normalized, data enhanced, and denoised to obtain a processed RGB image;

[0093] In this invention, the RGB images are normalized to ensure that all input images have the same size, so that the model can accept fixed-size inputs, simplifying network design and calculation;

[0094] The cropping is to remove irrelevant parts of the RGB image, focus on the area of interest, and improve the model's attention to specific targets and recognition accuracy;

[0095] The color correction includes adjusting the brightness, contrast, saturation, etc. of the image to reduce the impact of different lighting conditions and make the model more robust to lighting changes;

[0096] The pixel value normalization includes scaling the pixel value to a specific range to accelerate convergence during model training and prevent gradient disappearance or explosion.

[0097] The data enhancement includes increasing the diversity of data by rotating, flipping, scaling, shearing, adding noise, etc., improving the generalization ability of the model and reducing overfitting;

[0098] The denoising process includes reducing the noise in the image, which is used to improve the image quality, increase the robustness of the model to noise, and enhance recognition accuracy.

[0099] S402. The infrared image data is subjected to temperature normalization, histogram equalization, pseudo color enhancement, deblurring, background suppression, and data enhancement processing respectively; a preprocessed infrared image dataset is obtained;

[0100] Optionally, the temperature normalization includes scaling the representative temperature pixel values of the infrared image to a uniform thermal radiation range, so that the model can process images of different temperature ranges and improve the consistency of recognition;

[0101] The histogram equalization includes adjusting the histogram of the infrared image to make the distribution of pixel values more uniform, enhance the contrast of the image, and improve the visibility and recognition effect of the infrared image;

[0102] The pseudo color enhancement includes converting the grayscale image of the infrared image into a pseudo color image to highlight specific temperature areas, improve the visual interpretability of the infrared image, and help the model better identify the target;

[0103] The deblurring includes reducing infrared image blur caused by motion or atmospheric turbulence, improving the clarity of the infrared image, and enhancing the recognizability of the target;

[0104] The background suppression includes reducing or eliminating background interference in infrared images, highlighting the target, and improving the detection and recognition rate of the target;

[0105] The data enhancement includes increasing the diversity of data by means of rotation, flipping, etc., improving the generalization ability of the model and reducing overfitting.

[0106] In the present invention, the infrared image data is subjected to temperature normalization, histogram equalization, pseudo color enhancement, deblurring, background suppression and data enhancement processing to meet the requirements of detection accuracy and real-time performance.

[0107] S5. Input the target of interest of the hyperspectral image data at time t into the improved constrained energy minimization CEM filter B t , output the detection response graph at time t;

[0108] Perform square calculation on the detection response map output at time t to obtain the square enhancement response map at time t;

[0109] An unsupervised threshold calculation strategy is used to generate an adaptive threshold. Based on the adaptive threshold, the background residual speckle noise of the square enhancement response image at time t is eliminated, the person target of interest is enhanced, and the person target is segmented to obtain the final hyperspectral image of the person of interest at time t.

[0110] Alternatively, the squared augmented response plot expression is:

[0111]

[0112] Among them, D is the result of square calculation, D CEM This is the detection response diagram of the CEM filter.

[0113] Optionally, the expression of the between-class variance g is:

[0114] g=w1×(u1-u) 2 +w2×(u2-u) 2

[0115] Among them, u is the total average brightness value of the image, w1 is the percentage of target pixels, u1 is the average brightness value of target pixels, w2 is the percentage of background pixels, and u2 is the average brightness value of background pixels.

[0116] Furthermore, the expression of the total average brightness value of the image is:

[0117] u=w1×u1+w2×u2

[0118] Among them, u is the average brightness value of the image, w1 is the percentage of target pixels, u1 is the average brightness value of target pixels, w2 is the percentage of background pixels, and u2 is the average brightness value of background pixels.

[0119] When the inter-class variance g is the largest, the corresponding threshold is the adaptive segmentation threshold:

[0120]

[0121] Where T is the adaptive threshold, DN max The maximum brightness level of the hyperspectral detection image.

[0122] Optionally, the specific steps of establishing an improved constrained energy minimization CEM filter include:

[0123] Determine the constraint condition, the expression of the constraint condition is:

[0124]

[0125] Among them, d is the personnel prior spectrum vector, which refers to the typical spectrum feature vector of the person falling into the water obtained in advance through experiments, w is the filter output vector, c is an arbitrary scalar, usually 1, d k is the vector of the kth band of the personnel prior spectrum, w k is the weight coefficient of the filter in the kth band, k = 1, 2, 3…l.

[0126] The design of FIR linear filtering algorithm is regarded as a constrained energy minimization problem, which is expressed as:

[0127]

[0128] Among them, w is the filter weight coefficient, y i is the output of the linear filter for the i-th pixel, i = 1, 2, 3…N, N is the total number of pixels in the hyperspectral image, X l×l is the autocorrelation matrix of the band, w k is the weight coefficient of the filter in the kth band, d is the personnel prior spectrum vector, d k is the vector of the kth band of the person's prior spectrum, k = 1, 2, 3…l, and l represents the total number of bands.

[0129] Based on FIR linear filter and constrained energy minimization CEM problem, an improved constrained energy minimization CEM filter is obtained.

[0130] S6. Input the RGB image data at time t and the infrared image data at time t into the improved YOLOv8 model C t , output the target of interest in the RGB image at time t and the target of interest in the infrared image at time t respectively;

[0131] Optionally, a drone inspection target detection head, a convolution and attention fusion module (CAFM), and a sliding window module are added to the YOLOv8 model to obtain an improved YOLOv8 model.

[0132] Optionally, the specific steps of outputting the target of interest in the RGB image and the target of interest in the infrared image include:

[0133] Input the RGB image data at time t into the improved YOLOv8 model B t ,Based on the attention fusion module, it outputs multi-scale fusion features;

[0134] Input the multi-scale fusion features into the CBS-Concat-C2F module and output the RGB image;

[0135] The RGB image is input into the sliding window processing module, and the adjusted image features, i.e., the object of interest in the RGB image, are output.

[0136] In this paper, a drone inspection target detection head is added to the YOLOv8 model to enhance the recognition ability of small targets, such as Figure 4 As shown;

[0137] The present invention adds convolution and attention fusion module CAFM to improve the detection performance of small targets of UAV inspection personnel, such as Figure 5 As shown;

[0138] Specifically, an upsampling-concatenation-C2F layer (Upsample-Concat-C2F layer) is added after the 15-layer output of the attention head of the YOLOv8 model to output the 160×160 head of the small person target branch. Then, two convolution-normalization-SiLU activation layers (CBS-Concat-C2F) are used to first convert the 160×160 convolution to 80×80, and then the 15-layer concatenation layer outputs the 80×80 head.

[0139] The small human targets include people who fall into the water at a long distance, with low resolution or partial occlusion, specifically: (1) the head or arm of the person who falls into the water; (2) partially submerged targets.

[0140] Optionally, the attention fusion module CAFM integrates the complementary characteristics of convolutional neural networks (CNNs) and Transformer architectures, and fuses the local features extracted by the convolution operation in the convolutional neural network with the global features extracted by the attention mechanism in the Transformer architecture to achieve efficient representation of image details and overall structure, thereby enhancing the performance of target detection.

[0141] It is understandable that the details include: the limb outline, clothing texture, hair of the person who fell into the water, and the temperature difference characteristics between the human body and the water area in the infrared image.

[0142] The attention fusion module CAFM includes a local branch and a global branch;

[0143] The local branch is a neural network, including channel dimension adaptation, channel reorganization, and local feature capture;

[0144] The global branch is a Transformer architecture, including query, key and value generation, attention graph construction, and global output calculation;

[0145] The channel dimension adaptation achieves flexible configuration of the number of channels through convolution, keeping the size of the feature map unchanged to optimize subsequent feature processing.

[0146] The channel reorganization divides the input tensor into channels and applies depth-wise separable convolution to promote channel shuffling within the group, and then splices them to generate new tensors, thereby enhancing information fusion between channels.

[0147] The local feature capture is used to comprehensively extract features in spatial and spectral dimensions to improve characterization capabilities.

[0148] The query, key and value generation includes using convolution and depth-width convolution to generate query (Q), key (K) and value (V) tensors in different shapes.

[0149] Attention graph construction: After the query Q tensor and the key K tensor are reshaped, the attention graph is determined through their interaction, which reduces the computational pressure.

[0150] Optionally, the sliding window has a size of 640×640, and the original image is segmented by sliding to obtain a plurality of sliced images;

[0151] Set an overlap area of no less than 20% between adjacent slices;

[0152] Set the specified overlap ratio standard iou threshold. When the overlap ratio of the target frame of the cut image and the target frame on the original image is greater than the iou threshold, save the target frame information of the cut image. If it is less than the iou threshold, remove it and obtain the filtered cut image.

[0153] Perform global transformation on the position coordinates of the filtered cut image to obtain the adjusted image;

[0154] S7. Combining the target of interest in the RGB image at time t, the target of interest in the infrared image at time t, and the target of interest in the final hyperspectral image at time t to obtain a combined feature at time t;

[0155] Input the combined features at time t into the support vector machine SVMD t , based on the decision function, make a decision,

[0156] Obtain the final bounding box coordinates of the target of interest at time t, which represents the position of the target person falling into the water at time t;

[0157] S8. Determine whether t is greater than or equal to T. If so, obtain the position of the target person falling into the water at time t and the final UAV multi-source fusion target detection model. If not, set t = t + 1 and return to step S3;

[0158] S9. Use the final UAV multi-source fusion target detection model to perform UAV multi-source fusion target detection.

[0159] The decision function is expressed as:

[0160]

[0161] Among them, SV is the set of support vectors, sign is the sign function that returns +1 or -1, b is the bias term used to adjust the position of the classification hyperplane, and x is the combined feature; α n is the Lagrange multiplier of the nth support vector, y n is the category label of the nth support vector; K(.) is the kernel function.

[0162] Optionally, the constraint condition of the support vector machine SVM is expressed as:

[0163]

[0164] 0≤α n ≤C

[0165] Among them, C is the penalty parameter.

[0166] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by any technician familiar with this technical field within the technical scope disclosed by the present invention should be covered by the scope of protection of the present invention.

Claims

1. A multi-source fusion target detection method for UAV based on support vector machine, characterized in that: include: S1. Establish a multi-source fusion target detection model for UAVs; Collect images of people falling into the water and obtain hyperspectral image datasets, RGB image datasets, and infrared image datasets respectively; S2. Let t = 1. When t = 1, it represents the initial time; S3. Input the hyperspectral image data at time t into the preprocessing module A t Get the target of interest of clean hyperspectral image data at time t; S4 respectively obtains the RGB image data after processing at time t and the infrared image data after processing at time t; S5. Input the target of interest of the hyperspectral image data at time t into the improved constrained energy minimization CEM filter B t , and based on the strategy of unsupervised calculation threshold, the segmentation of the person target of interest is achieved, and the final hyperspectral image of the target of interest at time t is obtained; S6. Input the RGB image data at time t and the infrared image data at time t into the improved YOLOv8 model C t , output the target of interest in the RGB image at time t and the target of interest in the infrared image at time t respectively; S7. Combining the target of interest in the RGB image at time t, the target of interest in the infrared image at time t, and the target of interest in the final hyperspectral image at time t to obtain a combined feature at time t; Input the combined features at time t into the support vector machine D t , based on the decision function, obtain the position of the target person falling into the water at time t; S8. Determine whether t is greater than or equal to T. If so, obtain the position of the target person falling into the water at time t and the final UAV multi-source fusion target detection model. If not, set t = t + 1 and return to step S3; S9. Use the final UAV multi-source fusion target detection model to perform UAV multi-source fusion target detection.

2. The method for multi-source fusion target detection of unmanned aerial vehicles based on support vector machine according to claim 1 is characterized in that: Step S3 also includes the following specific steps for extracting the target of interest from the clean hyperspectral image data at time t: S301. Normalizing the hyperspectral image dataset; S302. Using wavelet transform to perform noise reduction on the normalized hyperspectral image dataset to obtain clean hyperspectral image data; S303. Performing man-overboard detection on the clean hyperspectral image data to obtain corresponding man-overboard pixel regions; Based on the random sampling technology strategy, multiple points of interest are extracted in the corresponding pixel area of the person falling into the water, and the average value of the multiple points of interest is taken as the target of interest in the hyperspectral image data.

3. The method for multi-source fusion target detection of unmanned aerial vehicle based on support vector machine according to claim 1 is characterized in that: The specific steps of respectively acquiring the processed RGB image data at time t and the processed infrared image data at time t in step S4 include: S401. The RGB image dataset is size normalized, cropped, color corrected, pixel value normalized, data enhanced, and denoised to obtain a processed RGB image; S402. Perform temperature normalization, histogram equalization, pseudo color enhancement, deblurring, background suppression, and data enhancement processing on the infrared image data to obtain a preprocessed infrared image dataset.

4. The method for multi-source fusion target detection of unmanned aerial vehicle based on support vector machine according to claim 1 is characterized in that: The specific steps to establish the improved constrained energy minimization CEM filter include: Identify constraints; The design of FIR linear filtering algorithm is regarded as a constrained energy minimization problem; Based on FIR linear filter and constrained energy minimization CEM problem, an improved constrained energy minimization CEM filter is obtained.

5. The method for multi-source fusion target detection of unmanned aerial vehicles based on support vector machine according to claim 4 is characterized in that: The constraint condition is expressed as: Among them, d is the personnel prior spectrum vector, which refers to the typical spectrum feature vector of the person falling into the water obtained in advance through experiments, w is the filter weight coefficient, c is an arbitrary scalar, usually 1, d k is the vector of the kth band of the personnel prior spectrum, w k is the weight coefficient of the filter in the kth band, k = 1, 2, 3…l.

6. The method for multi-source fusion target detection of unmanned aerial vehicles based on support vector machines according to claim 4 is characterized in that: The constrained energy minimization problem is expressed as: Among them, w is the filter weight coefficient, y i is the output of the linear filter for the i-th pixel, i = 1, 2, 3…N, N is the total number of pixels in the hyperspectral image, X l×l is the autocorrelation matrix of the band, w k is the weight coefficient of the filter in the kth band, d is the personnel prior spectrum vector, d k is the direction of the kth band of the personnel's prior spectrum, k = 1, 2, 3…l, and l represents the total number of bands.

7. The method for multi-source fusion target detection of unmanned aerial vehicles based on support vector machines according to claim 1, characterized in that: The improved YOLOv8 model includes a drone inspection target detection head, added convolution, an attention fusion module CAFM, and an added sliding window module.

8. The method for multi-source fusion target detection of unmanned aerial vehicles based on support vector machines according to claim 7 is characterized in that: The attention fusion module CAFM includes a local branch and a global branch; The local branch is a neural network, including channel dimension adaptation, channel reorganization and local feature capture; The global branch is a Transformer architecture, including query, key and value generation, attention graph construction and global output calculation.

9. The method for multi-source fusion target detection of unmanned aerial vehicles based on support vector machine according to claim 1, characterized in that: The decision function is expressed as: Among them, SV is the set of support vectors, sign is the sign function, b is the bias term, and x is the combined feature; α n is the Lagrange multiplier of the nth support vector, y n is the category label of the nth support vector; K(.) is the kernel function.

10. The method for multi-source fusion target detection of unmanned aerial vehicles based on support vector machines according to claim 9, characterized in that: It also includes the constraints of the support vector machine SVM, which are expressed as: 0≤α n ≤C Among them, C is the penalty parameter, α n is the Lagrange multiplier of the nth support vector, y n is the category label of the nth support vector.

Images