Unmanned aerial vehicle multi-source fusion target detection method based on support vector machine

Abstract

This invention relates to a multi-source fusion target detection method for UAVs based on support vector machines, belonging to the field of image processing technology. First, an improved constrained energy minimization (CEM) filter, a finite impulse response linear filter, is designed. Preprocessed data is passed through this linear filter, and an adaptive threshold segmentation method is used to obtain hyperspectral personnel target information. Second, for RGB images and infrared data, an improved YOLOv8 model is used to obtain more local information, improving the detection accuracy of small personnel targets on the UAV. Then, convolution and attention fusion modules are added to provide a more comprehensive and effective feature extraction capability for the multi-source fusion target detection model, and a sliding window is used to enhance the detection capability of small personnel targets. Finally, intelligent inspection is performed based on a support vector machine classifier, fusing target information identified from hyperspectral, RGB, and infrared data, and outputting the multi-source fused target of interest (a person who has fallen into the water).

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and specifically to a multi-source fusion target detection method for unmanned aerial vehicles based on support vector machines. Background Technology

[0002] According to data from the World Health Organization (WHO), approximately 300,000 people die from drowning globally each year, with about 59,000 drowning deaths annually in my country, and minors accounting for over 95% of these cases. Rapid and accurate identification of drowning victims is crucial for rescue operations. Studies indicate that without rescue measures, drowning victims begin to show signs of brain death after about 5 minutes; therefore, timely detection within 5 minutes is particularly critical. Advances in drone technology, with its low cost, remote operation capabilities, and high mobility, have led to its application in numerous fields and are playing an increasingly important role in routine inspection tasks.

[0003] When using drones for water surface inspections, intelligent recognition technology can be used to monitor the water surface in real time and effectively capture the activity patterns of people in the surrounding area. This helps to detect and prevent potentially dangerous behaviors and reduce the occurrence of people falling into the water. Currently, most drone inspections use RGB and infrared images to monitor people falling into the water, but these two methods rely on the shape characteristics of people for detection, which can easily lead to false detections and missed detections.

[0004] Advances in remote sensing technology have led to increased attention being paid to hyperspectral images in various fields, such as spectral unmixing, segmentation, classification, and target detection. Hyperspectral images are rich in spectral information, and subtle differences can be used to identify small targets. Hyperspectral target detection identifies targets based on the spectral differences between the target and the background. Several classic algorithms have been proposed and applied, such as matched filtering (MF) to enhance the target spectrum and weaken the background. Spectral matched filtering (SMF) is based on the Gaussian assumption and is suitable for simple backgrounds, but its practicality is limited by the need to know the signal abundance for sub-pixel targets. Robey proposed adaptive matched filtering (AMF) based on SMF. These basic algorithms have poor detection accuracy in scenarios with complex spectral correlations, such as water and clothing spectral confusion, algae / plankton interference, and dynamic water surface reflection, as well as scenarios with dynamic wave interference, underwater debris interference, and abrupt changes in lighting conditions, where the target background is highly variable.

[0005] In existing technologies, although multiple sensors are fused at the feature level, the existing fusion strategies are all at the feature level. The fusion algorithms are computationally complex, difficult, and require high-end computer equipment, making it difficult to achieve real-time detection and unable to be directly applied to drones for real-time inspection of people who have fallen into the water. Summary of the Invention

[0006] In view of the above problems, this invention provides a multi-source fusion target detection method for UAVs based on support vector machines, realizing the complementary advantages of multiple sensor sources. First, this invention designs an improved constrained energy minimization (CEM) filter, a finite impulse response linear filter, to pass the preprocessed data through the linear filter, and then uses an adaptive threshold segmentation method to obtain hyperspectral personnel target information. Second, for RGB images and infrared data, an improved YOLOv8 model is used to obtain more local information, improving the detection accuracy of small personnel targets on UAVs. Next, a convolutional and attention fusion module (CAFM) is added to provide a more comprehensive and effective feature extraction capability for the UAV multi-source fusion target detection model, and a sliding window is used to improve the detection capability of small personnel targets. Finally, intelligent inspection is performed based on a support vector machine (SVM) classifier, fusing the target information identified from hyperspectral, RGB, and infrared data, and outputting the multi-source fused target of interest (PMI) in the water.

[0007] This invention provides a multi-source fusion target detection method for UAVs based on support vector machines, comprising:

[0008] S1. Establish a multi-source fusion target detection model for UAVs;

[0009] Based on the multi-source sensors carried by the drone, images of people falling into the water were collected under different lighting and temperature scenarios, resulting in hyperspectral image datasets, RGB image datasets, and infrared image datasets.

[0010] S2. Let t = 1. When t = 1, it represents the initial time.

[0011] S3. Input the hyperspectral image data at time t into preprocessing module A. t Normalization and noise reduction are performed to obtain clean hyperspectral image data at time t; the target of interest is extracted from the clean hyperspectral image data at time t.

[0012] S4. Preprocess the RGB image data and infrared image data at time t respectively to obtain the processed RGB image data and infrared image data at time t respectively;

[0013] S5. Input the target of interest from the hyperspectral image data at time t into the improved constrained energy minimization CEM filter B. t Output the detection response graph at time t;

[0014] The squared response map of the output at time t is calculated to obtain the squared enhanced response map at time t;

[0015] An unsupervised thresholding strategy is adopted to generate an adaptive threshold; based on the adaptive threshold, the background residual speckle noise of the squared enhanced response map at time t is eliminated to enhance the target of interest, thereby achieving the segmentation of the target of interest and obtaining the target of interest in the final hyperspectral image at time t.

[0016] S6. Input the RGB image data at time t and the infrared image data at time t into the improved YOLOv8 model C. t Output the target of interest in the RGB image at time t and the target of interest in the infrared image at time t, respectively;

[0017] S7. Combine the target of interest in the RGB image at time t, the target of interest in the infrared image at time t, and the target of interest in the final hyperspectral image at time t to obtain the combined feature at time t;

[0018] Input the combined features at time t into the support vector machine (SVM) D. t Decisions are made based on decision functions.

[0019] Obtain the bounding box coordinates of the target of interest at time t, which represents the position of the person who fell into the water at time t;

[0020] S8. Determine if t is greater than or equal to T. If yes, obtain the location of the target person who fell into the water at time t and the final UAV multi-source fusion target detection model. If no, let t = t + 1 and return to step S3.

[0021] S9. Perform UAV multi-source fusion target detection using the final UAV multi-source fusion target detection model.

[0022] Optionally, the specific steps for extracting the target of interest from the clean hyperspectral image data at time t include:

[0023] S301. Normalize the hyperspectral image dataset;

[0024] S302. Wavelet transform is used to denoise the normalized hyperspectral image dataset to obtain clean hyperspectral image data;

[0025] S303. Perform water-fall detection on clean hyperspectral image data to obtain the corresponding pixel region of the person who fell into the water;

[0026] Based on a random sampling strategy, multiple points of interest are extracted within the pixel region of the person who fell into the water, and the average value of these multiple points of interest is taken as the target of interest in the hyperspectral image data.

[0027] Optionally, the specific steps for obtaining the RGB image data processed at time t and the infrared image data processed at time t respectively include:

[0028] S401. Normalize the size of the RGB image dataset, crop it, correct its color, normalize its pixel values, augment its data, and denoise it to obtain the processed RGB image.

[0029] S402. Perform temperature normalization, histogram equalization, pseudo-color enhancement, deblurring, background suppression, and data augmentation on the infrared image data to obtain the preprocessed infrared image dataset.

[0030] Optionally, the specific steps for establishing the improved constrained energy minimization (CEM) filter include:

[0031] Define the constraints;

[0032] The design of the FIR linear filtering algorithm is regarded as a constrained energy minimization problem;

[0033] An improved constrained energy minimization (CEM) filter is obtained based on FIR linear filters and the constrained energy minimization (CEM) problem.

[0034] Optionally, the expression for the constraint condition is:

[0035]

[0036] Where d is the prior spectral vector of the person, referring to the typical spectral feature vector of the person who fell into the water, obtained in advance through experiments; w is the filter weight coefficient; c is an arbitrary scalar, usually 1; d k Let w be the vector of the k-th band of the prior spectrum of the personnel. k Let be the weighting coefficient of the filter in the k-th band, where k = 1, 2, 3…l.

[0037] Alternatively, the constrained energy minimization problem is expressed as:

[0038]

[0039] Where w is the filter weight coefficient, y i X represents the output of the linear filter for the i-th pixel, where i = 1, 2, 3…N, and N is the total number of pixels in the hyperspectral image. l×l Let w be the autocorrelation matrix of the band. k Let d be the weighting coefficient of the filter in the k-th band, and d be the prior spectral vector of the person. k Let be the vector of the k-th band of the prior spectrum of the personnel, where k = 1, 2, 3…l, and l represents the total number of bands.

[0040] Optionally, an improved YOLOv8 model can be obtained by adding a drone inspection target detection head, a convolutional and attention fusion module (CAFM), and a sliding window module to the YOLOv8 model.

[0041] Optionally, the attention fusion module CAFM includes local branches and global branches;

[0042] The local branch is a neural network, including channel dimension adaptation, channel recombination, and local feature capture;

[0043] The global branch is a Transformer architecture, which includes querying, key and value generation, attention graph construction, and global output computation.

[0044] Optionally, the decision function is expressed as:

[0045]

[0046] Where SV is the set of support vectors, sign is the sign function, returning +1 or -1, b is the bias term used to adjust the position of the classification hyperplane, and x is the combined feature; α n Let y be the Lagrange multiplier of the nth support vector. n Let K(.) be the class label of the nth support vector; K(.) is the kernel function.

[0047] Optionally, the constraint conditions of the Support Vector Machine (SVM) are expressed as follows:

[0048]

[0049] 0≤α n ≤C

[0050] Where C is the penalty parameter, α n Let y be the Lagrange multiplier of the nth support vector. n Let be the category label of the nth support vector.

[0051] The small targets in this invention include (1) the head or arm of a person who has fallen into the water: when shooting from a distance, the target may only appear as a dot-like area of ​​a few pixels, which cannot be effectively captured by traditional shape detection. (2) partially submerged targets: such as a hand or clothing fragment that is only exposed above the water surface, with irregular shape and spectral characteristics similar to the background.

[0052] This invention uses the YOLO algorithm for visible light and infrared detection. The YOLO algorithm can identify dozens of frames of images per second, and the processing of hyperspectral images also uses a supervised algorithm at the second level. Combined with the inspection speed of the UAV at 10-20m / s and the algorithm processing delay (which can be optimized to the second level), the closed loop from data acquisition to alarm can be completed within 10 seconds to 2 minutes, meeting the 5-minute rescue window.

[0053] This invention achieves complementary advantages in spectral and spatial resolution, dynamic environmental adaptability, and data fusion to enhance detection confidence. Specifically, it utilizes a hyperspectral sensor to provide continuous spectral information, enabling the identification of target substances (such as the material of clothing or the spectral characteristics of skin of a person who has fallen into the water) through subtle spectral differences. A visible light RGB sensor provides high spatial resolution images, capturing the shape, texture, and movement trajectory of the target to aid in locating the specific position of the person who has fallen into the water. An infrared sensor detects targets based on differences in thermal radiation, allowing operation at night or in low-light conditions. It rapidly locates targets by exploiting the temperature difference between the human body and water (e.g., human body temperature is approximately 36°C, while water temperature may be below 20°C).

[0054] This invention uses an SVM classifier to integrate multi-source detection results and reduces misjudgments from a single sensor through weighted voting. Specifically, when hyperspectral imaging determines the presence of human spectra in a certain area but infrared imaging does not detect any temperature anomalies, secondary verification can be triggered to improve detection confidence.

[0055] Compared with the prior art, the present invention has at least the following beneficial effects:

[0056] (1) This invention achieves complementary advantages in spectral and spatial resolution, dynamic environmental adaptability, and data fusion to improve detection confidence.

[0057] (2) This invention uses an SVM classifier to integrate multi-source detection results and reduces misjudgments from a single sensor through weighted voting;

[0058] (3) The real-time detection time interval of the UAV multi-source fusion target detection model of the present invention is on the order of seconds;

[0059] (4) Based on the improved YOLOv8 model, this invention obtains more local information from RGB images and infrared data, thereby improving the detection accuracy of small targets such as personnel on UAVs;

[0060] (5) The improved YOLOv8 model adds a convolutional and attention fusion module (CAFM) to provide more comprehensive and effective feature extraction capabilities for the UAV multi-source fusion target detection model, and uses a sliding window to improve the detection capability of small human targets. Attached Figure Description

[0061] The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of the invention.

[0062] Figure 1 This is a schematic diagram of UAV multi-source fusion target detection based on support vector machine in an embodiment of the present invention;

[0063] Figure 2 This is a schematic diagram of the process for acquiring hyperspectral data of interest in an embodiment of the present invention;

[0064] Figure 3 This is a schematic diagram of the process of acquiring RGB and infrared data of the target of interest in an embodiment of the present invention;

[0065] Figure 4 This is a schematic diagram showing the addition of a fine detection head structure in an embodiment of the present invention;

[0066] Figure 5 This is a schematic diagram of the CAFM structure with the addition of a convolution and attention fusion module in an embodiment of the present invention. Detailed Implementation

[0067] To better understand the above-described objectives, features, and advantages of the present invention, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other. Furthermore, the present invention can be implemented in other ways different from those described herein; therefore, the scope of protection of the present invention is not limited to the specific embodiments disclosed below.

[0068] A specific embodiment of the present invention, such as Figure 1-5 This paper discloses a multi-source fusion target detection method for UAVs based on support vector machines. The specific implementation steps are as follows:

[0069] S1. Establish a multi-source fusion target detection model for UAVs, including: a preprocessing module, an improved constrained energy minimization (CEM) filter, an improved YOLOv8 model, and a support vector machine (SVM);

[0070] Based on the multi-source sensors carried by the drone, images of people falling into the water were collected under different lighting and temperature scenarios, resulting in hyperspectral image datasets, RGB image datasets, and infrared image datasets.

[0071] Optionally, the multi-source sensor data includes: a hyperspectral sensor, an RGB sensor, and an infrared sensor;

[0072] The different lighting and temperature scenarios include different lighting and temperature scenarios in the morning, noon, and evening;

[0073] S2. Let t = 1. When t = 1, it represents the initial time.

[0074] S3. Input the hyperspectral image data at time t into preprocessing module A. t Normalization and noise reduction are performed to obtain clean hyperspectral image data at time t; the target of interest is extracted from the clean hyperspectral image data at time t.

[0075] Optionally, the specific steps for extracting the target of interest from the clean hyperspectral image data at time t include:

[0076] S301. Normalize the hyperspectral image dataset using the following expression:

[0077]

[0078] Where X0 is the original data and X is the normalized data.

[0079] S302. Wavelet transform is used to denoise the normalized hyperspectral image dataset to obtain clean hyperspectral image data;

[0080] Furthermore, the specific steps to obtain clean hyperspectral image data include:

[0081] Hyperspectral data is decomposed into data in multiple bands, and the data in each band is decomposed into approximation coefficients and detail coefficients through wavelet transform; the approximation coefficients are the low-frequency part, and the detail coefficients are the high-frequency part.

[0082] Noise is removed from the approximation and detail coefficients of each band of data by using a threshold parameter, resulting in denoised hyperspectral data, which is characterized as clean hyperspectral image data. The expression is:

[0083]

[0084] Where α is the scale factor and is greater than 0, τ reflects the displacement, t is the duration, and WT x (α,τ) are wavelet coefficients, whose values ​​represent high-frequency and low-frequency signals, respectively, and x(t) is the spectral signal at time t, i.e., the spectral data of the hyperspectral image. For the mother wavelet function.

[0085] It is understood that the approximation coefficients contain the main information of the spectral signal, representing the background or overall trend of the image; during the denoising process, the approximation coefficients are used to reconstruct the denoised image, preserving the original structure and spectral characteristics of the image.

[0086] The detail coefficient contains both image details and noise, reflecting local variations in the spectral signal. During noise reduction, thresholding or other filtering methods can be used to reduce the impact of noise while preserving useful detail information as much as possible.

[0087] S303. Perform water-fall detection on clean hyperspectral image data to obtain the corresponding pixel region of the person who fell into the water;

[0088] Based on a random sampling strategy, multiple points of interest are extracted within the pixel region of the person who fell into the water, and the average value of these multiple points of interest is taken as the target of interest in the hyperspectral image data.

[0089] For example, the features corresponding to the point of interest include the spectral information of the pixel, that is, the reflectance information of the pixel at various wavelengths, and the reflectance forms a spectral reflectance curve in continuous bands.

[0090] S4. Preprocess the RGB image data and infrared image data at time t respectively to obtain the processed RGB image data and infrared image data at time t respectively;

[0091] Optionally, the specific steps for obtaining the RGB image data processed at time t and the infrared image data processed at time t respectively include:

[0092] S401. Normalize the size of the RGB image dataset, crop it, correct its color, normalize its pixel values, augment its data, and denoise it to obtain the processed RGB image.

[0093] In this invention, RGB images are normalized to ensure that all input images have the same size, enabling the model to accept fixed-size inputs and simplifying network design and computation.

[0094] The cropping process involves removing irrelevant parts of the RGB image, focusing on the region of interest, and improving the model's attention to and recognition accuracy for specific targets.

[0095] The color correction includes adjusting the brightness, contrast, saturation, etc. of the image to reduce the impact of different lighting conditions and make the model more robust to changes in lighting.

[0096] The pixel value normalization includes scaling pixel values ​​to a specific range to accelerate convergence during model training and prevent gradient vanishing or exploding.

[0097] The data augmentation includes increasing the diversity of data through methods such as rotation, flipping, scaling, cropping, and adding noise, thereby improving the generalization ability of the model and reducing overfitting;

[0098] The denoising includes reducing noise in the image to improve image quality, enhance the model's robustness to noise, and improve recognition accuracy.

[0099] S402. Perform temperature normalization, histogram equalization, pseudo-color enhancement, deblurring, background suppression, and data augmentation on the infrared image data to obtain the preprocessed infrared image dataset.

[0100] Optionally, the temperature normalization includes scaling the representative temperature pixel values ​​of the infrared image to a uniform thermal radiation range, enabling the model to process images with different temperature ranges and improving the consistency of recognition.

[0101] The histogram equalization includes adjusting the histogram of the infrared image to make the distribution of pixel values ​​more uniform, enhance the contrast of the image, and improve the visibility and recognition effect of the infrared image.

[0102] The pseudo-color enhancement involves converting the grayscale image of the infrared image into a pseudo-color image to highlight specific temperature regions, improve the visual interpretability of the infrared image, and help the model better identify targets.

[0103] The deblurring includes reducing the blurring of infrared images caused by motion or atmospheric turbulence, improving the clarity of infrared images, and enhancing the identifiability of targets.

[0104] The background suppression includes reducing or eliminating background interference in infrared images, highlighting the target, and improving the target detection and recognition rate;

[0105] The data augmentation includes increasing the diversity of data through methods such as rotation and flipping, improving the model's generalization ability, and reducing overfitting.

[0106] In this invention, infrared image data is subjected to temperature normalization, histogram equalization, pseudo-color enhancement, deblurring, background suppression, and data enhancement processing to meet the requirements of detection accuracy and real-time performance.

[0107] S5. Input the target of interest from the hyperspectral image data at time t into the improved constrained energy minimization CEM filter B. t Output the detection response graph at time t;

[0108] The squared response map of the output at time t is calculated to obtain the squared enhanced response map at time t;

[0109] An unsupervised thresholding strategy is adopted to generate an adaptive threshold; based on the adaptive threshold, the background residual speckle noise of the squared enhanced response map at time t is eliminated to enhance the target of interest, thereby achieving the segmentation of the target of interest and obtaining the target of interest in the final hyperspectral image at time t.

[0110] Optionally, the squared-enhanced response map expression is:

[0111]

[0112] Where D is the result after squaring, D CEM This is the detection response diagram of the CEM filter.

[0113] Optionally, the expression for the inter-class variance g is:

[0114] g = w1 × (u1 - u) 2 +w2×(u2-u) 2

[0115] Where u is the total average brightness value of the image, w1 is the percentage of target pixels, u1 is the average brightness value of target pixels, w2 is the percentage of background pixels, and u2 is the average brightness value of background pixels.

[0116] Furthermore, the expression for the overall average brightness value of the image is:

[0117] u = w1 × u1 + w2 × u2

[0118] Where u is the average brightness value of the image, w1 is the percentage of target pixels, u1 is the average brightness value of the target pixels, w2 is the percentage of background pixels, and u2 is the average brightness value of the background pixels.

[0119] When the inter-class variance g is maximized, the corresponding threshold is the adaptive segmentation threshold.

[0120]

[0121] Where T is the adaptive threshold, DN max This represents the maximum brightness level of the hyperspectral image.

[0122] Optionally, the specific steps for establishing the improved constrained energy minimization (CEM) filter include:

[0123] The constraints are defined as follows:

[0124]

[0125] Where d is the prior spectral vector of the person, referring to the typical spectral feature vector of the person who fell into the water, obtained in advance through experiments; w is the filter output vector; c is an arbitrary scalar, usually 1; and d k Let w be the vector of the k-th band of the prior spectrum of the personnel. k Let be the weighting coefficient of the filter in the k-th band, where k = 1, 2, 3…l.

[0126] The design of the FIR linear filtering algorithm can be viewed as a constrained energy minimization problem, expressed as:

[0127]

[0128] Where w is the filter weight coefficient, y i X represents the output of the linear filter for the i-th pixel, where i = 1, 2, 3…N, and N is the total number of pixels in the hyperspectral image. l×l Let w be the autocorrelation matrix of the band. k Let d be the weighting coefficient of the filter in the k-th band, and d be the prior spectral vector of the person. k Let be the vector of the k-th band of the prior spectrum of the personnel, where k = 1, 2, 3…l, and l represents the total number of bands.

[0129] An improved constrained energy minimization (CEM) filter is obtained based on FIR linear filters and the constrained energy minimization (CEM) problem.

[0130] S6. Input the RGB image data at time t and the infrared image data at time t into the improved YOLOv8 model C. t Output the target of interest in the RGB image at time t and the target of interest in the infrared image at time t, respectively;

[0131] Optionally, an improved YOLOv8 model can be obtained by adding a drone inspection target detection head, a convolutional and attention fusion module (CAFM), and a sliding window module to the YOLOv8 model.

[0132] Optionally, the specific steps for outputting the target of interest in the RGB image and the target of interest in the infrared image include:

[0133] The RGB image data at time t is input into the improved YOLOv8 model B. t Based on the attention fusion module, multi-scale fusion features are output;

[0134] The multi-scale fused features are input into the CBS-Concat-C2F module, and the output is an RGB image.

[0135] The RGB image is input into the sliding window processing module, which outputs the adjusted image features, i.e., the target of interest in the RGB image.

[0136] This invention adds a drone-based target detection head to the YOLOv8 model to enhance the ability to identify small targets, such as... Figure 4 As shown;

[0137] This invention adds a convolutional and attention fusion module, CAFM, to improve the detection performance of small targets by drone inspectors, such as... Figure 5 As shown;

[0138] Specifically, an upsample-concat-C2F layer is added after the attention head output of the 15th layer of the YOLOv8 model for the output of the 160×160 head of the small human target branch. Then, two convolution-normalization-SiLU activation layers (CBS-Concat-C2F) are used to first transform the 160×160 convolution into 80×80, and then the 80×80 head is output in the Contact layer of the 15th layer.

[0139] The small human targets include people who have fallen into the water at a distance, with low resolution, or partially obscured. Specifically, these include: (1) the head or arms of the person who has fallen into the water; and (2) partially submerged targets.

[0140] Optionally, the attention fusion module CAFM integrates the complementary characteristics of convolutional neural networks (CNNs) and the Transformer architecture, fusing the local features extracted by the convolution operation in the CNN with the global features extracted by the attention mechanism in the Transformer architecture, thereby achieving efficient representation of image details and overall structure, and thus enhancing the performance of object detection.

[0141] Understandably, the details include: the outline of the person who fell into the water, the texture of their clothing, their hair, and the temperature difference between the human body and the water in the infrared image.

[0142] The attention fusion module CAFM includes local branches and global branches;

[0143] The local branch is a neural network, including channel dimension adaptation, channel recombination, and local feature capture;

[0144] The global branch is a Transformer architecture, including querying, key and value generation, attention graph construction, and global output calculation;

[0145] The channel dimension adaptation achieves flexible configuration of the number of channels through convolution, while keeping the feature map size unchanged, in order to optimize subsequent feature processing.

[0146] The channel reorganization process involves dividing the input tensor by channel and applying depthwise separable convolution to promote channel shuffling within the group, thereby splicing together to generate new tensors and enhancing information fusion between channels.

[0147] The local feature capture is used to comprehensively extract features in both spatial and spectral dimensions to improve characterization capabilities.

[0148] The query, key, and value generation includes generating query (Q), key (K), and value (V) tensors using convolution and depth-width convolution, with varying forms.

[0149] Attention graph construction: After the query Q tensor and the key K tensor are reshaped, the attention graph is determined through their interaction, which reduces the computational burden.

[0150] Optionally, the sliding window has a window size of 640×640, and the original image is slidably segmented to obtain multiple slices;

[0151] Set an overlap area of ​​no less than 20% between adjacent slices;

[0152] Set a specified overlap ratio standard IOU threshold. When the overlap ratio between the target box of the sliced ​​image and the target box on the original image is greater than the IOU threshold, save the target box information of the sliced ​​image. If it is less than the threshold, remove it to obtain the filtered sliced ​​image.

[0153] The position coordinates of the filtered slices are globally transformed to obtain the adjusted image;

[0154] S7. Combine the target of interest in the RGB image at time t, the target of interest in the infrared image at time t, and the target of interest in the final hyperspectral image at time t to obtain the combined feature at time t;

[0155] Input the combined features at time t into the support vector machine SVMD t Decisions are made based on decision functions.

[0156] Obtain the bounding box coordinates of the target of interest at time t, which represents the position of the person who fell into the water at time t;

[0157] S8. Determine if t is greater than or equal to T. If yes, obtain the location of the target person who fell into the water at time t and the final UAV multi-source fusion target detection model. If no, let t = t + 1 and return to step S3.

[0158] S9. Utilize the final UAV multi-source fusion target detection model to perform UAV multi-source fusion target detection.

[0159] The decision function is expressed as follows:

[0160]

[0161] Where SV is the set of support vectors, sign is the sign function, returning +1 or -1, b is the bias term used to adjust the position of the classification hyperplane, and x is the combined feature; α n Let y be the Lagrange multiplier of the nth support vector. n Let K(.) be the class label of the nth support vector; K(.) is the kernel function.

[0162] Optionally, the constraint conditions of the Support Vector Machine (SVM) are expressed as follows:

[0163]

[0164] 0≤α n ≤C

[0165] Where C is the penalty parameter.

[0166] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A multi-source fusion target detection method for unmanned aerial vehicles (UAVs) based on support vector machines, characterized in that, include: S1. Establish a multi-source fusion target detection model for UAVs; Images of people who fell into the water were collected, resulting in hyperspectral image datasets, RGB image datasets, and infrared image datasets. S2. Let t = 1. When t = 1, it represents the initial time. S3. Input the hyperspectral image data at time t into preprocessing module A. t Obtain the target of interest from clean hyperspectral image data at time t; S4. Obtain the RGB image data and infrared image data processed at time t respectively; S5. Input the target of interest from the hyperspectral image data at time t into the improved constrained energy minimization CEM filter B. t Based on an unsupervised threshold calculation strategy, the target of interest is segmented to obtain the final hyperspectral image of the target of interest at time t. S6. Input the RGB image data at time t and the infrared image data at time t into the improved YOLOv8 model C. t Output the target of interest in the RGB image at time t and the target of interest in the infrared image at time t, respectively; S7. Combine the target of interest in the RGB image at time t, the target of interest in the infrared image at time t, and the target of interest in the final hyperspectral image at time t to obtain the combined feature at time t; Input the combined features at time t into the support vector machine D. t Based on the decision function, the location of the target person who fell into the water at time t is obtained; S8. Determine if t is greater than or equal to T. If yes, obtain the location of the target person who fell into the water at time t and the final UAV multi-source fusion target detection model. If no, let t = t + 1 and return to step S3. S9. Perform UAV multi-source fusion target detection using the final UAV multi-source fusion target detection model.

2. The UAV multi-source fusion target detection method based on support vector machine according to claim 1, characterized in that, Step S3 also includes specific steps for extracting the target of interest from the clean hyperspectral image data at time t, including: S301. Normalize the hyperspectral image dataset; S302. Wavelet transform is used to denoise the normalized hyperspectral image dataset to obtain clean hyperspectral image data; S303. Perform water-fall detection on clean hyperspectral image data to obtain the corresponding pixel region of the person who fell into the water; Based on a random sampling strategy, multiple points of interest are extracted within the pixel region of the person who fell into the water, and the average value of these multiple points of interest is taken as the target of interest in the hyperspectral image data.

3. The UAV multi-source fusion target detection method based on support vector machine according to claim 1, characterized in that, The specific steps for obtaining the RGB image data processed at time t and the infrared image data processed at time t respectively, as described in step S4, include: S401. Normalize the size of the RGB image dataset, crop it, correct its color, normalize its pixel values, augment its data, and denoise it to obtain the processed RGB image. S402. Perform temperature normalization, histogram equalization, pseudo-color enhancement, deblurring, background suppression, and data augmentation on the infrared image data to obtain the preprocessed infrared image dataset.

4. The UAV multi-source fusion target detection method based on support vector machine according to claim 1, characterized in that, The specific steps for establishing an improved constrained energy minimization (CEM) filter include: Define the constraints; The design of the FIR linear filtering algorithm is regarded as a constrained energy minimization problem; An improved constrained energy minimization (CEM) filter is obtained based on FIR linear filters and the constrained energy minimization (CEM) problem.

5. The UAV multi-source fusion target detection method based on support vector machine according to claim 4, characterized in that, The expression for the constraint condition is: Where d is the prior spectral vector of the person, referring to the typical spectral feature vector of the person who fell into the water, obtained in advance through experiments; w is the filter weight coefficient; c is an arbitrary scalar, usually 1; d k Let w be the vector of the k-th band of the prior spectrum of the personnel. k Let be the weighting coefficient of the filter in the k-th band, where k = 1, 2, 3…l.

6. The UAV multi-source fusion target detection method based on support vector machine according to claim 4, characterized in that, The constrained energy minimization problem is expressed as: Where w is the filter weight coefficient, y i X represents the output of the linear filter for the i-th pixel, where i = 1, 2, 3…N, and N is the total number of pixels in the hyperspectral image. l×l Let w be the autocorrelation matrix of the band. k Let d be the weighting coefficient of the filter in the k-th band, and d be the prior spectral vector of the person. k Let k be the direction of the k-th band of the prior spectrum of the personnel, where k = 1, 2, 3...l, and l represents the total number of bands.

7. The UAV multi-source fusion target detection method based on support vector machine according to claim 1, characterized in that, The improved YOLOv8 model includes a drone inspection target detection head, the addition of convolution and attention fusion modules (CAFM), and the addition of a sliding window module.

8. The UAV multi-source fusion target detection method based on support vector machine according to claim 7, characterized in that, The attention fusion module CAFM includes local branches and global branches; The local branch is a neural network, including channel dimension adaptation, channel recombination, and local feature capture; The global branch is a Transformer architecture, which includes querying, key and value generation, attention graph construction, and global output computation.

9. The UAV multi-source fusion target detection method based on support vector machine according to claim 1, characterized in that, The decision function is expressed as follows: Where SV is the set of support vectors, sign is the sign function, b is the bias term, and x is the combined feature; α n Let y be the Lagrange multiplier of the nth support vector. n Let K(.) be the class label of the nth support vector; K(.) is the kernel function.

10. The UAV multi-source fusion target detection method based on support vector machine according to claim 9, characterized in that, It also includes the constraints for the Support Vector Machine (SVM), expressed as: 0≤α n ≤C Where C is the penalty parameter, α n Let y be the Lagrange multiplier of the nth support vector. n Let be the category label of the nth support vector.

Images