Low, slow and small target monitoring method, medium, equipment and system

Abstract

The invention discloses a low, slow and small target monitoring method, medium, equipment and system. The invention discloses a multispectral low-slow small target monitoring method. The method comprises the following steps: acquiring a multispectral image comprising a plurality of spectral bands; inputting the multispectral image into a pre-trained target detection model to extract a position, a spectral feature vector, a texture feature vector and a shape feature vector of the low-slow small target in the multispectral image; the target detection model is constructed based on a YOLO model; taking the fusion feature vector as the input of a pre-trained target classifier, and obtaining the classification result of the low-slow small target output by the target classifier; the fusion feature vector fuses a spectral feature vector, a texture feature vector and a shape feature vector; and giving out an early warning signal in response to the condition that the position or the classification result of the low-slow small target meets the early warning condition. Through the above mode, the method can improve the detection accuracy of the position, shape, category and other information of the low, slow and small target, and achieves the efficient and precise monitoring of the low, slow and small target.

Description

Technical Field

[0001] The present application relates to the field of sensor recognition, and in particular to a method, medium, device and system for monitoring low, slow and small targets. Background Art

[0002] Traditional means of monitoring low-speed, slow, and small targets mainly include radar monitoring and radio monitoring. When radar monitors low-speed, slow, and small targets, due to the small radar scattering cross-section of the target, slow flight speed, and low altitude, it is easily interfered by ground clutter, multipath effects, etc., resulting in low detection accuracy and high false alarm rate. In addition, the detection effect is poor for some low-speed, slow, and small targets that use special materials to reduce radar reflectivity. Radio monitoring is mainly aimed at low-speed, slow, and small targets that use specific frequency bands, such as drones. Its monitoring range is limited, and when the target uses anti-interference technologies such as frequency hopping, the difficulty of monitoring increases significantly.

[0003] Therefore, in the prior art, whether it is radar or a single or simple combination of spectral monitoring methods, it is difficult to achieve high-precision positioning and identification of low, slow and small targets, and it is impossible to accurately determine the boundary, shape, model and other detailed information of the target, resulting in the inability to take accurate and effective measures when dealing with the threat of low, slow and small targets. On the other hand, visible light imaging is greatly affected by light and weather, and radar is greatly disturbed by the ground environment, which makes the monitoring performance of the prior art unstable under different environmental conditions, and it is difficult to meet the needs of continuous and reliable monitoring of low, slow and small targets in complex and changing environments. Due to insufficient feature extraction and limitations of data processing methods, the prior art is prone to misjudgment when distinguishing different types of low, slow and small targets, such as different models of drones or drones and birds, which greatly reduces the accuracy and reliability of the early warning. Summary of the invention

[0004] The present application mainly provides a low, slow and small target monitoring method, medium, equipment and system to solve the problem that low, slow and small targets cannot be accurately monitored.

[0005] To solve the above technical problems, a technical solution adopted in the present application is: to provide a multi-spectral low, slow and small target monitoring method, including: obtaining a multi-spectral image including multiple spectral bands; inputting the multi-spectral image into a pre-trained target detection model to extract the position, spectral feature vector, texture feature vector and shape feature vector of the low, slow and small target in the multi-spectral image; the low, slow and small target detection model is constructed based on the YOLO model; using the fused feature vector as the input of the pre-trained target classifier to obtain the classification result of the low, slow and small target output by the target classifier; the fused feature vector fuses the spectral feature vector, the texture feature vector and the shape feature vector; in response to the position of the low, slow and small target or the classification result meeting the warning condition, issuing a warning signal.

[0006] In some embodiments, the multispectral image is input into a pre-trained target detection model to extract the spectral feature vector of the low, slow and small target in the multispectral image, including: calculating the change law of the reflectivity curve of the low, slow and small target in each spectral band based on the multispectral image to extract the spectral feature parameters of the low, slow and small target; the spectral feature parameters include spectral peak position, spectral bandwidth, spectral slope, reflectivity ratio characteristics between different spectral bands and spectral absorption valley depth characteristics; the spectral feature parameters are subjected to dimensionality reduction processing to obtain the spectral feature vector.

[0007] In some embodiments, the multispectral image is input into a pre-trained target detection model to extract texture feature vectors of low, slow and small targets in the multispectral image, including: respectively calculating the grayscale co-occurrence matrices of images of different spectral bands at multiple scales and multiple directions to extract texture feature parameters of the low, slow and small targets; the texture feature parameters include contrast, correlation, entropy and inverse difference; performing multi-resolution decomposition of the texture feature parameters through wavelet transform to obtain texture information of different frequency sub-bands to obtain the multi-scale texture feature vectors.

[0008] In some embodiments, the multispectral image is input into a pre-trained target detection model to extract the shape feature vector of the low, slow and small targets in the multispectral image, including: detecting the edges of the low, slow and small targets in the multispectral image through an edge detection operator; refining and connecting the edges through morphological operations to form a complete and continuous edge contour; calculating the shape feature parameters of the low, slow and small targets based on the edge contour; the shape feature parameters include perimeter, area, circularity, rectangularity, and convex hull area ratio; and constructing the shape feature vector based on the shape feature parameters.

[0009] In some embodiments, before inputting the multispectral image into a pre-trained target detection model, it also includes: training the improved target detection model with a multispectral image set that annotates the target position, category, and multispectral feature information to improve the target detection model's ability to extract and locate features of the low, slow, and small targets.

[0010] In some embodiments, the acquiring of a multispectral image including multiple spectral bands includes: synchronously acquiring the multispectral image in all directions through a sensor array of multiple spectral bands; the spectral bands include visible light, near infrared, medium-wave infrared and long-wave infrared; and performing noise removal, radiation correction and geometric correction on the multispectral image.

[0011] In some embodiments, the noise removal of the multispectral image includes: performing median filtering on the salt and pepper noise of visible light in the multispectral image, performing Gaussian filtering on the Gaussian noise of near infrared, medium-wave infrared and long-wave infrared in the multispectral image, and dynamically optimizing the filtering parameters of the median filtering and the Gaussian filtering according to the local characteristics of the multispectral image.

[0012] In some embodiments, the radiation correction of the multispectral image includes: periodically calibrating the plurality of spectral band sensors through a blackbody radiation source to obtain radiation response characteristics of the plurality of spectral band sensors under different environmental conditions; constructing a radiation correction model based on the radiation response characteristics; and inputting the environmental parameters obtained in real time into the radiation correction model to perform radiation correction on the multispectral image.

[0013] In some embodiments, the geometric correction of the multispectral image includes: obtaining preset feature points in space and the spatial coordinates corresponding to the feature points; identifying the feature points in the multispectral image, and calculating the image coordinates of the feature points in the multispectral image; based on the image coordinates and the spatial coordinates of the feature points, constructing a mapping relationship between an image coordinate system and a spatial coordinate system by a least squares method to perform geometric correction on the multispectral image.

[0014] In order to solve the above technical problems, another technical solution adopted by the present application is: providing a storage medium on which program data is stored, and when the program data is executed by a processor, the steps of the low, slow and small target monitoring method as mentioned above are implemented.

[0015] The present application also provides a computer device, including a processor and a memory connected to each other, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the low, slow and small target monitoring method as described above are implemented.

[0016] The present application also provides a multi-spectral low-speed, slow, and small target monitoring system, comprising multiple spectral band sensor arrays and a computer device as described above; the multiple spectral band sensor arrays are used to collect multi-spectral images of visible light, near infrared, medium-wave infrared, and long-wave infrared in all directions, and send them to the computer device; the computer device synchronously receives the multi-spectral images sent by the multiple spectral band sensor arrays to execute the low-speed, slow, and small target monitoring method as described above.

[0017] The beneficial effects of the present application are as follows: Different from the prior art, the present application discloses a method, medium, device and system for monitoring low, slow and small targets. A multispectral image including multiple spectral bands is obtained to ensure full coverage of the monitoring area at different spatial positions and angles, thereby reducing monitoring blind spots. The target is a target with a small radar scattering cross-sectional area, a slow flight speed and a low altitude. The multispectral image is input into a pre-trained target detection model to extract the position, spectral feature vector, texture feature vector and shape feature vector of the low, slow and small target in the multispectral image; wherein the target detection model is constructed based on the YOLO model, so that the YOLO algorithm network can better learn and extract target features in the multispectral image, and improve the detection ability of the YOLO algorithm network for low, slow and small targets. The fused feature vector is used as the input of the pre-trained target classifier to obtain the classification result of the low, slow and small target output by the target classifier; the fused feature vector fuses the spectral feature vector, the texture feature vector and the shape feature vector to form a comprehensive and unique description of the target, thereby improving the accuracy and reliability of target recognition. At the same time, in response to the position or classification result of the low, slow and small target meeting the warning condition, a warning signal is issued. Target detection and positioning are performed through a target detection model based on deep learning and the YOLO algorithm network. Target classification and identification are performed by combining the advantages of the target classifier in small sample classification, achieving efficient and accurate low, slow and small target monitoring, and issuing early warnings for dangerous or suspicious targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work, among which: Figure 1 This is a flow chart of an embodiment of a multi-spectral low-speed, slow, small target monitoring method provided by the present application; Figure 2 yes Figure 1 The flowchart of the embodiment of step 10 of the method shown is as follows; Figure 3 yes Figure 2 The flowchart of the embodiment of step 12 of the method shown; Figure 4 yes Figure 2 The flowchart of another embodiment of step 12 of the method is shown; Figure 5 yes Figure 1 The flowchart of the embodiment of step 20 of the method is shown; Figure 6 yes Figure 1The flowchart of another embodiment of method step 20 is shown; Figure 7 yes Figure 1 The flowchart of another embodiment of method step 20 is shown; Figure 8 It is a structural schematic diagram of an embodiment of a storage medium provided by the present application; Fig. 9 is a structural schematic diagram of an embodiment of a computer device provided by the present application; Fig.10 It is a structural schematic diagram of an embodiment of a multi-spectral low-speed, slow, small target monitoring system provided in the present application. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0020] The terms "first", "second", "third" in the embodiments of the present application are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined as "first", "second", "third" can expressly or implicitly include at least one of the features. In the description of the present application, the meaning of "multiple" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices.

[0021] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0022] See also Figure 1 , Figure 1 : is a flow chart of an embodiment of a multi-spectral low-speed, slow, and small target monitoring method provided by the present application, and the low-speed, slow, and small target monitoring method comprises: Step 10: Acquire a multispectral image including multiple spectral bands.

[0023] Multispectral imaging is a method for acquiring and analyzing image data from different spectral bands. Unlike traditional color images, multispectral imaging can capture information in a wider spectral range, including bands other than visible light, such as infrared light. This technology can achieve the acquisition of all-round, multispectral images through the synchronous acquisition of multiple spectral band sensor arrays.

[0024] Optionally, the sampling frequency, resolution and other parameters of the sensor can be flexibly adjusted according to the size and accuracy requirements of the monitoring scene. For example, when monitoring the clear area of ​​a large airport, a relatively low resolution but high sampling frequency is used to cover a wide area and capture the dynamics of the target in a timely manner; in a small-scale high-precision monitoring scene such as an important event venue, the resolution is increased and the sampling frequency is appropriately reduced to obtain more detailed features of the target.

[0025] See also Figure 2 , step 10 further comprises: Step 11: Synchronously collect a full range of multispectral images through a sensor array of multiple spectral bands, including visible light, near infrared, mid-wave infrared, and long-wave infrared.

[0026] The array of sensors with multiple spectral bands is designed so that the multispectral image acquisition camera covers multiple different spectral bands, including visible light (400-700nm), near infrared (700-1100nm), medium-wave infrared (3-5μm), long-wave infrared (8-14μm), etc. The layout of sensors of each band in the array is optimized to ensure full coverage of the monitoring area at different spatial positions and angles, reducing monitoring blind spots.

[0027] Specifically, the spatial resolution of the sensor is determined according to the application requirements, and the size and shape of the sensor array are designed to ensure full coverage of the monitoring area. The field of view and overlapping area of ​​the sensor are adjusted to reduce blind spots and ensure that sensors of different bands do not interfere with each other in the array.

[0028] A high-precision sensor synchronization control module is equipped at the receiving end of the multispectral image to ensure the synchronous acquisition of data in each spectral band and avoid data mismatch problems caused by time difference.

[0029] Step 12: Perform noise removal, radiation correction and geometric correction on the multispectral image.

[0030] By preprocessing multispectral images such as noise removal, radiation correction and geometric correction, the high quality and accuracy of the data are guaranteed, providing a reliable foundation for subsequent processing.

[0031] Among them, the noise removal of multispectral images includes: The salt-and-pepper noise of visible light in multispectral images is subjected to median filtering, and the Gaussian noise of near-infrared, medium-wave infrared and long-wave infrared in multispectral images is subjected to Gaussian filtering. The filtering parameters of median filtering and Gaussian filtering are dynamically optimized according to the local characteristics of the multispectral images.

[0032] According to the noise characteristics of data in different spectral bands, appropriate filtering algorithms are used respectively: For the visible light band that is easily affected by salt and pepper noise, median filtering is used for preliminary processing; median filtering removes isolated noise points by replacing pixel values ​​with the median of the pixel values ​​in its neighborhood. At the same time, the size or shape of the filter window is dynamically adjusted by analyzing the local features of the image to achieve dynamic optimization of the median filter.

[0033] For Gaussian noise in the infrared band, the Gaussian filter algorithm is used. The Gaussian filter smoothes the image by calculating the weighted average of the pixel values ​​in the neighborhood around each pixel, and the weight is determined by the Gaussian function. In combination with the adaptive threshold adjustment technology, the filter parameters are dynamically optimized according to the local characteristics of the image. For example, a larger σ value is used in areas with high noise levels, and a smaller σ value is used in areas with rich details. The adaptive threshold adjustment technology can help the Gaussian filter remove noise interference to the greatest extent while retaining the target edge and detail information.

[0034] By performing noise removal on multispectral images, the noise in the multispectral images can be effectively removed while retaining the target edge and detail information, providing high-quality image data for subsequent analysis and application.

[0035] Further, see Figure 3 , perform radiometric correction on multispectral images, including: Step 121: periodically calibrate multiple spectral band sensors using a black body radiation source to obtain radiation response characteristics of the multiple spectral band sensors under different environmental conditions.

[0036] Select a blackbody radiation source with high precision and good stability as the blackbody radiation source to ensure that its radiation output at different temperatures can accurately simulate the radiation conditions in the actual environment. Regularly (such as every quarter or every six months) place the sensors of each spectral band in front of the blackbody radiation source for calibration. During the calibration process, it is necessary to record the response values ​​of the sensor at different temperatures (or radiation intensities) to obtain the radiation response characteristics of the sensor. The sensor response data obtained from each calibration is sorted and stored to form a radiation response database. The database should contain the response characteristics of sensors in each spectral band under different environmental conditions to provide basic data for building a radiation correction model.

[0037] Step 122: Construct a radiation correction model based on the radiation response characteristics.

[0038] Analyze the data in the radiation response database to identify and remove outliers or noise data. At the same time, preprocess the data, such as normalization and smoothing, to improve the accuracy and stability of model construction. According to the complexity of the sensor radiation response characteristics, select a suitable mathematical model (such as polynomial regression, neural network, etc.) for fitting. Through parameter optimization, the model can accurately describe the nonlinear relationship between the sensor response and the real radiation and the radiation response characteristics.

[0039] Step 123: Input the environmental parameters acquired in real time into the radiation correction model to perform radiation correction on the multispectral image.

[0040] Environmental parameter sensors such as light intensity sensors, temperature sensors, and humidity sensors are deployed in the monitoring area to ensure that changes in these parameters can be measured and recorded in real time. In the actual monitoring process, ensure the synchronous acquisition of multispectral image data and environmental parameter data. Input the real-time acquired environmental parameters (such as light intensity, atmospheric temperature, humidity, etc.) into the radiation correction model, and obtain the corrected sensor response values ​​based on the model prediction. Then, use these corrected response values ​​to perform radiation correction on the collected multispectral image data to eliminate radiation errors caused by environmental changes and the sensor's own characteristics.

[0041] Further, see Figure 4 , geometric correction of multispectral images, including: Step 124: Obtain preset feature points in the space and the spatial coordinates corresponding to the feature points.

[0042] Select high-precision coordinate points on the ground as feature points. These points are usually the vertices of landmark buildings, specific marking points on the ground, etc. Their geographic coordinates are known and accurate. Use GPS measurement or other high-precision measurement equipment to obtain the spatial coordinates of these feature points and record them.

[0043] Step 125: Identify feature points in the multispectral image and calculate image coordinates of the feature points in the multispectral image.

[0044] Image processing algorithms are used to detect feature points in multispectral images. These algorithms can identify significant features in the image, such as edges, corners, etc., and match the detected feature points with feature points on the ground. For each successfully matched feature point, its image coordinates in the multispectral image are recorded.

[0045] Step 126: Based on the image coordinates and the spatial coordinates of the feature points, a mapping relationship between the image coordinate system and the spatial coordinate system is constructed by the least square method to perform geometric correction on the multispectral image.

[0046] The image coordinates and spatial coordinates of the feature points are organized into a data set, and the least squares method is used to construct a mapping relationship between the image coordinate system and the spatial coordinate system based on the image coordinates and spatial coordinates of the feature points. The mapping relationship can be expressed as a polynomial function or a transformation matrix.

[0047] Optionally, the mapping relationship is verified using feature points that are not involved in constructing the mapping relationship to ensure the accuracy of the mapping relationship.

[0048] The constructed mapping relationship is applied to each pixel of the multispectral image to convert the image coordinates into geographic coordinates. The image is geometrically corrected by interpolation, resampling and other methods to make the target position in the image accurately correspond to the actual geographic space position. The corrected multispectral image is output to provide a reliable geometric basis for subsequent target positioning and monitoring.

[0049] Step 20: Input the multispectral image into the pre-trained target detection model to extract the position, spectral feature vector, texture feature vector and shape feature vector of the low, slow and small targets in the multispectral image. The target detection model is built based on the YOLO model.

[0050] Select a version in the YOLO series such as YOLOv3, YOLOv4 or YOLOv5 as the basic model, and according to the characteristics of multispectral images, add additional convolutional layers and pooling layers for multispectral data features to the structure of the original YOLO algorithm network, so that the network can better learn and extract target features in multispectral images.

[0051] Then, the location information of the target is extracted from the output of the model, and the spectral feature vector, texture feature vector and shape feature vector are extracted from the fully connected layer or convolutional layer of the model.

[0052] The position information is mapped to the spatial coordinate system after geometric correction and serves as the spatial coordinates of the low, slow and small targets in the actual scene.

[0053] Specifically, a feature fusion strategy is designed to effectively fuse the features of different spectral channels and enhance the model's ability to understand multispectral images.

[0054] Among them, low, slow and small targets are targets with small radar scattering cross-section, slow flying speed and low altitude, such as birds, drones, floating objects and hydrogen balloons.

[0055] Further, see Figure 5 ,The method of extracting the spectral feature vector of low, slow and small targets in multi-spectral images includes: Step 21: Based on the multispectral image, calculate the reflectivity curve variation law of the low, slow and small target in each spectral band to extract the spectral characteristic parameters of the low, slow and small target. The spectral characteristic parameters include spectral peak position, spectral bandwidth, spectral slope, reflectivity ratio characteristics between different spectral bands, and spectral absorption valley depth characteristics.

[0056] For low, slow and small targets in multispectral images, the region of interest is selected, the reflectivity of the region in each spectral band is calculated, and the reflectivity curve is obtained.

[0057] Among them, the wavelength corresponding to the point with the highest reflectivity in the reflectivity curve is the spectral peak position, which reflects the point where the target substance has the strongest reflectivity to light of a specific wavelength; the spectral bandwidth refers to the wavelength range in the reflectivity curve where the reflectivity is higher than a certain threshold; the spectral slope is used to measure the degree of dependence of the reflectivity on the wavelength; two or more specific spectral bands are selected to calculate the reflectivity ratio between them, and the ratio characteristics can reflect the reflectivity difference of the target substance in different bands; the spectral absorption valley refers to the point or area with lower reflectivity in the reflectivity curve, which reflects the absorption ability of the target substance to light of a specific wavelength.

[0058] Step 22: Perform dimension reduction processing on the spectral feature parameters to obtain a spectral feature vector.

[0059] Hyperspectral images usually contain dozens or even hundreds of bands, resulting in very high dimensions of spectral feature parameters. High-dimensional data not only increases computational complexity, but may also cause problems such as information redundancy and noise interference. Therefore, dimensionality reduction of spectral feature parameters helps to extract key information, reduce data dimensions, and improve processing efficiency and classification accuracy.

[0060] The spectral feature parameters are further extracted by reducing the dimension of the spectral feature parameters through methods such as principal component analysis or minimum noise separation. After the dimensionality reduction process, the original high-dimensional spectral feature parameters are converted into low-dimensional spectral feature vectors. The spectral feature vector retains the main information in the original data while reducing the data dimension and computational complexity.

[0061] The dimensionality reduction process is performed on a large number of extracted spectral feature parameters. While reducing data redundancy, the ability of spectral features to distinguish different types of low, slow and small targets is enhanced, highlighting the essential spectral differences of the targets.

[0062] See also Figure 6 ,The method of extracting texture feature vector of low, slow and small targets in multispectral images includes: Step 23: Calculate the gray level co-occurrence matrix of different spectral band images at multiple scales and multiple directions to extract the texture feature parameters of low, slow and small targets. The texture feature parameters include contrast, correlation, entropy and inverse difference.

[0063] Select different spectral band images to be processed, such as infrared, visible light, etc. For each spectral band image, calculate its gray-level co-occurrence matrix (GLCM) matrix at different scales (such as 3×3, 5×5, 7×7 pixel windows) and multiple directions (such as 0°, 45°, 90°, 135°, etc.). Extract texture feature parameters such as contrast, correlation, entropy and inverse gap from each gray-level co-occurrence matrix.

[0064] Among them, contrast reflects the clarity of the image and the depth of the texture grooves; correlation measures the similarity between row or column elements in the gray-level co-occurrence matrix; entropy reflects the complexity or amount of information of the image texture; and inverse gap reflects the homogeneity of the image texture.

[0065] Step 24: Perform multi-resolution decomposition on the texture feature parameters through wavelet transform to obtain texture information of different frequency sub-bands to obtain multi-scale texture feature vectors.

[0066] The texture feature parameters at each scale and direction of each spectral band are transformed by wavelet transform to obtain the texture information of different frequency sub-bands. Specifically, the texture feature parameters are decomposed using two-dimensional discrete wavelet transform (such as Haar wavelet, Daubechies wavelet, etc.) to obtain low-frequency and high-frequency sub-bands. The texture information in different frequency sub-bands is combined to construct a multi-scale texture feature vector.

[0067] The multi-scale and multi-directional texture feature vectors of images in different spectral bands are comprehensively analyzed to fully reflect the texture structure and change law of the target surface. By comparing the texture feature vectors of different targets, low-speed and small targets with similar spectral characteristics but different texture structures can be effectively distinguished.

[0068] See also Figure 7 ,The method of extracting the shape feature vector of low, slow and small targets in multispectral images includes: Step 25: Detect the edges of low, slow and small targets in the multispectral image through edge detection operators.

[0069] Use edge detection operators (such as Canny operator, Sobel operator, Prewitt operator, etc.) to detect the edges of low, slow and small targets in multispectral images. These operators can identify areas in the image where the gray value changes dramatically, thereby outlining the edges of the target.

[0070] Step 26: The edges are refined and connected through morphological operations to form a complete and continuous edge contour.

[0071] Through the morphological thinning operation, the detected edge contour is thinned to remove redundant pixels and make the edge clearer. Due to noise, uneven lighting and other factors, the edge may be broken. Through the morphological connection operation, the broken edges are connected to form a complete and continuous edge contour.

[0072] Step 27: Calculate the shape feature parameters of the low, slow and small target based on the edge contour. The shape feature parameters include perimeter, area, circularity, rectangularity and convex hull area ratio.

[0073] Calculate the perimeter of the thinned and connected edge contour, that is, the total distance of all pixels on the edge; calculate the area of ​​the region enclosed by the edge contour; calculate the circularity through the perimeter and area of ​​the target, which is an indicator to measure the degree to which the target shape is close to a circle; rectangularity is an indicator to measure the degree to which the target shape is close to a rectangle, which can be calculated by the area of ​​the target and the area of ​​the smallest rectangle surrounding the target; convex hull area ratio is an indicator to measure the convexity and concavity of the target shape, which can be calculated by the area of ​​the target and the area of ​​the convex hull of the target.

[0074] Step 28: Construct a shape feature vector based on the shape feature parameters.

[0075] The calculated shape feature parameters such as perimeter, area, circularity, rectangularity, convex hull area ratio, etc. are combined to construct a shape feature vector.

[0076] The shape features are invariant to the rotation, scaling and translation of the target, and can effectively identify the shape category of the target in complex backgrounds and when the target posture changes, and assist in distinguishing low, slow and small targets of different shapes.

[0077] Before feeding the multispectral image into the pre-trained object detection model, it also includes: The improved target detection model is trained by using a multispectral image set that annotates the target position, category, and multispectral feature information to improve the target detection model's ability to extract features and locate small, slow, and low-profile targets.

[0078] Collect multispectral image datasets containing low, slow and small targets such as drones and birds. Use annotation tools to annotate the targets in the images, mark the target's location (bounding box), category and multispectral feature information, and generate annotation files containing the target location and category.

[0079] Optionally, a multi-task loss function including position loss, spectral feature loss, texture feature loss and shape feature loss is designed to guide the optimization direction of the target recognition model; a learning rate decay strategy or cosine annealing strategy is adopted to optimize the training process; a batch normalization layer is used to accelerate training and reduce overfitting.

[0080] Optionally, use metrics such as precision, recall, and mean average precision (mAP) to evaluate model performance. Analyze the model's performance on hard-to-detect targets (such as small-sized, low-contrast targets) to identify and fix model weaknesses.

[0081] Optionally, data enhancement operations such as rotation, scaling, and flipping are performed on the multispectral image to increase data diversity.

[0082] The model is trained using a labeled multispectral image dataset, hyperparameters such as learning rate and batch size are adjusted, and the validation set is used to evaluate the performance of the model, including target detection accuracy and feature vector extraction accuracy.

[0083] Step 30: Using the fused feature vector as the input of the pre-trained target classifier, the classification result of the low, slow and small target output by the target classifier is obtained. The fused feature vector combines the spectral feature vector, the texture feature vector and the shape feature vector.

[0084] In this embodiment, a support vector machine (SVM) is used as the core algorithm of the target classifier.

[0085] Collect a large amount of representative low-speed, slow, and small target sample data, including various types of drones, birds, etc., label each sample, and record its category and model information. According to the characteristics of the data and experience, select the appropriate SVM kernel function (such as linear kernel, polynomial kernel, radial basis function kernel, etc.). Use the cross-validation method to optimize the SVM kernel function parameters (such as the degree of the polynomial kernel, the gamma value of the radial basis function kernel, etc.) and the penalty parameter C. Cross-validation can ensure that the model has good generalization ability on unseen data. Use the optimized parameters and labeled sample data to train the SVM classifier.

[0086] Optionally, the performance of the SVM classifier is evaluated using indicators such as accuracy, recall, and F1 score, and based on the evaluation results, the feature extraction method, feature fusion strategy, SVM parameters, etc. are adjusted and optimized to improve the performance of the classifier.

[0087] Integrate the trained SVM classifier into the target detection system or platform to achieve real-time classification and recognition of low, slow and small targets. Use a suitable fusion strategy to fuse the standardized spectral feature vector, the texture feature vector after dimensionality reduction and the normalized shape feature vector into a comprehensive feature vector and input it into the trained SVM classifier. The SVM classifier outputs the classification result of the target, that is, the category and model information of the target.

[0088] Build a target classification system based on fused feature vector and SVM to achieve accurate classification and recognition of low, slow and small targets, and determine the specific type and model information of the target.

[0089] Step 40: In response to the position or classification result of the low, slow and small target meeting the warning condition, a warning signal is issued.

[0090] Record the time when the target appears, as well as the target's trajectory information in continuous frames or time periods, design a standardized data format to store the content of the monitoring report, including the target's geographic location, type, model, multispectral characteristic parameters, time series information, etc., and store the generated monitoring report in a standardized data format in the database.

[0091] Define specific sensitive areas on the map, such as airport runway clearance areas, military restricted areas, etc. Define the types of dangerous targets, such as drones carrying suspicious items, aircraft of specific models, etc. Set warning thresholds based on actual needs, such as the distance between the target and the sensitive area, the matching degree between the target type and the dangerous target type, etc.

[0092] Connect the early warning system to the monitoring results database to query the latest monitoring report in real time; judge the data in the monitoring report according to the preset early warning rules to determine whether the early warning signal needs to be triggered.

[0093] When the warning conditions are met, the warning signal is automatically triggered, and the warning information is sent to relevant personnel or systems through various means, including sound and light alarm devices, SMS notifications, and linkage with command and control systems.

[0094] See also Figure 8 , Figure 8 It is a structural diagram of an embodiment of the storage medium provided by the present application.

[0095] The storage medium 300 stores program data 310. When the program data 310 is executed by the processor, the following is achieved: Figure 1 The steps of the described low, slow and small target monitoring method.

[0096] The program data 310 is stored in a storage medium 300, and includes a number of instructions for enabling a network device (such as a router, a personal computer, a server, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application.

[0097] Optionally, the storage medium 300 may be a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, an optical disk, or other medium that can store program data.

[0098] See also Fig. 9 , Fig. 9It is a structural schematic diagram of an embodiment of a computer device provided by the present application.

[0099] The computer device 400 includes a processor 420 and a memory 410 that are connected to each other. The memory 410 stores a computer program. When the processor 420 executes the computer program, the steps of the low, slow and small target monitoring method described above are implemented.

[0100] See also Fig.10 , Fig.10 It is a structural schematic diagram of an embodiment of a low-speed, slow, and small target monitoring system provided in the present application. The low-speed, slow, and small target monitoring system 1000 includes a plurality of spectral band sensor arrays 200 and a computer device 400 as described above.

[0101] The multiple spectral band sensor arrays 200 are used to collect multispectral images of visible light, near infrared, medium-wave infrared and long-wave infrared in all directions and send them to the computer device 400; these sensor arrays can capture images of multiple spectral bands such as visible light, near infrared, medium-wave infrared and long-wave infrared, which helps to more accurately identify target type, shape, temperature and other characteristics. The sensor arrays may be arranged in a specific geometric layout (such as a circular array, a rectangular array, etc.) to ensure all-round coverage and reduce blind spots.

[0102] The computer device 400 synchronously receives the multi-spectral images sent by the multiple spectral band sensor arrays 200 to execute the low, slow and small target monitoring method as described above.

[0103] Different from the prior art, this application uses data preprocessing technologies such as noise removal, radiation correction and geometric correction to ensure the high quality and accuracy of data, providing a reliable basis for subsequent processing; at the same time, the rules of noise removal, radiation correction and geometric correction are adaptively adjusted to further optimize the collected multispectral images. Target features are extracted from multiple dimensions such as spectrum, texture, shape, etc., and these features are integrated using effective fusion strategies to form a comprehensive and unique description of the target, which helps to improve the accuracy and reliability of target recognition. The target is then classified based on the extracted features through the target classifier; the target detection module is used to detect and locate targets based on the powerful feature automatic learning ability of deep learning, and the target classifier is combined with the advantages of small sample classification to perform target classification and identification, giving full play to the strengths of both to achieve efficient and accurate low-speed and slow small target monitoring. And warn targets that meet the warning conditions to prevent dangerous targets from entering important areas.

[0104] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the storage medium embodiment and the computer device embodiment, since they are basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0105] The present application can be used in many general or special computing system environments or configurations, such as personal computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, network PCs, minicomputers, and distributed computing environments including any of the above systems or devices.

[0106] In the several embodiments provided in this application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device embodiments described above are only illustrative, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not performed.

[0107] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0108] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0109] The above descriptions are merely embodiments of the present application and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A multi-spectral low-speed, slow, small target monitoring method, characterized in that: include: Acquiring a multispectral image including a plurality of spectral bands; Input the multispectral image into a pre-trained target detection model to extract the position, spectral feature vector, texture feature vector and shape feature vector of low, slow and small targets in the multispectral image; the target detection model is constructed based on the YOLO model; Using the fused feature vector as the input of the pre-trained target classifier, obtaining the classification result of the low, slow and small target output by the target classifier; the fused feature vector fuses the spectral feature vector, the texture feature vector and the shape feature vector; In response to the position of the low, slow and small target or the classification result meeting a warning condition, a warning signal is issued.

2. The low, slow, small target monitoring method according to claim 1, characterized in that: The step of inputting the multispectral image into a pre-trained target detection model to extract spectral feature vectors of low, slow and small targets in the multispectral image includes: Based on the multispectral image, the reflectivity curve variation law of the low, slow and small target in each spectral band is calculated to extract the spectral characteristic parameters of the low, slow and small target; the spectral characteristic parameters include spectral peak position, spectral bandwidth, spectral slope, reflectivity ratio characteristics between different spectral bands and spectral absorption valley depth characteristics; The spectral feature parameters are subjected to dimensionality reduction processing to obtain the spectral feature vector.

3. The low, slow and small target monitoring method according to claim 1 is characterized in that: The step of inputting the multispectral image into a pre-trained target detection model to extract texture feature vectors of low, slow and small targets in the multispectral image includes: Calculating the gray level co-occurrence matrix of images of different spectral bands at multiple scales and multiple directions respectively to extract the texture feature parameters of the low, slow and small target; the texture feature parameters include contrast, correlation, entropy and inverse gap; The texture feature parameters are decomposed at multiple resolutions by wavelet transform to obtain texture information of different frequency sub-bands, so as to obtain the texture feature vectors at multiple scales.

4. The low, slow and small target monitoring method according to claim 1 is characterized in that: The step of inputting the multispectral image into a pre-trained target detection model to extract shape feature vectors of low, slow and small targets in the multispectral image includes: Detecting the edge of the low, slow and small target in the multispectral image by an edge detection operator; Refining and connecting the edges through morphological operations to form a complete and continuous edge contour; Calculate the shape feature parameters of the low, slow and small target based on the edge contour; the shape feature parameters include perimeter, area, circularity, rectangularity, and convex hull area ratio; The shape feature vector is constructed based on the shape feature parameters.

5. The low, slow and small target monitoring method according to claim 2 is characterized in that: Before inputting the multispectral image into the pre-trained target detection model, the method further includes: The improved target detection model is trained by a multispectral image set that annotates the target position, category and multispectral feature information, so as to improve the feature extraction and positioning capability of the target detection model for the low, slow and small targets.

6. The low, slow, small target monitoring method according to claim 1, characterized in that: The acquiring of a multispectral image having a plurality of spectral bands comprises: The multi-spectral image is collected synchronously by a plurality of spectral band sensor arrays; the spectral bands include visible light, near infrared, medium-wave infrared and long-wave infrared; The multispectral image is subjected to noise removal, radiation correction and geometric correction.

7. The low, slow, small target monitoring method according to claim 6, characterized in that: The step of removing noise from the multispectral image comprises: A median filter is performed on the salt and pepper noise of visible light in the multispectral image, and a Gaussian filter is performed on the Gaussian noise of near infrared, medium-wave infrared and long-wave infrared in the multispectral image, and the filtering parameters of the median filter and the Gaussian filter are dynamically optimized according to the local characteristics of the multispectral image.

8. The low, slow, small target monitoring method according to claim 6, characterized in that: The performing radiation correction on the multispectral image comprises: Periodically calibrating the plurality of spectral band sensors through a black body radiation source to obtain radiation response characteristics of the plurality of spectral band sensors under different environmental conditions; constructing a radiation correction model based on the radiation response characteristics; The environmental parameters acquired in real time are input into the radiation correction model to perform radiation correction on the multispectral image.

9. The low, slow, small target monitoring method according to claim 6, characterized in that: The geometric correction of the multispectral image comprises: Obtaining preset feature points in the space and the spatial coordinates corresponding to the feature points; Identifying the feature points in the multispectral image and calculating image coordinates of the feature points in the multispectral image; Based on the image coordinates and the space coordinates of the feature points, a mapping relationship between an image coordinate system and a space coordinate system is constructed by a least square method to perform geometric correction on the multispectral image.

10. A storage medium having program data stored thereon, characterized in that: When the program data is executed by the processor, the steps of the low, slow and small target monitoring method as described in any one of claims 1-9 are implemented.

11. A computer device, characterized in that: It includes a processor and a memory connected to each other, the memory stores a computer program, and when the processor executes the computer program, the steps of the low, slow and small target monitoring method as described in any one of claims 1 to 9 are implemented.

12. A multi-spectral low-speed, slow-speed, small-target monitoring system, characterized in that: A computer device comprising a plurality of spectral band sensor arrays and claim 11; The plurality of spectral band sensor arrays are used to collect multispectral images of visible light, near infrared, medium-wave infrared and long-wave infrared in all directions, and send the images to the computer device; The computer device synchronously receives multispectral images sent by multiple spectral band sensor arrays to execute the low, slow and small target monitoring method as described in any one of claims 1-9.

Images