Unmanned aerial vehicle detection device and method based on periodically arranged multispectral imaging chips

Abstract

The invention discloses an unmanned aerial vehicle detection device and method based on periodically arranged multispectral imaging chips. Relates to the field of unmanned aerial vehicle detection devices based on periodically arranged multispectral imaging chips, in particular to the technical field of unmanned aerial vehicle detection based on periodically arranged multispectral imaging chips. The problem that visible light and infrared detection is easily affected by shape information and temperature information of the unmanned aerial vehicle during long-distance detection, and consequently the false alarm rate is too high is solved. The device comprises an optical lens, a multispectral imaging chip based on periodic arrangement, a circuit board, a power supply interface, a data transmission unit, an image display screen and a data processing unit, the method comprises the following steps: acquiring multispectral image data by using an unmanned aerial vehicle detection device to obtain a multispectral data block; and according to the optimized # imgabs0 # model and the optimized yolo model, acquiring the central position of the target object and the central coordinate of the target object, and outputting a final target position.

Description

Technical Field

[0001] The present invention belongs to the technical field of point target detection, and in particular to the field of unmanned aerial vehicle detection based on periodically arranged multispectral imaging chips. Background Art

[0002] With the development and popularization of the low-altitude economy, the threshold for drone flight is getting lower and lower. Due to the lack of understanding of some flight permissions of the Civil Aviation Administration, some drones will mistakenly fly to some important places; therefore, identifying illegally flying drones is extremely important for military and public security management. Traditional drone detection mainly relies on visible light photoelectric detection, infrared imaging detection, radar wave detection and other technologies, but when detecting at a long distance, the number of pixels occupied by the drone target is relatively small, and the shape, texture and other information will be lost. Therefore, it is difficult for traditional detection technology to achieve accurate detection of long-distance drone targets.

[0003] The traditional radar detection steps for drones are: the radar emits electromagnetic waves outward, and drone targets within the transmission path will radiate part of the radar signal. The sensor analyzes the received radar cross section (RCS) echo to determine the drone's position, shape, and speed. With the development of information technology, drone identification technology based on acoustic features has also developed rapidly. Small drones have a small rotor diameter and a low rotation speed, and the main noise source is load noise. The motor speed can reach 20,000 rpm, which is mainly high-frequency noise. These noises are superimposed together to generate harmonic noise signals. Yakubovskiy et al. proposed a method to extract features outside the spectrum peak, which can realize the classification of aircraft.

[0004] Photoelectric detection technology is also the main technology for drone detection. Currently, the most commonly used technologies are visible light detection technology and infrared detection technology. Visible light detection technology mainly uses the morphological information of drones for detection, while infrared detection technology mainly relies on the thermal radiation information of drones. Sheu et al. developed a dual-axis rotation tracking platform for anti-drone, which mainly uses visible light cameras and infrared thermal imagers to detect drone targets.

[0005] The above detection technologies all have their limitations. Most of the current drones have adaptive flight and fixed-point cruising functions. These drones do not generate radio frequency signals when working. When the detection distance is far, the drone surface is small and its RCS is weak, so radar detection is more difficult. In terms of acoustic detection, the current signal processing and deep learning technologies are developing rapidly, but drone identification based on sound signals still has shortcomings such as short recognition distance and poor anti-interference. In terms of photoelectric detection, visible light and infrared detection are easily affected by the shape and temperature information of the drone during long-distance detection, resulting in a high false alarm rate. Summary of the invention

[0006] In order to solve the problem that visible light and infrared detection are easily affected by the shape information and temperature information of the drone during long-distance detection, resulting in an excessively high false alarm rate, the present invention provides a drone detection device and method based on a periodically arranged multispectral imaging chip, which combines multispectral information with image information to perform target detection and reduce the probability of misidentification of the target.

[0007] The drone detection device based on periodically arranged multispectral imaging chips includes: an optical lens, a periodically arranged multispectral imaging chip, a circuit board, a power supply interface, a data transmission unit, an image display screen, and a data processing unit; The circuit board is connected to the optical lens, the power supply interface and the data transmission unit respectively; the multi-spectral imaging chip based on periodic arrangement is connected to the circuit board through pins, and the optical lens is connected to the multi-spectral imaging chip based on periodic arrangement through an optical path; the data processing unit is connected to the data transmission unit and the image display screen respectively; The optical lens is used for spectral image acquisition, the multispectral imaging chip based on periodic arrangement is used for outputting multispectral image data, the circuit board is used for camera startup and multispectral image data storage, the power supply interface is used for transmitting power, the data transmission unit is used for transmitting the multispectral image data output by the multispectral imaging chip based on periodic arrangement to the data processing unit, the image display screen is used for displaying the final target position imaging, and the data processing unit completes data preprocessing and final target position determination according to the multispectral image data output by the multispectral imaging chip based on periodic arrangement; Set the minimum periodic unit of the multispectral imaging chip based on periodic arrangement to 3*3; The data processing unit includes a preprocessing model, an optimized Model and optimized yolo model; The preprocessing model converts multispectral image data into multispectral data blocks. The model obtains the position of the target object according to the multispectral data block, and the optimized YOLO model converts the multispectral data block into the center coordinates of the target object.

[0008] Furthermore, the preprocessing model is specifically as follows: the multispectral image data is sequentially subjected to background and shadow processing, mosaic processing, over-boundary data truncation and data type conversion.

[0009] Further, the optimized The specific model is as follows: the inversion spectrum data input is changed to nine-channel data input, and the optimized ,in is the square value of the cosine of the angle between two spectral vectors, indicating the similarity between each pixel and the target. , Represents a sequence of pixels, represents the reflectance spectrum of the target, express The transposed matrix of It is the third dimension data of the multispectral data block, representing the spectrum of each pixel on the spectral image. express The transposed matrix of Represents the autocorrelation coefficient matrix between the spectra corresponding to each pixel obtained by processing the spectral image The inverse matrix of .

[0010] Furthermore, the optimized yolo model is specifically as follows: the three-channel model input is changed to a nine-channel model input; the C2 layer in the backbone network is replaced by the Effivient-v2 network; the original three anchor frames in the anchor frame part are increased to nine anchor frames; a small target detection layer is added to the neck part to obtain the optimized yolo model. Furthermore, it is characterized in that the minimum periodic unit of the multi-spectral imaging chip based on periodic arrangement includes four light-transmitting films, and the transmittance of the four light-transmitting films is set as follows: Film a (21): the main transmission wavelength range is 550nm-700nm, and the remaining bands are 10% transmitted; Film b (22): the main transmission wavelength range is 350nm-950nm, and the remaining bands are 0% transmitted; Film c (23): the main transmission wavelength range is 500nm-950nm, and the remaining bands are 10% transmitted; Film d (24): the main transmission wavelength range is 350nm-500nm and 600nm-850nm, and the remaining bands are 10% transmitted.

[0011] The drone detection method based on periodically arranged multispectral imaging chips is implemented by using the drone detection device based on periodically arranged multispectral imaging chips, and is characterized in that the method comprises the following steps: S1. Collect multispectral image data using an unmanned aerial vehicle detection device and perform preprocessing to obtain a multispectral data block; S2, according to the optimized Model, calculate the similarity between each pixel and the target, and obtain the location of the target; S3. Obtain the center coordinates of the target object according to the optimized YOLO model; S4. Calculate the Euclidean distance between the position of the target object and the center coordinates of the target object, and output the final target position.

[0012] Furthermore, the preprocessing specifically includes: inputting the multispectral image data into a preprocessing model.

[0013] Furthermore, a judgment is made based on the similarity between each pixel and the target, and pixel points with a similarity with the target greater than 0.85 are subjected to threshold segmentation processing to obtain a segmented image; the segmented image is then binarized, connected domain judged, and medium wave filtering processed in sequence to obtain the position of the target object.

[0014] Furthermore, the obtaining of the center coordinates of the target object is specifically as follows: inputting the multispectral data block into the optimized YOLO model to obtain the xywh coordinates of the target object, wherein x represents the horizontal coordinate of the target object, y represents the vertical coordinate, w represents the width, and h represents the height; performing coordinate conversion on the xywh coordinates of the target object to obtain the center coordinates of the target object.

[0015] Further, when the Euclidean distance between the position of the target object and the central coordinates of the target object is less than 10, the position of the target object is the final target position.

[0016] The beneficial effects of the method of the present invention are: Compared with the traditional UAV detection method, the present invention makes full use of the periodic structure of the multi-spectral imaging chip based on periodic arrangement, combines the spectral data with the image data, so that each pixel can independently output the spectral information and image information, and uses the optimized The model and the optimized YOLO model can realize the accurate identification of small and medium-sized UAVs, and make up for the defects of shape and texture, or the unrecognizable and misidentified birds caused by the relatively small number of pixels occupied by UAV targets during detection by traditional technologies. The spectral information is combined with the image information to improve the recognition rate while reducing the false alarm rate and reduce the waste of police resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a structural diagram of a detection device according to Embodiment 1 of the present invention; Figure 2 Schematic diagram of the periodic structure of the multi-spectral imaging chip based on periodic arrangement in Example 1 of the present invention, wherein 21 represents film a, 22 represents film b, 23 represents film c, and 24 represents film d; Figure 3 Schematic diagram of the band transmittance curve of each thin film unit in the periodic structure of the multi-spectral imaging chip based on periodic arrangement in Example 1 of the present invention, wherein the abscissa represents the band and the ordinate represents the transmittance; Figure 4 This is a flow chart of the method of embodiment 2 of the present invention. DETAILED DESCRIPTION

[0018] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of the present invention.

[0019] Embodiment 1, This embodiment provides a drone detection device based on a periodically arranged multi-spectral imaging chip, such as Figure 1 As shown, the device includes: an optical lens 1, a multi-spectral imaging chip 2 based on a periodic arrangement, a circuit board 3, a power supply interface 4, a data transmission unit 5, an image display screen 6 and a data processing unit 7.

[0020] The circuit board 3 is respectively connected to the optical lens 1, the power supply interface 4 and the data transmission unit 5; the multi-spectral imaging chip 2 based on periodic arrangement is connected to the circuit board 3 through pins, and the optical lens 1 is adapted to the multi-spectral imaging chip 2 based on periodic arrangement and connected through an optical path; the data processing unit 7 is respectively connected to the data transmission unit 5 and the image display screen 6.

[0021] The optical lens 1 is used for spectral image acquisition, the multispectral imaging chip 2 based on periodic arrangement is used for outputting multispectral image data, the circuit board 3 is used for camera startup and multispectral image data storage, the power supply interface 4 is used for transmitting power, the data transmission unit 5 is used for transmitting the multispectral image data output by the multispectral imaging chip 2 based on periodic arrangement to the data processing unit 7, the image display screen 6 is used for displaying the final target position imaging, and the data processing unit 7 completes data preprocessing and final target position determination according to the multispectral image data output by the multispectral imaging chip 2 based on periodic arrangement.

[0022] At present, the materials of drones are mainly carbon fiber, glass fiber, plastic, etc. The optical properties of these materials are relatively stable, so their reflectivity spectrum is relatively constant and does not change with distance. At the same time, the background spectrum also has special properties. According to the difference between the background and target spectra, a periodically arranged multi-spectral imaging chip can be designed.

[0023] The multispectral imaging chip 2 based on periodic arrangement consists of a photoelectric conversion base and a light-transmitting film with a periodic structure. For the specific preparation process, please refer to the Chinese invention patent "Imaging spectral chip with both spectral and imaging functions and its preparation method" (CN202010189838.5).

[0024] The target object has good reflection in the 400nm-950nm band, and the background spectrum has a higher response in the blue band; therefore, the film transmittance of the multi-spectral imaging chip 2 based on the periodic arrangement is designed as follows: the transmittance in the blue band is lower, thereby reducing the transmittance of the background, while increasing the transmittance of the spectrum band with high target reflectivity, which is conducive to adding more target spectral information and image information; Figure 2 As shown, based on a Si-based thin film substrate, the minimum periodic unit of the multi-spectral imaging chip 2 based on periodic arrangement is etched into a 3*3 periodic structure, and each periodic unit has a spectral modulation film with different transmittance; the wavelength range that the multi-spectral imaging chip 2 based on periodic arrangement can transmit is: 350nm-950nm, and the spectral transmittance on each periodic unit is related to the spectrum of the point target and the background, that is, each pixel can independently output spectral information and image information; the periodic structure of the multi-spectral imaging chip 2 based on periodic arrangement can not only improve the resolution of the image, but also amplify the target spectral characteristics according to the difference between the spectrum of the point target and the background spectrum.

[0025] The minimum period unit includes four light-transmitting films, and the transmittance of the four light-transmitting films is set as follows: film a21: the main transmission wavelength range is 550nm-700nm, and the remaining bands are 10% transmitted; film b22: the main transmission wavelength range is 350nm-950nm, and the remaining bands are 0% transmitted; film c23: the main transmission wavelength range is 500nm-950nm, and the remaining bands are 10% transmitted; film d24: the main transmission wavelength range is 350nm-500nm and 600nm-850nm, and the remaining bands are 10% transmitted; the band transmittance curves of each film unit are as follows: Figure 3 shown.

[0026] Through the design of transparent thin film spectroscopic materials, the incident light is split and processed, the captured target spectrum is transmitted with a higher transmittance, and the background spectrum is transmitted with a lower transmittance. When imaging, the brightness of the background is reduced and the brightness of the target is increased. The difference in brightness between the background and the target can provide favorable conditions for subsequent algorithm recognition.

[0027] The field of view angle and focal length of the optical lens 1 are set according to the size of the multispectral imaging chip 2 based on the periodic arrangement and the size of the pixel, so as to detect the UAV within the field of view. The field of view angle and focal length of the optical lens 1 are 150 degrees and 1.7 mm respectively.

[0028] The data processing unit 7 includes a preprocessing model, an optimized Model and optimized yolo model; The preprocessing model converts multispectral image data into multispectral data blocks. The model obtains the position of the target object according to the multispectral data block, and the optimized YOLO model converts the multispectral data block into the center coordinates of the target object.

[0029] The preprocessing model specifically includes: subjecting the multispectral image data to background and shadow processing, mosaic processing, over-boundary data truncation and data type conversion in sequence.

[0030] The background and shadow processing includes background subtraction and shadow correction; the background subtraction means subtracting the signal data output by the multispectral imaging chip 2 based on periodic arrangement under a dark background, so as to achieve the effect of data correction; the shadow correction means correcting the situation where the multispectral image data presents a bright center and dark edges.

[0031] The mosaic processing refers to performing interpolation processing on the multispectral image data to obtain multispectral data having spectrum and image information.

[0032] The out-of-boundary data truncation and data type conversion specifically include: truncating the portion of the multispectral data block whose value is greater than 1023 or less than 0, and converting the decimal data into octal data to obtain the multispectral data block.

[0033] The optimized The specific model is as follows: the inversion spectrum data input is changed to nine-channel data input, and the optimized ,in is the square value of the cosine of the angle between two spectral vectors, indicating the similarity between each pixel and the target. , Represents a sequence of pixels, represents the reflectance spectrum of the target, express The transposed matrix of It is the third dimension data of the multispectral data block, representing the spectrum of each pixel on the spectral image. express The transposed matrix of Represents the autocorrelation coefficient matrix between the spectra corresponding to each pixel obtained by processing the spectral image The inverse matrix of Represents the magnitude of the target's reflection spectrum vector; Represents the modulus of the spectral vector of each pixel of the spectral image, Represents the inner product calculation function.

[0034] Since the conventional YOLO model has more layers and parameters, the inference time is longer, and the UAV flying speed is relatively fast, it is necessary to update the recognition rate of the UAV. Therefore, the conventional YOLO model is optimized as follows: (1) The three-channel model input of the conventional YOLO model is changed to a nine-channel model input. The number of channels of the original model is RGB three channels, and the number of channels of the optimized YOLO model is nine-channel data. A single channel of the nine-channel data is image information, and the longitudinal nine channels are spectral information. After changing to the nine-channel model input, both image information and spectral information can be used to detect the target object; (2) The C2 layer in the backbone network is replaced by the Effivient-v2 network, which has a simple structure and faster inference time. By introducing the Effivient-v2 network, the training speed and the number of parameters are increased without reducing the accuracy of the training set, which can improve the network prediction speed for faster UAVs. (3) Increase the number of anchor frames from the original three to nine, and increase the detection range of the anchor frames; (4) Adding a small target detection layer to the neck can increase the recognition of long-range drones.

[0035] After steps (1) to (4), the optimized YOLO model is obtained.

[0036] Embodiment 2, This embodiment is a further limitation of Embodiment 1. The detection method is implemented by using the above-mentioned drone detection device based on periodically arranged multi-spectral imaging chips. Figure 4 As shown, the method comprises the following steps: S1. Use the UAV detection device to collect multispectral image data and perform preprocessing to obtain multispectral data blocks.

[0037] The relevant operations of step S1 are introduced with a specific example: Input the multispectral image data into the preprocessing model.

[0038] S2, according to the optimized Model, calculate the similarity between each pixel and the target, and obtain the position of the target.

[0039] The relevant operations of step S2 are introduced with a specific example: The similarity between each pixel and the target is judged, and the pixel points with similarity greater than 0.85 are subjected to threshold segmentation processing to obtain a segmented image; the segmented image is then binarized, connected domain judged, and medium wave filter processed in sequence to obtain the position of the target object.

[0040] S3. According to the optimized YOLO model, obtain the center coordinates of the target object.

[0041] The relevant operations of step S3 are introduced with a specific example: The multispectral data block is input into the optimized YOLO model to obtain the xywh coordinates of the target object, where x represents the horizontal coordinate of the target object, y represents the vertical coordinate, w represents the width, and h represents the height; the xywh coordinates of the target object are converted to obtain the center coordinates of the target object.

[0042] S4. Calculate the Euclidean distance between the position of the target object and the center coordinates of the target object, and output the final target position.

[0043] The relevant operations of step S4 are introduced with a specific example: When the Euclidean distance between the position of the target object and the central coordinates of the target object is less than 10, the position of the target object is the final target position.

[0044] When there is only a little image and texture information of the drone due to environmental and weather factors, the improved YOLO model cannot recognize it. The model identifies the spectral dimension of the multispectral data block to obtain pixel-by-pixel spectral data, processes the pixel-by-pixel spectral data with the spectrum in the database, and makes full use of the spectral information to detect the drone. When the drone image information is less, the spectral information and image information can also be used to realize the identification of the drone.

[0045] The database is drone spectral data of different weather conditions and distances collected in advance.

Claims

1. A drone detection device based on a periodically arranged multispectral imaging chip, characterized in that: include: An optical lens (1), a multi-spectral imaging chip (2) based on a periodic arrangement, a circuit board (3), a power supply interface (4), a data transmission unit (5), an image display screen (6), and a data processing unit (7); The circuit board (3) is respectively connected to the optical lens (1), the power supply interface (4) and the data transmission unit (5); the multi-spectral imaging chip (2) based on a periodic arrangement is connected to the circuit board (3) via pins, and the optical lens (1) is connected to the multi-spectral imaging chip (2) based on a periodic arrangement via an optical path; the data processing unit (7) is respectively connected to the data transmission unit (5) and the image display screen (6); The optical lens (1) is used for spectral image acquisition, the multispectral imaging chip (2) based on a periodic arrangement is used for outputting multispectral image data, the circuit board (3) is used for camera startup, camera parameter control and multispectral image data storage, the power supply interface (4) is used for transmitting power, the data transmission unit (5) is used for transmitting the multispectral image data output by the multispectral imaging chip (2) based on a periodic arrangement to a data processing unit (7), the image display screen (6) is used for displaying the final target position imaging, and the data processing unit (7) completes data preprocessing and final target position determination according to the multispectral image data output by the multispectral imaging chip (2) based on a periodic arrangement; The minimum periodic unit of the multispectral imaging chip (2) based on periodic arrangement is set to 3*3; The data processing unit (7) includes a preprocessing model, an optimized Model and optimized yolo model; The preprocessing model converts multispectral image data into multispectral data blocks. The model obtains the position of the target object according to the multispectral data block, and the optimized YOLO model converts the multispectral data block into the center coordinates of the target object.

2. The drone detection device based on periodically arranged multispectral imaging chips according to claim 1 is characterized in that: The preprocessing model specifically includes: subjecting the multispectral image data to background and shadow processing, mosaic processing, over-boundary data truncation and data type conversion in sequence.

3. The drone detection device based on periodically arranged multispectral imaging chips according to claim 1, characterized in that: The optimized The specific model is as follows: the inversion spectrum data input is changed to nine-channel data input, and the optimized ,in is the square value of the cosine of the angle between two spectral vectors, indicating the similarity between each pixel and the target. , Represents a sequence of pixels, represents the reflectance spectrum of the target, express The transposed matrix of It is the third dimension data of the multispectral data block, representing the spectrum of each pixel on the spectral image. express The transposed matrix of Represents the autocorrelation coefficient matrix between the spectra corresponding to each pixel obtained by processing the spectral image The inverse matrix of .

4. The drone detection device based on periodically arranged multispectral imaging chips according to claim 1, characterized in that: The optimized yolo model is specifically as follows: the three-channel model input is changed to a nine-channel model input; the C2 layer in the backbone network is replaced by the Effivient-v2 network; the original three anchor frames in the anchor frame part are increased to nine anchor frames; a small target detection layer is added to the neck part to obtain the optimized yolo model.

5. The drone detection device based on periodically arranged multispectral imaging chips according to claim 1, characterized in that: The minimum periodic unit of the multi-spectral imaging chip (2) based on periodic arrangement includes four light-transmitting films, and the transmittances of the four light-transmitting films are set as follows: film a (21): the main transmission wavelength range is 550nm-700nm, and the remaining bands are 10% transmittance; film b (22): the main transmission wavelength range is 350nm-950nm, and the remaining bands are 0% transmittance; film c (23): the main transmission wavelength range is 500nm-950nm, and the remaining bands are 10% transmittance; film d (24): the main transmission wavelength range is 350nm-500nm and 600nm-850nm, and the remaining bands are 10% transmittance.

6. A method for detecting unmanned aerial vehicles based on a periodically arranged multispectral imaging chip, wherein the method is implemented by using a device for detecting unmanned aerial vehicles based on a periodically arranged multispectral imaging chip according to any one of claims 1 to 5, characterized in that: The method comprises the following steps: S1. Collect multispectral image data using an unmanned aerial vehicle detection device and perform preprocessing to obtain a multispectral data block; S2, according to the optimized Model, calculate the similarity between each pixel and the target, and obtain the location of the target; S3. Obtain the center coordinates of the target object according to the optimized YOLO model; S4. Calculate the Euclidean distance between the position of the target object and the center coordinates of the target object, and output the final target position.

7. The method for detecting unmanned aerial vehicles based on periodically arranged multispectral imaging chips according to claim 6, characterized in that: The preprocessing specifically includes: inputting the multispectral image data into a preprocessing model.

8. The method for detecting unmanned aerial vehicles based on periodically arranged multispectral imaging chips according to claim 6, characterized in that: The similarity between each pixel and the target is judged, and the pixel points with similarity greater than 0.85 are subjected to threshold segmentation processing to obtain a segmented image; the segmented image is then binarized, connected domain judged, and medium wave filter processed in sequence to obtain the position of the target object.

9. The method for detecting unmanned aerial vehicles based on periodically arranged multispectral imaging chips according to claim 6, characterized in that: The method of obtaining the center coordinates of the target object is as follows: inputting the multispectral data block into the optimized YOLO model to obtain the xywh coordinates of the target object, wherein x represents the horizontal coordinate of the target object, y represents the vertical coordinate, w represents the width, and h represents the height; performing coordinate conversion on the xywh coordinates of the target object to obtain the center coordinates of the target object.

10. The method for detecting unmanned aerial vehicles based on periodically arranged multispectral imaging chips according to claim 9, characterized in that: When the Euclidean distance between the position of the target object and the central coordinates of the target object is less than 10, the position of the target object is the final target position.

Images