Target real-time detection system and method based on unmanned aerial vehicle platform

Abstract

The invention relates to a target real-time detection system and method based on an unmanned aerial vehicle platform, the target real-time detection system comprises an unmanned aerial vehicle, and an image acquisition device and a supercomputing board card which are installed on the unmanned aerial vehicle, the image acquisition device is used for acquiring a target image and sending the target image to the supercomputing board card; the supercomputing board card comprises a decoding module used for decoding the target image in real time to obtain decoded data; and the target identification module is used for carrying out real-time target identification by utilizing the decoded data. Compared with the prior art, the method has the advantages that all-time and azimuth target detection capacity is achieved, quick response and high-precision target recognition are facilitated, and the automation degree is high.

Description

Technical Field

[0001] The present invention relates to the technical field of target recognition, and in particular to a real-time target detection system and method based on an unmanned aerial vehicle platform. Background Art

[0002] Target recognition and detection have broad application significance in traffic management, security monitoring, military applications, environmental protection and other fields. For example, by detecting and tracking vehicles and ships, traffic management systems can monitor traffic flow on roads and waters in real time, identify various types of violations, and help improve the safety of roads and waters.

[0003] In traditional target (vehicles, ships, etc.) identification and tracking, ground-based sensors, satellite remote sensing and other methods are mainly relied on. These monitoring methods cannot meet the actual application needs of real-time detection. With the rapid development of drone technology, it has flexible maneuverability, can quickly adjust the flight altitude and position, and is easier to cover the target area, realize real-time monitoring of important targets, and provide more comprehensive information for target identification. Existing drones are equipped with sensors to obtain remote sensing images and process data through ground computers. It is difficult to achieve real-time interpretation on the drone, resulting in low work efficiency and inability to quickly deal with it. Summary of the invention

[0004] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a real-time target detection system and method with all-day and all-round target detection capabilities, which facilitates rapid response, high-precision target recognition, and a high degree of automation.

[0005] The purpose of the present invention can be achieved by the following technical solutions:

[0006] According to a first aspect of the present invention, a real-time target detection system based on an unmanned aerial vehicle platform is provided, the system comprising: an unmanned aerial vehicle and an image acquisition device and a supercomputing board installed on the unmanned aerial vehicle, wherein:

[0007] The image acquisition device is used to acquire the target image and send it to the supercomputing board;

[0008] The supercomputing board includes:

[0009] A decoding module, used for decoding the target image in real time to obtain decoding data;

[0010] The target recognition module is used to perform real-time target recognition using the decoded data.

[0011] As a preferred technical solution, the image acquisition device is a multi-spectral thermal infrared camera;

[0012] The target image includes at least one type of image among a panchromatic remote sensing image, a multispectral image and a thermal infrared remote sensing image.

[0013] As a preferred technical solution, the target recognition module includes:

[0014] An image type determination unit, used to determine the image type of the target image;

[0015] The recognition algorithm calling unit is used to call a target recognition algorithm of a corresponding type according to the image type to perform target recognition on the target image.

[0016] As a preferred technical solution, the supercomputing board also includes:

[0017] The shooting control module is used to determine in real time whether the drone has reached the imaging area, and control the image acquisition device to acquire the target image when the drone reaches the imaging area.

[0018] As a preferred technical solution, the supercomputing board also includes:

[0019] A shooting mode configuration module, used to obtain the shooting mode configuration of the image acquisition device;

[0020] The shooting control module is specifically used to: determine in real time whether the drone has reached the imaging area, and control the image acquisition device to acquire the target image according to the corresponding shooting mode when the drone reaches the imaging area.

[0021] As a preferred technical solution, the system further includes:

[0022] A wireless communication module, communicating with the supercomputing board, for uploading the data acquired by the supercomputing board to a server in real time, so that the mobile terminal can call the data from the server in real time;

[0023] The supercomputing board also includes:

[0024] The data transmission module communicates with the wireless communication module and is used to send the data to be uploaded acquired by the supercomputing board to the wireless communication module.

[0025] As a preferred technical solution, the supercomputing board also includes:

[0026] The data compression module communicates with the data transmission module and is used to compress the data to be uploaded obtained by the supercomputing board and send it to the wireless communication module.

[0027] As a preferred technical solution, the target recognition module includes:

[0028] The image preprocessing unit is used to perform at least one of image registration, image stitching and radiation equation correction on the decoded target image.

[0029] As a preferred technical solution, the target recognition module includes:

[0030] The coordinate conversion unit is used to convert the target image coordinates obtained by performing real-time target recognition using the decoded target image into geographic coordinates based on a pre-established conversion model between image coordinates and geographic coordinates.

[0031] According to a second aspect of the present invention, a real-time target detection method based on an unmanned aerial vehicle platform is provided, which is applied to a supercomputing board in a real-time target detection system. The method comprises:

[0032] Get the target image;

[0033] Decoding the target image in real time to obtain decoded data;

[0034] Based on the image type of the target image, calling a target recognition algorithm of a corresponding type to perform target recognition on the target image to obtain a target recognition result;

[0035] The decoded data and the target recognition result are uploaded to a server.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] 1. Possessing all-day and all-round target detection capabilities: The real-time target detection system and method provided by the present invention can achieve all-day and all-round target detection capabilities through real-time processing of target images and real-time target recognition.

[0038] 2. It helps to respond quickly: The real-time target detection system and method provided by the present invention can upload relevant data to the server in real time through the cooperation of the image acquisition device, the supercomputing board and the wireless communication module, thereby providing data support for relevant personnel and helping to respond and deal with the problem quickly.

[0039] 3. High-precision target recognition: The real-time target detection system and method provided by the present invention configures corresponding types of target recognition algorithms for different types of target images, so that real-time target detection can select appropriate target recognition algorithms for target detection according to specific scenarios, which is conducive to improving target recognition accuracy.

[0040] 4. High degree of automation: The real-time target detection system and method provided by the present invention can realize automated processes such as online mapping, online interpretation, online sending, and online identification, which reduces manual intervention and improves the overall automation level of the system.

[0041] 5. Strong flexibility: The real-time target detection system and method provided by the present invention can flexibly configure the imaging area and the shooting mode, so that the system can meet the application requirements in different scenarios and has strong flexibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 This is a schematic diagram of the architecture of a real-time target detection system in an embodiment of the present invention;

[0043] Figure 2 Schematic diagram of the process of real-time target detection method in an embodiment of the present invention. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work should fall within the scope of protection of the present invention. The "one embodiment" or "embodiment" referred to herein refers to specific features, structures or characteristics that can be included in at least one implementation of the present application.

[0045] like Figure 1 As shown, this embodiment provides a real-time target detection system 100 based on an unmanned aerial vehicle platform, the system comprising: an unmanned aerial vehicle, an image acquisition device 110 and a supercomputing board 120 installed on the unmanned aerial vehicle, wherein:

[0046] An image acquisition device 110 is used to acquire a target image and send it to a supercomputing board 120;

[0047] Supercomputing board 120, including:

[0048] The decoding module 121 is used to decode the target image in real time and obtain decoded data;

[0049] The target recognition module 122 is used to perform real-time target recognition using the decoded data.

[0050] The above-mentioned supercomputing board 120 is also called a supercomputing board or a high-performance computing board. This type of board usually integrates high-performance processors, memory and other dedicated chips, such as graphics processing units GPU, field programmable gate arrays FPGA, etc., to achieve large-scale parallel computing and high-speed data processing capabilities.

[0051] The main steps of the above decoding module 121 to decode the target image in real time include:

[0052] (1) Data acquisition: Read the image storage location through the camera file API to obtain the imaging data of the target image;

[0053] (2) Data decoding: converting the acquired hexadecimal-encoded string image data into UInt16 format;

[0054] This step converts the image data from the camera-specific encoding format into a digital format that can be directly processed by the computer;

[0055] (3) Band separation: read the decoded data in array format in band order so that image data of different bands can be processed separately later;

[0056] (4) Data recording: Allocate temporary storage space in the memory of the supercomputer board 120 and write the decoded band data into the allocated storage space; at the same time, record the metadata of the remote sensing image, such as longitude, latitude, altitude, heading and time, which is crucial for subsequent data processing and target identification.

[0057] (5) Generating data index: In order to facilitate the subsequent processing modules to quickly access the required data, the decoding module 121 also generates a data index.

[0058] The decoded data obtained may include:

[0059] (1) Decoded target image: image data separated by bands and stored in array format. These data can be directly used in subsequent image processing and recognition algorithms.

[0060] (2) Metadata of remote sensing images: including longitude, latitude, altitude, heading, and time. These data provide information about the geographic location and shooting conditions of the image, which is crucial for target identification and geographic coordinate conversion.

[0061] (3) Data index: It is used to facilitate the subsequent processing modules to quickly access the index information of the required data.

[0062] Optionally, the image acquisition device 110 is a multispectral thermal infrared camera, and the target image includes at least one type of image among a panchromatic remote sensing image, a multispectral image and a thermal infrared remote sensing image.

[0063] Optionally, the target identification module 122 includes:

[0064] The image type determination unit 1221 is used to determine the image type of the target image, such as a panchromatic remote sensing image, a multispectral image, and a thermal infrared remote sensing image.

[0065] The recognition algorithm calling unit 1222 is used to call a target recognition algorithm of a corresponding type according to the image type to perform target recognition on the target image.

[0066] The target recognition algorithms include: the YOLO-RotDetect algorithm for target recognition in multispectral images, the YOLOv8 algorithm or the threshold detection algorithm for target recognition in thermal infrared images.

[0067] In addition, the recognition results obtained by using the target recognition algorithm to identify the target image can include: target category, confidence level and image coordinates. If you choose to turn on the acquisition of thermal infrared information during the shooting process, the Celsius temperature corresponding to the image pixel will be calculated through the thermal infrared image to temperature matrix equation, and the average temperature, maximum temperature and minimum temperature of the target recognition result will be stored.

[0068] Optionally, the supercomputing board 120 further includes:

[0069] The shooting control module 123 is used to determine in real time whether the drone has reached the imaging area, and control the image acquisition device 110 to acquire the target image when the drone reaches the imaging area.

[0070] According to the imaging area parameter file (such as kml file) injected into the system in the early stage of the flight plan, the shooting control module 120 can autonomously determine whether the real-time position of the drone is within the preset imaging area. If the drone is within the imaging area, the module will control the multi-spectral thermal infrared camera to shoot; if it is not within the imaging area, the camera will not perform the shooting operation.

[0071] Optionally, the supercomputing board 120 further includes:

[0072] A shooting mode configuration module 124, used to obtain the shooting mode configuration of the image acquisition device 110;

[0073] At this time, the shooting control module 123 is specifically used to: determine in real time whether the drone has reached the imaging area, and control the image acquisition device 110 to acquire the target image according to the corresponding shooting mode when the drone reaches the imaging area.

[0074] The shooting mode configured by the shooting mode configuration module 124 may take into account the camera imaging band and shooting time, for example, a certain mode is set to enable the camera to simultaneously enable the multispectral band and the thermal infrared band from 7:00 to 19:00, and enable the corresponding band as needed from 19:00 to 7:00 the next day. This is conducive to obtaining the best remote sensing images under different lighting and meteorological conditions.

[0075] After the camera is finished shooting, the camera shooting control module obtains the imaging capture completion instruction through the camera status API and uses the camera file API to read the image storage location. The module then transfers the image data from the camera to the cache of the supercomputing board for subsequent data processing. During the data transmission process, the camera shooting control module is also responsible for recording metadata related to the image, including longitude, latitude, altitude, heading, time, etc. This information is crucial for subsequent data processing, geographic coordinate conversion, and data analysis.

[0076] Optionally, the real-time target recognition system 100 further includes:

[0077] The wireless communication module 130 communicates with the supercomputing board 120 and is used to upload the data acquired by the supercomputing board 120 to the server 200 in real time, so that the mobile terminal 300 can call the data from the server 200 in real time;

[0078] The supercomputing board 120 also includes:

[0079] The data transmission module 125 communicates with the wireless communication module 130 and is used to send the data to be uploaded acquired by the supercomputing board 120 to the wireless communication module 130 .

[0080] The wireless communication module 130 may be a 5G wireless communication module, and the data to be uploaded transmitted by the data transmission module 125 may include:

[0081] (1) Target recognition results: including the identified target category, confidence level, and location information of the target in the image;

[0082] (2) Target image: high-quality remote sensing image data after preprocessing (such as image registration, image stitching, and radiation equation correction);

[0083] (3) Image metadata: Metadata related to remote sensing images, such as longitude, latitude, altitude, heading, time, etc., which are used to describe the geographic information and shooting conditions of the image.

[0084] (4) Geographic coordinate information of the target: The image coordinates in the target recognition result are converted into geographic coordinates (longitude and latitude) through a coordinate conversion unit so as to accurately locate the target on the map.

[0085] (5) Compressed data packet: The processed image data, target detection results and corresponding information are converted into an appropriate format (such as JPEG images and XML files) and packaged and compressed into zip format to reduce the amount of data transmission.

[0086] Optionally, the supercomputing board 120 further includes:

[0087] The data compression module 126 communicates with the data transmission module 125 and is used to compress the data to be uploaded obtained by the supercomputing board 120 and send it to the wireless communication module 130.

[0088] It is understandable that after completing target recognition, the target recognition module 122 converts the processed image data into JPEG format to retain geographic information and generates an XML file containing image metadata and target detection results and corresponding information. The above-mentioned data compression module 126 converts the packaged JPEG image and XML file into a more compact format through a compression algorithm, thereby reducing the amount of data during data transmission. Since the compressed data is smaller in size, it can be transmitted to the destination faster through a wireless communication module (such as a 5G communication module).

[0089] In addition, the data compression module 126 is also responsible for packaging the compressed image data, image metadata, target detection results and corresponding information into a unified file format (such as zip format), and applying an appropriate compression algorithm for further compression. This not only facilitates data transmission and storage, but also facilitates the access and decompression of data by subsequent processing modules.

[0090] Optionally, the target identification module 122 includes:

[0091] The image preprocessing unit 1223 is used to perform at least one of image registration, image stitching and radiation equation correction on the decoded target image.

[0092] The preprocessing that can be performed by the image preprocessing unit 1223 includes:

[0093] (1) Image registration: By registering multispectral and thermal infrared images, the consistency of images acquired in different bands or at different times in spatial position is ensured, providing a basis for subsequent image stitching and target recognition;

[0094] (2) Based on the overlapping areas of images, feature point matching and image fusion algorithms are used to stitch multiple images into a larger range and higher resolution image, so as to facilitate the detection and analysis of targets in a wider area;

[0095] (3) Radiation equation correction: The image is radiometrically corrected based on the camera calibration parameters and atmospheric parameters to eliminate image distortion caused by atmospheric conditions, camera performance and other factors, thereby improving image quality and target detection accuracy.

[0096] Optionally, the target identification module 122 includes:

[0097] The coordinate conversion unit 1224 is used to convert the target image coordinates obtained by performing real-time target recognition using the decoded target image into geographic coordinates based on a pre-established conversion model between image coordinates and geographic coordinates.

[0098] It is understandable that the target coordinates in the target identification information acquired by the target identification module 122 are image coordinates, so the image coordinates of the target can be converted into geographic coordinates, such as the longitude and latitude of the target, by the coordinate conversion unit 1224, so as to facilitate display in the mobile terminal 300.

[0099] Optionally, the supercomputing board 120 may further include:

[0100] The UAV position update module is used to call and update the UAV's attitude information in real time, including key data such as latitude, longitude, altitude, time, etc.

[0101] Status check module, used to check the drone's attitude information, camera status, network status, and storage space;

[0102] The storage module 127 is used to store relevant data.

[0103] like Figure 2 As shown, based on the same inventive concept, this embodiment also provides a real-time target detection method based on an unmanned aerial vehicle platform, which is applied to a supercomputing board in a real-time target detection system, including:

[0104] S410: Acquire a target image;

[0105] S420: Decode the target image in real time to obtain decoded data;

[0106] S430: Based on the image type of the target image, calling a target recognition algorithm of a corresponding type to perform target recognition on the target image, and obtaining a target recognition result;

[0107] S440: Upload the decoded data and target recognition results to the server.

[0108] It can be understood that, in addition to implementing the above steps, the above target real-time detection method can also implement the steps that can be implemented by the various software modules in the above supercomputing board 120, which will not be repeated in this embodiment.

[0109] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A real-time target detection system based on an unmanned aerial vehicle platform, characterized in that: The system comprises: a drone and an image acquisition device and a supercomputing board installed on the drone, wherein: The image acquisition device is used to acquire the target image and send it to the supercomputing board; The supercomputing board includes: A decoding module, used for decoding the target image in real time to obtain decoding data; The target recognition module is used to perform real-time target recognition using the decoded data.

2. The real-time target detection system based on an unmanned aerial vehicle platform according to claim 1 is characterized in that: The image acquisition device is a multi-spectral thermal infrared camera; The target image includes at least one type of image among a panchromatic remote sensing image, a multispectral image and a thermal infrared remote sensing image.

3. The target real-time detection system according to claim 2, characterized in that: The target recognition module comprises: An image type determination unit, used to determine the image type of the target image; The recognition algorithm calling unit is used to call a target recognition algorithm of a corresponding type according to the image type to perform target recognition on the target image.

4. The target real-time detection system according to claim 1, characterized in that: The supercomputing board also includes: The shooting control module is used to determine in real time whether the drone has reached the imaging area, and control the image acquisition device to acquire the target image when the drone reaches the imaging area.

5. The target real-time detection system according to claim 4, characterized in that: The supercomputing board also includes: A shooting mode configuration module, used to obtain the shooting mode configuration of the image acquisition device; The shooting control module is specifically used to: determine in real time whether the drone has reached the imaging area, and control the image acquisition device to acquire the target image according to the corresponding shooting mode when the drone reaches the imaging area.

6. The target real-time detection system according to claim 1, characterized in that: The system further comprises: A wireless communication module, communicating with the supercomputing board, for uploading the data acquired by the supercomputing board to a server in real time, so that the mobile terminal can call the data from the server in real time; The supercomputing board also includes: The data transmission module communicates with the wireless communication module and is used to send the data to be uploaded acquired by the supercomputing board to the wireless communication module.

7. The target real-time detection system according to claim 6, characterized in that: The supercomputing board also includes: The data compression module communicates with the data transmission module and is used to compress the data to be uploaded obtained by the supercomputing board and send it to the wireless communication module.

8. The target real-time detection system according to any one of claims 1 to 7, characterized in that: The target recognition module comprises: The image preprocessing unit is used to perform at least one of image registration, image stitching and radiation equation correction on the decoded target image.

9. The target real-time detection system according to any one of claims 1 to 7, characterized in that: The target recognition module comprises: The coordinate conversion unit is used to convert the target image coordinates obtained by performing real-time target recognition using the decoded target image into geographic coordinates based on a pre-established conversion model between image coordinates and geographic coordinates.

10. A real-time target detection method based on an unmanned aerial vehicle platform, characterized in that: A supercomputing board card applied to a target real-time detection system, the method comprising: Get the target image; Decoding the target image in real time to obtain decoded data; Based on the image type of the target image, calling a target recognition algorithm of a corresponding type to perform target recognition on the target image to obtain a target recognition result; The decoded data and the target recognition result are uploaded to a server.

Images