A target detection system for drone aerial images

Abstract

This utility model discloses a drone aerial image target detection system. The camera is not directly connected to the microcontroller unit (MCU) and the parallel computing unit (GPU), but is relayed through a MIPI CSI-2 module. The MIPI CSI-2 module provides an interface for adapting to cameras of different specifications and outputs the aerial data format required by the GPU, facilitating connection between cameras of different specifications and the GPU. Based on the control commands of the MCU, the MIPI CSI-2 module selects aerial images from cached data and inputs them to the GPU, reducing the computational and storage burden on the GPU.

Description

Technical Field

[0001] This utility model belongs to the field of unmanned aerial vehicles (UAVs), specifically relating to a UAV aerial image target detection system. Background Technology

[0002] As a representative device of intelligent transformation that perfectly combines informatization and mechanization, unmanned aerial vehicles (UAVs) fully embody the technical characteristics of intelligent equipment: unmanned operation, networking, informatization, and non-contact operation. Their demand in socio-economic and military applications is growing stronger, and their applications are becoming increasingly widespread. With the increasing density of UAV usage, the requirements for UAV intelligence are also rising. Target detection in UAV aerial imagery, as the foundation of UAV intelligence technology, is receiving increasing attention.

[0003] In recent years, the main research direction in object detection technology has been based on deep learning. Deep learning-based object detection algorithms are mainly divided into two categories: two-stage detection algorithms and one-stage detection algorithms. Two-stage detection algorithms first generate candidate bounding boxes for potential targets, and then classify and regress these boxes. One-stage detection algorithms do not require generating candidate regions; they directly locate and classify the target from the input image. Both types of algorithms require dedicated parallel computing equipment. Therefore, running aerial image object detection algorithms independently on a UAV requires designing circuits that integrate parallel computing equipment.

[0004] Furthermore, drone aerial cameras continuously output image frames during operation, but the performance of parallel computing devices that can be integrated on drones is limited by size and other constraints, making it difficult to process each image frame in real time. The computational delay accumulates over time, causing the target detection results to lose real-time performance. At the same time, it generates a large amount of data storage and retrieval processes, exacerbating the energy consumption and complexity of the target detection algorithm. Utility Model Content

[0005] The purpose of this invention is to provide a target detection system for drone aerial images, which enables the drone aerial image target detection system to control the data entering a parallel computing device.

[0006] The purpose of this utility model is achieved through the following technical means: a target detection system for drone aerial images, comprising,

[0007] Cameras are used to capture and output aerial images;

[0008] The MIPI CSI-2 module is electrically connected to the camera. The camera inputs the aerial data stream into the MIPI CSI-2 module for transcoding and temporary storage of the aerial images captured by the camera.

[0009] The parallel computing unit (GPU), electrically connected to the MIPI CSI-2 module, is used for target detection in aerial images.

[0010] The microcontroller unit (MCU) is electrically connected to the camera, the MIPI CSI-2 module, and the parallel computing unit (GPU) to control the working status of these three components.

[0011] The power module is electrically connected to and supplies power to the camera, MIPI CSI-2 module, parallel computing unit GPU, and microcontroller unit MCU.

[0012] The camera is connected to the MIPI CSI-2 module via an I2C interface and a MIPI interface.

[0013] The parallel computing unit GPU is connected to the MIPI CSI-2 module via the DP / DN interface and the CLKp / CLKn interface.

[0014] The camera is connected to the microcontroller unit (MCU) via a GPIO interface; the MIPI CSI-2 module is connected to the MCU via a GPIO interface and an I2C port.

[0015] The microcontroller unit (MCU) is also connected to external devices via serial port and signal connection cables.

[0016] The parallel computing unit (GPU) and the microcontroller unit (MCU) are connected via PCIe data lines and GPIO lines.

[0017] The advantages of this invention are as follows: the camera is not directly connected to the microcontroller unit (MCU) and the parallel computing unit (GPU), but is relayed through the MIPI CSI-2 module; the MIPI CSI-2 module provides an interface for adapting to cameras of different specifications, outputs the aerial data format required by the parallel computing unit GPU, and facilitates the connection between cameras of different specifications and the parallel computing unit GPU; the MIPI CSI-2 module selects aerial images from the cached data and inputs them to the parallel computing unit GPU according to the control instructions of the MCU, thereby reducing the computational and storage burden on the parallel computing unit GPU;

[0018] The MCU can transmit more accurate instructions to the MIPI CSI-2 module based on the working status and calculation results of the parallel computing unit GPU, select specified aerial images from the cached data and input them into the parallel computing unit GPU. Based on the changes of target objects in the UAV aerial images, the MCU can more accurately extract the aerial images that need to be detected and calculated, further reducing the computational burden of the parallel computing unit GPU, saving energy consumption, and providing more accurate target information for UAV intelligent algorithms. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the circuit structure of this utility model;

[0020] Figure 2 This is a schematic diagram of the control logic of this utility model;

[0021] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Detailed Implementation

[0022] A target detection system for drone aerial images, comprising,

[0023] Cameras are used to capture and output aerial images;

[0024] The MIPI CSI-2 module is electrically connected to the camera. The camera inputs the aerial data stream into the MIPI CSI-2 module for transcoding and temporary storage of the aerial images captured by the camera.

[0025] The parallel computing unit (GPU), electrically connected to the MIPI CSI-2 module, is used for target detection in aerial images; that is, it runs a deep learning-based target detection algorithm to identify targets in aerial images.

[0026] The microcontroller unit (MCU) is electrically connected to the camera, the MIPI CSI-2 module, and the parallel computing unit (GPU) to control the working status of these three components.

[0027] The power module is electrically connected to and supplies power to the camera, MIPI CSI-2 module, parallel computing unit GPU, and microcontroller unit MCU.

[0028] The camera is connected to the MIPI CSI-2 module via an I2C interface and a MIPI interface.

[0029] The parallel computing unit GPU is connected to the MIPI CSI-2 module via the Dp / Dn interface and the CLKp / CLKn interface.

[0030] The camera is connected to the microcontroller unit (MCU) via a GPIO interface; the MIPI CSI-2 module is connected to the MCU via a GPIO interface and an I2C port.

[0031] The microcontroller unit (MCU) is also connected to external devices via serial port and signal connection cables.

[0032] The parallel computing unit (GPU) and the microcontroller unit (MCU) are connected via PCIe data lines and GPIO lines.

[0033] like Figure 1As shown, in this embodiment, the camera model is OV2640. The camera and the microcontroller unit (MCU) are connected via a GPIO interface: the MCU connects to the camera's VSYNC signal output terminal via the GPIO interface. When the rising or falling edge of the VSYNC signal is detected, the MCU can know that a new frame of image acquisition has begun or ended. The camera and the MCU are also connected via a GPIO interface and a TRIGGER interface. The MCU controls the camera's operating state through the TRIGGER interface, using the rising edge signal to control the camera to take a picture or start recording video.

[0034] The microcontroller unit (MCU) is model STM32F411CEU6. The MCU connects to the MIPI CSI-2 module via a GPIO interface (used for transmitting control signals): the MCU outputs a specific level signal (such as a high-level pulse) through the GPIO interface to trigger the MIPI CSI-2 module to start transmitting the current frame of the image; when the MIPI CSI-2 module completes the transmission of a frame, it pulls the pin high or low to notify the MCU that the transmission is complete.

[0035] The MIPI CSI-2 module is model STMIPID02. The microcontroller unit (MCU) connects to the MIPI CSI-2 module via an I2C port: the MCU sends configuration commands to the MIPI CSI-2 module through the I2C bus to set its operating mode; the MCU can also read relevant information from the MIPI CSI-2 module's status register via I2C to understand its operating status. The MCU sends a signal to the MIPI CSI-2 module's RESET interface to restore it to its initial normal operating state when an malfunction occurs, ensuring it can correctly receive and process data from the camera and transmit data to the parallel computing unit (GPU).

[0036] The camera and the MIPI CSI-2 module are connected via an I2C port, a MIPI interface, and a RESET interface. I2C is a bidirectional serial communication bus used by the camera to transmit configuration and operational status information to the MIPI CSI-2 interface, which in turn transmits this information to the microcontroller unit (MCU). Simultaneously, the MCU can send various configuration commands to the camera via the MIPI CSI-2's I2C interface. The MIPI interface is used for high-speed image data transmission, employing differential signal transmission to transmit raw image data captured by the camera to the MIPI CSI-2 interface at extremely high rates. The RESET interface activates when there is an malfunction in the camera's internal MIPI data transmission module or related circuitry. The MIPI CSI-2 module uses the RESET interface to restore the camera's MIPI interface to its initial state, allowing it to resume normal operation.

[0037] The parallel computing unit (GPU) is an NVIDIA Jetson Nano. There are two data connection lines between the GPU and the MIPI CSI-2 module: a DP / DN differential data line for data transmission and CLKp / CLKn for clock signals. There are also two data connection lines between the GPU and the microcontroller unit (MCU): a PCIe data line for data and result transmission between the MCU and GPU; and a GPIO line for controlling the GPU's functions and reading its status information.

[0038] The control logic of this utility model is as follows: Figure 2 As shown, at the beginning, all units are initialized. The microcontroller unit (MCU) determines whether the MIPI CSI-2 module and the parallel computing unit (GPU) are idle, i.e., whether the camera should output data, based on the storage status of the MIPI CSI-2 module and the working status of the parallel computing unit (GPU). During operation, based on the working status of the parallel computing unit (GPU) and the output target detection algorithm results, the MCU calculates the next aerial image that the GPU needs to process, controls the MIPI CSI-2 module to output the specified data to the GPU, clears the unnecessary data in the buffer, and continues to receive data output from the camera.

[0039] Specifically, after the camera starts working, it continuously transmits aerial image data to the MIPI CSI-2 module, which caches the data in a time sequence for later use. When the microcontroller unit (MCU) determines that the parallel computing unit (GPU) is idle, it controls the MIPI CSI-2 module to input the latest aerial image into the GPU, runs a deep learning target detection algorithm, and clears other data from the MIPI CSI-2 module's cache. After the GPU returns the target detection result, the MCU determines the time difference between the next aerial image to be detected and the currently detected image based on the target detection result, and controls the MIPI CSI-2 module to input the required aerial image into the GPU, while clearing other data from the MIPI CSI-2 module's cache. Simultaneously, the MCU transmits the target detection result to other circuits via serial port and signal connection lines as needed.

[0040] In practical use, the microcontroller unit (MCU) controls the data flow between components based on their operating states. When the parallel computing unit (GPU) is operational, the aerial image data captured by the camera is temporarily stored in the MIPI CSI-2 module. After the GPU completes target detection calculations, the MCU controls the MIPI CSI-2 module to select a specific aerial image for input to the GPU based on the detection results. This prevents the GPU from accumulating computational delays due to its inability to process all data in real time, thus ensuring the target detection results are timely. Furthermore, the MCU locks down subsequent key images based on the target detection results, further improving the drone's intelligence level.

Claims

1.A UAV aerial image target detection system, characterized in that: The utility model relates to a kind of unmanned aerial vehicle image processing system, including, Camera, for collecting and output aerial image; Mipi CSI-2 module, electrically connected with camera, camera inputs aerial data stream into mipi CSI-2 module, for transcoding and temporary storage aerial image that camera shoots; Parallel computing unit GPU, electrically connected with mipi CSI-2 module, for target detection to aerial image; Micro control unit MCU, respectively with camera, mipi CSI-2 module and parallel computing unit GPU electrically connected, for controlling the working state of three; Power module, electrically connected with camera, mipi CSI-2 module, parallel computing unit GPU, micro control unit MCU and power supply for it. 2.The UAV aerial image target detection system of claim 1, wherein: The camera and the mipi CSI-2 module are connected through the I2C interface and the mipi interface. 3.The UAV aerial image target detection system of claim 1, wherein: The parallel computing unit GPU and the mipi CSI-2 module are connected through the DP / DN interface and the CLKp / CLKn interface. 4.The UAV aerial image target detection system of claim 1, wherein: The camera and the micro control unit MCU are connected through the GPIO interface;The mipi CSI-2 module and the micro control unit MCU are connected through the GPIO interface and the I2C port. 5.The UAV aerial image target detection system of claim 1, wherein: The micro control unit MCU is also connected with external equipment through the serial port connection line and the signal connection line. 6.The UAV aerial image target detection system of claim 1, wherein: The parallel computing unit GPU and the micro control unit MCU are connected through the PCLE data line and the GPIO line.

Images