A method, device, equipment and medium for target detection based on drone
By introducing the redundant detection module of brain neurons into the YOLO model, the model structure is optimized, and the problem of high computational complexity of the object detection algorithm model and excessive parameters is solved, the detection efficiency and adaptability are improved, and more efficient goals are achieved Test.
Patent Information
- Application Number
- CN202411186337.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-28
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2044-08-28
AI Technical Summary
The existing object detection algorithm model has high computational complexity and too many model parameters, resulting in low detection efficiency and difficulty in effectively deploying in resource-constrained environments.
The improved YOLO model is adopted to replace the original C3 module through the brain neuron redundant detection module, optimize the model structure, reduce the calculation complexity and model parameters, and improve the detection efficiency.
It significantly improves the efficiency of target detection, reduces the amount of calculation and parameters, and improves the adaptability and detection performance of the model on the mobile terminal.
Smart Images

Figure CN118711091B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of target detection technology, and in particular to a target detection method, device, equipment and medium based on an unmanned aerial vehicle. Background Art
[0002] In modern urban management and security systems, drones are increasingly being used. Drones can detect and track targets in complex environments. In urban environments, drones can identify and track a variety of targets, such as pedestrians, vehicles (including bicycles, electric vehicles, tricycles, cars, trucks), etc. Therefore, they can perform tasks such as urban monitoring, security patrols, and search and rescue operations, and can ensure the safe and efficient operation of tasks.
[0003] (1) Urban monitoring: In urban environments, traffic monitoring is an important application scenario. Drones can monitor traffic flow in real time, identify traffic violations, and assist traffic management departments in making decisions. For example, in the event of traffic congestion, drones can fly high in the sky to capture panoramic images, identify and track the movement path of each car, thereby facilitating the development of effective traffic diversion plans.
[0004] (2) Security patrol: In urban security, drones can conduct regular patrols to identify and track suspicious people or vehicles. In large-scale events or public gatherings, drones can monitor crowd dynamics in real time, identify abnormal behavior and issue timely warnings to ensure public safety.
[0005] (3) Search and rescue operations: In disaster relief, drones can quickly cover large areas and identify trapped people and damaged vehicles. Through the high-altitude perspective of drones, targets can be quickly located, guiding the actions of ground rescue teams and improving rescue efficiency.
[0006] Drones usually use target detection algorithm models to complete target detection, but current target detection algorithm models usually rely on deep neural networks (such as the YOLO series, Faster R-CNN, etc.). These target detection algorithm models have excellent performance in recognition accuracy and feature richness. However, they usually have problems such as high computational complexity, too many model parameters, and low detection efficiency. Summary of the invention
[0007] The purpose of this application is to provide a drone-based target detection method, device, equipment and medium, which can solve the problems of high computational complexity and excessive model parameters in existing target detection algorithm models and significantly improve detection efficiency.
[0008] To achieve the above objectives, this application provides the following solutions.
[0009] In a first aspect, the present application provides a target detection method based on a drone, and the target detection method based on a drone comprises:
[0010] Acquire a captured image; the captured image is obtained by using a drone to capture the urban scene to be detected;
[0011] Taking the captured image as input, using the trained target detection model to identify the detection target in the captured image to obtain a target detection result; the detection target includes pedestrians and vehicles; the target detection result includes the category and position of each detection target in the captured image;
[0012] The trained target detection model adopts an improved YOLO model, which is obtained by replacing the C3 module in the YOLO model with a brain neuron redundancy detection module, and the brain neuron redundancy detection module includes:
[0013] The first convolution block is used to convolve the input feature to obtain the first feature;
[0014] A segmentation layer, used to segment the first feature to obtain a second feature and a third feature;
[0015] A convolution residual block, used to convolve the second feature to obtain a fourth feature;
[0016] A splicing layer, used for splicing the third feature and the fourth feature to obtain a fifth feature;
[0017] The second convolution block is used to convolve the fifth feature to obtain an output feature.
[0018] Optionally, the output of the first convolution block is connected to the input of the segmentation layer; the first output of the segmentation layer is connected to the first input of the splicing layer; the second output of the segmentation layer is connected to the input of the convolution residual block; the output of the convolution residual block is connected to the second input of the splicing layer; the output of the splicing layer is connected to the input of the second convolution block.
[0019] Optionally, the first convolution block includes several first convolution layers connected in sequence; the second convolution block includes several second convolution layers connected in sequence; the convolution residual block includes several third convolution layers, residual units and several fourth convolution layers connected in sequence, the residual unit includes several fifth convolution layers and an addition layer connected in sequence, and the input end of the addition layer is also connected to the output end of the last third convolution layer.
[0020] Optionally, the number of the first convolutional layers is 1; the number of the second convolutional layers is 1; the number of the third convolutional layers is 1; and the number of the fourth convolutional layers is 1.
[0021] Optionally, the number of input channels of the first convolutional layer is , the number of output channels of the first convolutional layer is ;in, , is the number of output channels of the brain neuron redundancy detection module, To expand the ratio;
[0022] The number of input channels of the second convolutional layer is , the number of output channels of the second convolutional layer is ;in, , is the scaling ratio;
[0023] The number of input channels of the third convolutional layer is , the number of output channels of the third convolutional layer is ;
[0024] The number of input channels of the fourth convolutional layer is , the number of output channels of the fourth convolutional layer is ;
[0025] The number of input channels of the fifth convolutional layer is , the number of output channels of the fifth convolutional layer is .
[0026] Optionally, the convolution kernel size of the first convolution layer is 1; the convolution kernel size of the second convolution layer is 1; the convolution kernel size of the third convolution layer is 3; the convolution kernel size of the fourth convolution layer is 1; and the convolution kernel size of the fifth convolution layer is 3.
[0027] Optionally, the target detection result also includes a confidence score of each detected target in the captured image.
[0028] In a second aspect, the present application provides a target detection device based on a drone, and the target detection device based on a drone includes:
[0029] An image acquisition module is used to acquire a captured image; the captured image is obtained by using a drone to capture the urban scene to be detected;
[0030] An image recognition module, used to use the captured image as input, and use the trained target detection model to identify the detection target in the captured image to obtain a target detection result; the detection target includes pedestrians and vehicles; the target detection result includes the category and position of each detection target in the captured image;
[0031] The trained target detection model adopts an improved YOLO model, which is obtained by replacing the C3 module in the YOLO model with a brain neuron redundancy detection module, and the brain neuron redundancy detection module includes:
[0032] The first convolution block is used to convolve the input feature to obtain the first feature;
[0033] A segmentation layer, used to segment the first feature to obtain a second feature and a third feature;
[0034] A convolution residual block, used to convolve the second feature to obtain a fourth feature;
[0035] A splicing layer, used for splicing the third feature and the fourth feature to obtain a fifth feature;
[0036] The second convolution block is used to convolve the fifth feature to obtain an output feature.
[0037] In a third aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any of the above-mentioned drone-based target detection methods.
[0038] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-mentioned drone-based target detection methods.
[0039] According to the specific embodiments provided in this application, this application discloses the following technical effects:
[0040] The present application provides a target detection method, device, equipment and medium based on drone, which improves the existing YOLO model. Specifically, the C3 module in the YOLO model is replaced by the brain neuron redundancy detection module to obtain an improved YOLO model, and the improved YOLO model is used to identify the detection target in the captured image obtained by using the drone to shoot the urban scene to be detected, and the target detection result is obtained. The target detection result includes the category and position of each detection target in the captured image. By improving the model structure, the problems of high computational complexity and excessive model parameters in the existing target detection algorithm model can be solved, and the detection efficiency can be significantly improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0042] Figure 1 This is a diagram of the application environment of a drone-based target detection method provided in Example 1 of the present application.
[0043] Figure 2 A flowchart of a target detection method based on a drone provided in Example 1 of the present application.
[0044] Figure 3 This is a schematic diagram of the structure of the brain neuron redundancy detection module provided in Example 1 of the present application.
[0045] Figure 4 A schematic diagram of the functional modules of a drone-based target detection device provided in Example 2 of the present application.
[0046] Figure 5 A schematic diagram of the structure of a computer device provided in Example 3 of the present application. DETAILED DESCRIPTION
[0047] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0048] Example 1
[0049] The target detection method based on drone provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal communicates with the server through the network. The data storage system can store the data that the server needs to process. The data storage system can be set up separately, integrated on the server, or placed on the cloud or other servers. The terminal can send the captured image to be processed to the server. After the server receives the captured image to be processed, for the captured image to be processed, the server uses the captured image as input, and uses the trained target detection model to identify the detection target in the captured image to obtain the target detection result. The server can feed back the target detection result for the captured image to the terminal.
[0050] In addition, in some embodiments, the drone-based target detection method can also be implemented independently by a server or a terminal. For example, the terminal can directly identify the captured images to be processed, or the server can obtain the captured images to be processed from a data storage system and identify the captured images to be processed.
[0051] The terminal may be, but is not limited to, various desktop computers, laptops, smart phones, tablet computers, IoT devices, and portable wearable devices. The server may be implemented as an independent server or a server cluster consisting of multiple servers, or may be a cloud server.
[0052] like Figure 2 As shown, a target detection method based on a drone is provided. The method is executed by a computer device, and can be executed by a computer device such as a terminal or a server alone, or by a terminal and a server together. In the embodiment of the present application, the method is applied to Figure 1 Taking the server in as an example, the target detection method based on drone includes the following steps.
[0053] Step S1, obtaining a captured image; the captured image is obtained by using a drone to capture a city scene to be detected.
[0054] Step S2, taking the captured image as input, using the trained target detection model to identify the detection target in the captured image to obtain a target detection result; the detection target includes pedestrians and vehicles; the target detection result includes the category and position of each detection target in the captured image.
[0055] The trained target detection model adopts an improved YOLO model, which is obtained by replacing the C3 module in the YOLO model with a brain neuron redundancy detection module, and the brain neuron redundancy detection module includes:
[0056] The first convolution block is used to convolve the input feature to obtain the first feature;
[0057] A segmentation layer, used to segment the first feature to obtain a second feature and a third feature;
[0058] A convolution residual block, used to convolve the second feature to obtain a fourth feature;
[0059] A splicing layer, used for splicing the third feature and the fourth feature to obtain a fifth feature;
[0060] The second convolution block is used to convolve the fifth feature to obtain an output feature.
[0061] By implementing the above-mentioned steps S1 to S2, this embodiment improves the existing YOLO model. Specifically, the C3 module in the YOLO model is replaced by the brain neuron redundancy detection module to obtain an improved YOLO model, and the improved YOLO model is used to identify the detection target in the captured image to obtain the target detection result. By improving the model structure, the problems of high computational complexity and too many model parameters in the existing target detection algorithm model can be solved, and the detection efficiency can be significantly improved.
[0062] In S1, the city scene to be detected is determined according to user needs, which is the area that the user wants to detect. After the city scene to be detected is determined according to the user's needs, the city scene to be detected is photographed by a drone to obtain a photographed image.
[0063] In S2, the captured image is directly used as input, and the detection target in the captured image is identified using the trained target detection model to obtain the target detection result. The detection targets include pedestrians and vehicles. The vehicles can be bicycles, electric vehicles, tricycles, cars, trucks, etc. The target detection result includes the category and position of each detection target in the captured image. The category represents whether the detection target is a pedestrian or a bicycle, electric vehicle, tricycle, car or truck among vehicles. The position can be represented by bounding box coordinates. The bounding box coordinates can be the upper left corner coordinates and the lower right corner coordinates of the rectangular bounding box.
[0064] The target detection result of this embodiment also includes a confidence score for each detected target in the captured image, which indicates the credibility of the target detection result output by the trained target detection model.
[0065] In this embodiment, the trained target detection model adopts the improved YOLO model. The improved YOLO model is obtained by replacing the C3 module in the YOLO model with the brain neuron redundancy detection module (which can be referred to as NeuroRepDet module for short). The YOLO model can be any YOLO model containing the C3 module, such as the YOLOv5 model and the YOLOv8 model. In the architecture of the YOLOv5 model and the YOLOv8 model, the C3 module is mainly used for feature extraction and fusion. The brain neuron redundancy detection module proposed in this embodiment is used to replace the C3 module in the YOLOv5 model and the YOLOv8 model. All C3 modules appearing in the YOLOv5 model and the YOLOv8 model are replaced with the brain neuron redundancy detection module to improve the performance of the model. It should be noted that the brain neuron redundancy detection module is introduced into the YOLOv5 model and the YOLOv8 model to replace the C3 module. This replacement method belongs to the existing technology, because the YOLOv5 model and the YOLOv8 model themselves have a modular design, allowing users to replace or improve specific modules (such as the C3 module) while maintaining the overall structure of the model.
[0066] like Figure 3 As shown, the functions of each structure in the brain neuron redundancy detection module are as follows.
[0067] The first convolutional block (i.e. Figure 3 Conv1 in ), which is used to transform the input features (i.e. Figure 3 Convolution is performed on the input of the vector to obtain the first feature.
[0068] Segmentation layer (i.e. Figure 3 Spilt in ), is used to segment the first feature to obtain the second feature and the third feature.
[0069] Convolutional residual block (i.e. Figure 3 Conv3, Block, and Conv4 in ) are used to convolve the second feature to obtain the fourth feature.
[0070] The stitching layer (i.e. Figure 3 Concat in ) is used to concatenate the third feature and the fourth feature to obtain the fifth feature.
[0071] The second convolutional block (i.e. Figure 3 Conv2 in ) is used to convolve the fifth feature to obtain the output feature (i.e. Figure 3 in the Output).
[0072] The connection relationship between the various structures in the brain neuron redundancy detection module is as follows: the output end of the first convolution block is connected to the input end of the segmentation layer, the first output end of the segmentation layer is connected to the first input end of the splicing layer, the second output end of the segmentation layer is connected to the input end of the convolution residual block, the output end of the convolution residual block is connected to the second input end of the splicing layer, and the output end of the splicing layer is connected to the input end of the second convolution block.
[0073] The specific structures of each structure in the brain neuron redundancy detection module are as follows: the first convolution block includes several first convolution layers connected in sequence, the second convolution block includes several second convolution layers connected in sequence, the convolution residual block includes several third convolution layers, residual units and several fourth convolution layers connected in sequence, the residual unit includes several fifth convolution layers and an addition layer connected in sequence, the input end of the addition layer is also connected to the output end of the last third convolution layer, that is, the first fifth convolution layer processes the output feature map of the last third convolution layer, and the number of output channels is maintained at , the subsequent fifth convolutional layer receives the output feature map of the previous fifth convolutional layer in turn and processes it, and continues to maintain the number of output channels as The addition layer adds the output feature map of the last fifth convolutional layer to the output feature map of the last third convolutional layer (through skip connection) to achieve residual connection. This addition operation can make the network deeper while alleviating the gradient vanishing problem.
[0074] Among them, the number of the first convolutional layer is 1, the number of the second convolutional layer is 1, the number of the third convolutional layer is 1, and the number of the fourth convolutional layer is 1. Of course, other numbers can also be selected according to needs.
[0075] The initialization process of the brain neuron redundancy detection module is as follows: Initialize the following parameters: Input channel number , Output Channels , expansion ratio , Zoom and the number of fifth convolutional layers in the residual unit .
[0076] The settings of each convolutional layer in the brain neuron redundancy detection module are as follows.
[0077] The first convolutional layer: the number of input channels Convert to number of channels , the convolution kernel size is 1. Among them, the number of hidden channels , is the floor function.
[0078] The second convolutional layer: the number of channels after the concatenation layer Convert to output channel number , the convolution kernel size is 1. Among them, the number of intermediate channels .
[0079] The third convolutional layer: the number of channels is changed from Transformed to , the convolution kernel size is 3.
[0080] The fourth convolutional layer: the number of channels is changed from Transformed to , the convolution kernel size is 1.
[0081] The residual unit contains There are five convolutional layers, each of which increases the number of channels from Transformed to , the convolution kernel size is 3.
[0082] In this embodiment, the number of channels of each convolutional layer is set as follows.
[0083] The number of input channels of the first convolutional layer is , the number of output channels of the first convolutional layer is , , is the number of output channels of the brain neuron redundancy detection module, To expand the ratio.
[0084] The number of input channels of the second convolutional layer is , the number of output channels of the second convolutional layer is , , is the scaling factor.
[0085] The number of input channels of the third convolutional layer is , the number of output channels of the third convolutional layer is .
[0086] The number of input channels of the fourth convolutional layer is , the number of output channels of the fourth convolutional layer is .
[0087] The number of input channels of the fifth convolutional layer is , the number of output channels of the fifth convolutional layer is .
[0088] In this embodiment, the convolution kernel size of each convolution layer is set as follows.
[0089] The convolution kernel size of the first convolution layer is 1, the convolution kernel size of the second convolution layer is 1, the convolution kernel size of the third convolution layer is 3, the convolution kernel size of the fourth convolution layer is 1, and the convolution kernel size of the fifth convolution layer is 3.
[0090] The existing target detection algorithm model is specific to the model structure, such as YOLOv5 and other target detection architectures based on the combination of backbone network (Backbone) and detection head (Head). Although it effectively improves the detection accuracy, it is still limited by the large computational overhead and model complexity, especially the deployment in the mobile terminal needs to be improved, which limits its practical application in resource-constrained environments. The trained target detection model proposed in this embodiment is based on the brain's redundant information processing mechanism. By introducing the brain neuron redundant detection module, it aims to optimize the limitations of the existing technology in terms of computational complexity, model parameters and mobile terminal deployment, as follows.
[0091] The objective defects of existing target detection algorithm models are as follows.
[0092] (1) High computational complexity
[0093] The existing target detection algorithm models have high computational complexity and usually require a lot of computing resources. Especially in high-resolution images and complex scenes, the computational complexity increases significantly, limiting their widespread use in real-time applications.
[0094] (2) Too many model parameters
[0095] Existing object detection algorithm models often have a large number of model parameters, which not only increases the storage and transmission costs of the model, but also affects the inference speed and efficiency of the model on resource-constrained devices.
[0096] (3) Difficult to deploy on mobile devices
[0097] Due to the limitations of computing and storage resources, the real-time performance and stability (may freeze or crash) of existing target detection algorithm models on mobile terminals (such as smartphones and embedded systems) are greatly challenged, limiting the scope and effectiveness of their practical applications.
[0098] The lightweight target detection model based on the brain's redundant information processing mechanism proposed in this embodiment has the following advantages and can solve the objective defects of the prior art.
[0099] (1) Reduce computational complexity and model parameter count
[0100] In order to address the problems of high computational complexity and too many model parameters, this embodiment introduces redundant feature generation technology to simulate the redundancy of neurons in the brain. By generating and utilizing redundant feature maps, the computational complexity and the number of model parameters are effectively reduced, thereby reducing the computational overhead of the model in the inference stage and improving detection efficiency.
[0101] Specifically, the existing target detection algorithm model usually relies on a fixed feature extraction network structure, which has high computational complexity, many model parameters and lacks effective use of redundant features. This embodiment introduces a brain neuron redundancy detection module, which allows flexible deployment and optimization on different target detection architectures (such as YOLOv5, YOLOv8). The brain neuron redundancy detection module replaces the traditional single feature extraction mode by generating and utilizing multiple redundant feature maps, effectively reducing the computational complexity and model parameter amount, and improving the detection accuracy and reasoning speed. The specific implementation method is: during the initial feature extraction, the input feature is converted into The first characteristic of the channel, and use The function splits the first feature into two The second and third features of the channel, the second feature is further processed by the third convolutional layer, the residual unit and the fourth convolutional layer, and the third feature remains unchanged. By introducing the residual unit, multiple redundant feature maps are generated. The number of redundant feature maps is the same as the number of the fifth convolutional layer included in the residual unit, which provides diversity for subsequent feature extraction, effectively utilizes redundant information, and realizes redundant feature processing.
[0102] (2) Enhanced gradient flow and feature extraction capabilities
[0103] This embodiment introduces a brain neuron redundancy detection module to simulate the efficient signal transmission mechanism between neurons in the brain, improves the capabilities of feature extraction and gradient flow, and further improves the accuracy and robustness of the target detection model.
[0104] Specifically, existing target detection algorithm models usually use simple convolution operations in gradient propagation and feature fusion, which limits the efficiency of information transmission and fusion. This embodiment introduces a brain neuron redundancy detection module to simulate the efficient signal transmission mechanism between brain neurons, which not only enhances the feature extraction capability, but also optimizes the flow of gradients in the network, thereby improving the overall robustness and accuracy. The specific implementation method is as follows: when processing and fusing feature maps, the second feature is processed by the third convolution layer, and then multiple convolutions are performed by the residual unit, and finally by the fourth convolution layer. The convolution operation in the residual unit not only extracts more features, but also enhances the efficiency of information transmission and fusion through the connection operation of the feature map, and optimizes the gradient flow and feature extraction capabilities.
[0105] (3) Lightweight design, compatible with mobile terminals
[0106] This embodiment achieves efficient use of computing resources by optimizing the model structure and parameter settings, making it more suitable for mobile terminals, thereby solving the performance limitation problem of the prior art in mobile terminal scenarios.
[0107] Specifically, although the existing target detection algorithm model has a certain lightweight design, it still faces the problem that it is difficult to further reduce the model parameters and computational complexity. In this embodiment, by optimizing the model structure and parameter settings, the brain neuron redundancy detection module can flexibly adjust its size (through scaling and expansion ratios) to take into account the needs of different target detection architectures. This design enables the model to perform well on a variety of hardware platforms, thereby adapting to different scenarios and application requirements. The specific implementation method is: in flexible parameter settings, through scaling ratios and expansion ratio These two parameters flexibly adjust the number of channels in the middle layer and the hidden layer, so that the model can be scaled according to different needs. This parameter controls the number of intermediate convolutional layers. You can increase or decrease the convolutional layers according to actual needs to balance the computational complexity and detection accuracy. When the final feature map is generated, all processed feature maps are concatenated in the concatenation layer through the concatenation function. They are connected in the channel dimension to form a new feature map (i.e., the fifth feature), and the final output features are generated through the second convolutional layer. This process ensures the lightweight design of the model while taking into account efficient feature extraction and information transmission.
[0108] Hereinafter, this embodiment gives a specific example to illustrate the superiority of the target detection method of this embodiment.
[0109] (1) Dataset selection
[0110] This example is based on the VisDrone dataset for experiments. The VisDrone dataset is a public drone-view dataset that contains a large number of images taken in complex environments. These images contain a variety of target types, such as pedestrians, cars, trucks, bicycles, and tricycles, which can fully simulate the working scenarios of drones in actual applications. In this example, pedestrians, cars, and bicycles are selected as detection targets.
[0111] (2) Overall architecture of the target detection model
[0112] The target detection model in this example is an improved YOLO model obtained by replacing the C3 module in the YOLO model with the brain neuron redundancy detection module. Its overall architecture includes two main parts: the backbone network (Backbone) and the detection head (Head). The backbone network is responsible for feature extraction, and the detection head is responsible for target classification and positioning. The partial architecture of the target detection model is as follows.
[0113] (2.1) Backbone
[0114] 1) Initial convolutional layer and downsampling layer
[0115] Conv layer (64, 6, 2, 2): initial convolution processing, extracting basic features and downsampling.
[0116] Conv layer (128, 3, 2): further convolution and downsampling.
[0117] It should be noted that (64, 6, 2, 2) and (128, 3, 2) are the number of output channels of the output feature map of the convolution layer, the size of the convolution kernel, the stride of the convolution operation, and whether to pad. If the last digit is omitted, it means no padding. For example, 64: the number of output channels, indicating that the number of channels of the output feature map generated after the convolution operation is 64. 6: the size of the convolution kernel is 6 6, that is, the height and width of the convolution kernel are both 6. 2 (the first 2): stride, which means that the convolution kernel moves on the feature map with a step length of 2, which will cause the feature map size to be reduced, achieving the effect of downsampling. 2 (the second 2): padding, which means that 2 pixels are padded at the edge of the feature map, so that the convolution operation will not significantly reduce the size of the feature map.
[0118] 2) Feature extraction module
[0119] Brain neuron redundancy detection module: used to enhance feature extraction capabilities.
[0120] Multi-layer Conv and brain neuron redundancy detection module: extract higher-level features layer by layer.
[0121] SPPF (Spatial Pyramid Pooling-Fast) module: adds a spatial pyramid pooling layer to the feature pyramid to further extract features.
[0122] (2.2) Detection head
[0123] Combining multiple layers of features: Combine features at different levels through Upsample and Concat operations.
[0124] Brain neuron redundancy detection module: further processes the combined features.
[0125] RT-DETR Decoder: The final decoding layer that outputs the category and location of the detected target.
[0126] Taking a brain neuron redundancy detection module as an example, this module is used for feature enhancement and extraction in the backbone network. The specific processing flow of the brain neuron redundancy detection module is as follows.
[0127] 1) Input image: The original images in the dataset are input into the target detection model that integrates the brain neuron redundant detection module. These images contain various detection targets (such as pedestrians, cars, bicycles) and complex background information.
[0128] 2) Initial feature extraction: The input image is processed by the module located before the brain neuron redundancy detection module in the target detection model to obtain the input features. The input features are subjected to the first convolution operation through the first convolutional layer to extract the low-level features of the input features to obtain the first features. The low-level features include edge, texture, color and other information.
[0129] 3) Feature enhancement: The extracted first feature is divided into two channel branches, and the second feature of one channel branch is further convolved through the third convolutional layer to enhance the expressive power of the feature.
[0130] 4) Multi-layer convolution processing: The features obtained after the third convolution layer are processed in turn through The fifth convolutional layers perform multi-layer convolution processing, which further extracts and optimizes features to make them more suitable for target detection tasks.
[0131] 5) Final convolution processing: The output features of the last fifth convolutional layer are convolved through the fourth convolutional layer to generate higher-level features and obtain the fourth features.
[0132] 6) Feature concatenation: The third and fourth features of the other channel branch are concatenated together and passed through the second convolutional layer for the final convolution process to generate the output feature.
[0133] (3) Image processing flow
[0134] Taking an image in the VisDrone dataset as an example, the image processing flow is as follows.
[0135] (3.1) Input image
[0136] The input image contains a pedestrian, a car, and a bicycle.
[0137] (3.2) Feature extraction and processing
[0138] The input image passes through the initial convolutional layer and downsampling layer of the backbone network to extract basic features.
[0139] The basic features pass through multiple layers of brain neuron redundancy detection modules and convolutional layers to extract higher-level features layer by layer.
[0140] Finally, multi-scale features are fused through the SPPF module.
[0141] (3.3) Combining multiple layers of features
[0142] Features at different levels are combined together through the Upsample and Concat operations of the detection head.
[0143] After further processing by the brain neuron redundancy detection module, the final feature representation is generated.
[0144] (3.4) Target detection and output
[0145] The final feature representation passes through the RT-DETR Decoder decoding layer to output the target detection result.
[0146] The object detection results include the object category (such as "pedestrian", "car", "bicycle"), bounding box (bounding box coordinates of each detected object), and confidence score.
[0147] That is, the final output of the target detection model includes the following.
[0148] 1) Target Class (Class Labels): The class label of the identified detection target, such as "pedestrian", "car", "bicycle", etc.
[0149] 2) Bounding Boxes: The bounding box coordinates of each detected object, usually expressed as the upper left corner coordinates and lower right corner coordinates of the rectangular bounding box , They are the x-coordinate, y-coordinate of the upper left corner and the x-coordinate, y-coordinate of the lower right corner of the rectangular bounding box.
[0150] 3) Confidence Scores: The target detection model gives a confidence score to each detected target, indicating the degree of confidence in the model's detection results for the target.
[0151] For ease of understanding, an output example of a target detection result is given below.
[0152] 1) Target categories: The detected target categories are “pedestrian”, “car” and “bicycle”.
[0153] 2) Bounding box: Pedestrians: (100, 200, 150, 300). Cars: (250, 150, 400, 350). Bicycles: (500, 220, 550, 280).
[0154] 3) Confidence score: Pedestrian: 0.95. Car: 0.90. Bicycle: 0.85.
[0155] Through the above image processing process, input features are extracted from the original image, and after multi-layer convolution and enhancement processing, the target detection results are finally output, including target category, bounding box and confidence score. These target detection results can be used to identify and track target objects in real-time target detection of drones.
[0156] The experimental results are shown in Table 1 below.
[0157] Table 1 Experimental results
[0158]
[0159] This embodiment experiments on different models, and the experimental results show that: on the YOLOv5 model, by introducing the brain neuron redundancy detection module, the amount of calculation (GFlops) is reduced from 10.7G to 9.9G, a reduction of about 7.48%; the amount of parameters (Params) is reduced from 5.599M to 5.288M, a reduction of about 5.56%; at the same time, mAP (Mean Average Precision) is increased from 0.156 to 0.214, and mAP_50 (Mean Average Precision at IoU 50%, the value of the mean average precision when the recall rate reaches 50%) is increased from 0.279 to 0.372. On the YOLOv8 model, the amount of computation was reduced from 11.7G to 9.7G, a reduction of about 17.09%; the number of parameters was reduced from 6.102M to 5.290M, a reduction of about 13.31%; mAP increased from 0.176 to 0.211, and mAP_50 increased from 0.307 to 0.361.
[0160] By introducing the brain neuron redundancy detection module, the YOLOv5 model and the YOLOv8 model have significantly reduced the amount of calculation and parameters, and the target detection performance (mAP and mAP_50) has also been significantly improved. This shows that the brain neuron redundancy detection module can not only improve the lightweight degree of the model, but also improve its detection performance, proving its effectiveness and superiority in practical applications.
[0161] The embodiment of the present application also provides an application scenario, which applies the above-mentioned drone-based target detection method. Specifically, the drone-based target detection method provided by this embodiment can be applied in urban monitoring scenarios. The urban monitoring scenario includes an image acquisition link, an image processing link, and a monitoring link. The image acquisition link is used to acquire the captured image obtained by shooting the urban scene to be detected. The image processing link is used to identify the captured image and obtain the category and position of each detection target in the captured image. The monitoring link is used to monitor the detection target based on the category and position of each detection target. The drone-based target detection method provided by this embodiment belongs to the image processing link.
[0162] Example 2
[0163] Based on the same inventive concept, the embodiment of the present application also provides a drone-based target detection device for implementing the drone-based target detection method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in the drone-based target detection device embodiment provided below can refer to the above limitations on the drone-based target detection method, and will not be repeated here.
[0164] like Figure 4 As shown, a target detection device based on a drone is provided, and the target detection device based on a drone includes the following modules.
[0165] The image acquisition module M1 is used to acquire a captured image; the captured image is obtained by using a drone to shoot the urban scene to be detected.
[0166] The image recognition module M2 is used to use the captured image as input, and use the trained target detection model to identify the detection target in the captured image to obtain a target detection result; the detection target includes pedestrians and vehicles; the target detection result includes the category and position of each detection target in the captured image.
[0167] The trained target detection model adopts an improved YOLO model, which is obtained by replacing the C3 module in the YOLO model with a brain neuron redundancy detection module, and the brain neuron redundancy detection module includes:
[0168] The first convolution block is used to convolve the input feature to obtain the first feature;
[0169] A segmentation layer, used to segment the first feature to obtain a second feature and a third feature;
[0170] A convolution residual block, used to convolve the second feature to obtain a fourth feature;
[0171] A splicing layer, used for splicing the third feature and the fourth feature to obtain a fifth feature;
[0172] The second convolution block is used to convolve the fifth feature to obtain an output feature.
[0173] Example 3
[0174] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the captured image data to be processed. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a target detection method based on a drone is implemented.
[0175] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0176] In an exemplary embodiment, a computer device is also provided, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the drone-based target detection method described in Example 1.
[0177] Example 4
[0178] An embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the drone-based target detection method described in Embodiment 1 is implemented.
[0179] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0180] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.
Claims
1. A target detection method based on drone, characterized in that: The target detection method based on drone includes: Acquire a captured image; the captured image is obtained by using a drone to capture the urban scene to be detected; Taking the captured image as input, using the trained target detection model to identify the detection target in the captured image to obtain a target detection result; the detection target includes pedestrians and vehicles; the target detection result includes the category and position of each detection target in the captured image; The trained target detection model adopts an improved YOLO model, which is obtained by replacing the C3 module in the YOLO model with a brain neuron redundancy detection module, and the brain neuron redundancy detection module includes: The first convolution block is used to convolve the input feature to obtain the first feature; A segmentation layer, used to segment the first feature to obtain a second feature and a third feature; A convolution residual block, used to convolve the second feature to obtain a fourth feature; A splicing layer, used for splicing the third feature and the fourth feature to obtain a fifth feature; A second convolution block, used to convolve the fifth feature to obtain an output feature; The first convolution block includes a plurality of first convolution layers connected in sequence; the second convolution block includes a plurality of second convolution layers connected in sequence; the convolution residual block includes a plurality of third convolution layers, residual units and a plurality of fourth convolution layers connected in sequence, the residual unit includes a plurality of fifth convolution layers connected in sequence and an addition layer, the input end of the addition layer is also connected to the output end of the last third convolution layer; The number of the first convolutional layers is 1; the number of the second convolutional layers is 1; the number of the third convolutional layers is 1; the number of the fourth convolutional layers is 1; The number of input channels of the first convolutional layer is , the number of output channels of the first convolutional layer is ;in, , is the number of output channels of the brain neuron redundancy detection module, To expand the ratio; The number of input channels of the second convolutional layer is , the number of output channels of the second convolutional layer is ;in, , is the scaling ratio; The number of input channels of the third convolutional layer is , the number of output channels of the third convolutional layer is ; The number of input channels of the fourth convolutional layer is , the number of output channels of the fourth convolutional layer is ; The number of input channels of the fifth convolutional layer is , the number of output channels of the fifth convolutional layer is .
2. The target detection method based on drone according to claim 1, characterized in that: The output end of the first convolution block is connected to the input end of the segmentation layer; the first output end of the segmentation layer is connected to the first input end of the splicing layer; the second output end of the segmentation layer is connected to the input end of the convolution residual block; the output end of the convolution residual block is connected to the second input end of the splicing layer; the output end of the splicing layer is connected to the input end of the second convolution block.
3. The target detection method based on drone according to claim 1, characterized in that: The convolution kernel size of the first convolution layer is 1; the convolution kernel size of the second convolution layer is 1; the convolution kernel size of the third convolution layer is 3; the convolution kernel size of the fourth convolution layer is 1; and the convolution kernel size of the fifth convolution layer is 3.
4. The target detection method based on drone according to claim 1, characterized in that: The target detection result also includes a confidence score for each detected target in the captured image.
5. A target detection device based on a drone, characterized in that: The target detection device based on drone includes: An image acquisition module is used to acquire a captured image; the captured image is obtained by using a drone to capture the urban scene to be detected; An image recognition module, used to use the captured image as input, and use the trained target detection model to identify the detection target in the captured image to obtain a target detection result; the detection target includes pedestrians and vehicles; the target detection result includes the category and position of each detection target in the captured image; The trained target detection model adopts an improved YOLO model, which is obtained by replacing the C3 module in the YOLO model with a brain neuron redundancy detection module, and the brain neuron redundancy detection module includes: The first convolution block is used to convolve the input feature to obtain the first feature; A segmentation layer, used to segment the first feature to obtain a second feature and a third feature; A convolution residual block, used to convolve the second feature to obtain a fourth feature; A splicing layer, used for splicing the third feature and the fourth feature to obtain a fifth feature; A second convolution block, used to convolve the fifth feature to obtain an output feature; The first convolution block includes a plurality of first convolution layers connected in sequence; the second convolution block includes a plurality of second convolution layers connected in sequence; the convolution residual block includes a plurality of third convolution layers, residual units and a plurality of fourth convolution layers connected in sequence, the residual unit includes a plurality of fifth convolution layers connected in sequence and an addition layer, the input end of the addition layer is also connected to the output end of the last third convolution layer; The number of the first convolutional layers is 1; the number of the second convolutional layers is 1; the number of the third convolutional layers is 1; the number of the fourth convolutional layers is 1; The number of input channels of the first convolutional layer is , the number of output channels of the first convolutional layer is ;in, , is the number of output channels of the brain neuron redundancy detection module, To expand the ratio; The number of input channels of the second convolutional layer is , the number of output channels of the second convolutional layer is ;in, , is the scaling ratio; The number of input channels of the third convolutional layer is , the number of output channels of the third convolutional layer is ; The number of input channels of the fourth convolutional layer is , the number of output channels of the fourth convolutional layer is ; The number of input channels of the fifth convolutional layer is , the number of output channels of the fifth convolutional layer is .
6. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the drone-based target detection method according to any one of claims 1 to 4.
7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the drone-based target detection method described in any one of claims 1 to 4 is implemented.
