An image target detection method and system based on improved YOLOv8
By improving the backbone and head structure of YOLOv8, and adopting a novel recurrent vision module, CReToNeXt module, and LADH detection head, the network structure of YOLOv8 is optimized, solving the problem of backbone feature extraction delay and achieving more efficient target detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HUAIYIN INSTITUTE OF TECHNOLOGY
- Filing Date
- 2023-07-20
- Publication Date
- 2026-06-26
AI Technical Summary
The existing YOLOv8 backbone network has long feature extraction latency and inference time, resulting in low efficiency.
All C2f modules in the YOLOv8 backbone were replaced with novel recurrent vision modules, and the head portion was replaced with a CReToNeXt module and an LADH detection head. The network structure was optimized by combining feature pyramid multi-scale fusion and asymmetric structure.
It improves the efficiency and accuracy of target detection, and reduces computational latency and complexity.
Smart Images
Figure CN116844030B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer vision technology, and in particular to an image target detection method and system based on an improved YOLOv8. Background Technology
[0002] Image object detection aims to identify the category and location of specified objects in an image and is now widely used in various fields, such as autonomous driving, remote sensing images, video surveillance, and medical detection. YOLO, as a classic object detection network, has undergone continuous version updates since 2016, evolving from YOLOv1 to YOLOv8. The YOLO series of object detection methods has developed alongside the advancement of one-stage object detection and has become a typical representative of single-stage methods. The current technology, YOLOv8, includes a backbone, a head, and a detection head, such as... Figure 1 As shown, its backbone network has a long delay in feature extraction and inference time, resulting in low efficiency, which needs to be improved. Summary of the Invention
[0003] Purpose of the invention: This invention addresses the problems existing in the prior art by providing a more efficient image target detection method and system based on the improved YOLOv8.
[0004] Technical solution: The image target detection method based on improved YOLOv8 described in this invention includes:
[0005] The image to be detected is input into a trained target detection network to obtain the detected target; wherein, the target detection network replaces all C2f modules in the YOLOv8 backbone with a novel recurrent vision module; wherein, the novel recurrent vision module includes a convolutional submodule, a local self-attention submodule, a first multilayer perceptron submodule, a global self-attention submodule, a second multilayer perceptron submodule, and a feature pyramid multi-scale fusion submodule connected in sequence, and the output of each submodule is also connected to the next submodule through residuals.
[0006] Preferably, the target detection network can also replace all C2f modules in the YOLOv8 header with CReToNeXt modules, wherein CReToNeXt is sequentially connected to CSP sub-modules and several reverse basic sub-modules, and each reverse basic sub-module is further connected to the next reverse basic sub-module through residuals.
[0007] Preferably, the target detection network can also replace all the detectors in the YOLOv8 detector head with an asymmetric LADH detector head, which includes a first convolutional layer, a first depthwise separable convolutional layer, a second depthwise separable convolutional layer, a second convolutional layer, and a Segmoid activation function connected in sequence.
[0008] Furthermore, the method also includes a step of training the object detection network. The object detection network is trained using a dataset including a training set, a test set, and a validation set.
[0009] The image target detection system based on the improved YOLOv8 described in this invention includes:
[0010] The image acquisition unit is used to acquire the image to be detected;
[0011] The target detection network is used to extract the target to be detected from the image to be detected. The target detection network replaces all C2f modules in the YOLOv8 backbone with a novel recurrent vision module. The novel recurrent vision module includes a convolutional submodule, a local self-attention submodule, a first multilayer perceptron submodule, a global self-attention submodule, a second multilayer perceptron submodule, and a feature pyramid multi-scale fusion submodule connected in sequence. The output of each submodule is also connected to the next submodule through residuals.
[0012] Preferably, the target detection network can also replace all C2f modules in the YOLOv8 header with CReToNeXt modules, wherein CReToNeXt is sequentially connected to CSP sub-modules and several reverse basic sub-modules, and each reverse basic sub-module is further connected to the next reverse basic sub-module through residuals.
[0013] Preferably, the target detection network can also replace all the detectors in the YOLOv8 detector head with an asymmetric LADH detector head, which includes a first convolutional layer, a first depthwise separable convolutional layer, a second depthwise separable convolutional layer, a second convolutional layer, and a Segmoid activation function connected in sequence.
[0014] Preferably, the system further includes a training unit for training the object detection network. The object detection network is trained using a dataset including a training set, a test set, and a validation set.
[0015] Beneficial effects: Compared with the prior art, the significant advantage of this invention is that it improves YOLOv8, resulting in higher target detection efficiency. Attached Figure Description
[0016] Figure 1 This is a diagram of the network structure of YOLOv8 in the existing technology;
[0017] Figure 2 This is a structural diagram of the backbone network of the target detection network of this invention;
[0018] Figure 3This is a structural diagram of the novel cyclic vision module of the present invention;
[0019] Figure 4 This is a structural diagram of the head of the target detection network of the present invention;
[0020] Figure 5 This is a structural diagram of CSP;
[0021] Figure 6 This is a structural diagram of the detection head of the target detection network of the present invention;
[0022] Figure 7 This is a diagram showing the connection method of the detection head of the target detection network of the present invention. Detailed Implementation
[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0024] This invention provides an image target detection method based on an improved YOLOv8. The method inputs the image to be detected into a trained target detection network to obtain the detected target.
[0025] In this embodiment, the target detection network replaces all C2f modules in the YOLOv8 backbone with novel recurrent vision modules. The replaced backbone is as follows: Figure 2 As shown. Figure 3As shown, the novel recurrent vision module includes a convolutional submodule, a local self-attention submodule, a first multilayer perceptron (MLP) submodule, a global self-attention submodule, a second multilayer perceptron (MLP) submodule, and a feature pyramid multi-scale fusion submodule connected in sequence. The output of each submodule is further connected to the next submodule via residuals. The local self-attention submodule is a local block attention mechanism (block-sa), and the global self-attention submodule is a global grid attention mechanism (Grid-SA). The novel recurrent vision module is an improvement on the RVT Block (RecurrentVision Transformers Block), which is described in the literature "Recurrent VisionTransformers for Obiect Detection with Event Cameras" and will not be elaborated upon here. The novel recurrent vision module, based on the RVT Block, improves its LSTM into a Feature Pyramid Multi-Scale Fusion Submodule (FPP). This eliminates the need for the LSTM to reuse the state from the previous time step. The FPP achieves multi-scale object detection by fusing features from different levels. Starting from higher levels, it upsamples lower-level features to achieve the same resolution as higher-level features. Then, it fuses the upsampled features with the higher-level features to obtain a multi-scale feature representation. This allows the algorithm to simultaneously capture target information at different scales, enabling the FPP to achieve more accurate object detection at various scales and improving the algorithm's performance. The novel recurrent vision module also reduces latency and improves feature extraction efficiency within the YOLOv8 backbone.
[0026] In other embodiments, to further improve efficiency, the object detection network can also replace all C2f modules in the YOLOv8 header with CReToNeXt modules, as shown in the following header structure. Figure 4 As shown. The CReToNeXt includes CSP sub-modules (Cross Stage Partial Network, CSPNet) and several reverse base (BasicBlock_3x3_Reverse) sub-modules connected in sequence. Each reverse base sub-module is also residually connected to the next reverse base sub-module. The CSP sub-module structure is as follows: Figure 5As shown, it integrates gradient changes from start to finish into the feature map, reducing computation while maintaining accuracy. It's primarily used to fuse features from different branches. It divides the input features into two branches, then performs further processing on one branch, such as convolution and batch normalization. Finally, the features from the two branches are concatenated by channel to achieve feature fusion. The CSP method can reduce model computation and improve running speed without compromising accuracy. BasicBlock is a network structure used in ResNet. Multiple inverse basic sub-modules can be used to enhance feature representation. These blocks perform multiple convolution operations on the input features through stacking and use residual connections to preserve information from the original features. In the middle part of the CReToNeXt module, SPP (Spatial Pyramid Pooling) pooling layers can be added. SPP pooling layers can perform pooling operations on features at different scales and concatenate the pooling results, enhancing the multi-scale representation capability of features. The CReToNeXt module also includes some convolution operations and channel number adjustment to adjust the feature dimension and channel number to adapt to subsequent network operations. Adding the CReToNeXt structure to the head of YOLOv8 enhances the model's ability to extract features from targets and improves detection performance. It effectively captures feature information at different scales and improves the model's perceptual and expressive capabilities through feature fusion and enhancement.
[0027] In other embodiments, considering the large number of model parameters and high computational complexity of the YOLOv8 detection head, the object detection network of this invention can also replace the entire YOLOv8 detection head with an asymmetric LADH (Lightweight and Adaptive Detection Head) detection head. The YOLOv8 detection head is a symmetric structure, such as... Figure 1 As shown, the LADH detection head of this invention has an asymmetric structure, including a first convolutional layer (conv), a first depthwise separable convolutional layer (DWConv), a second depthwise separable convolutional layer (DWConv), a second convolutional layer (conv), and a Segmoid activation function connected in sequence, as follows: Figure 6 As shown. In the proposed LADH, the network is divided according to various task types, and enters three LADH detectors through three different network paths. The connection method in YOLOv8 is as follows. Figure 7As shown. The first convolutional layer is a 3x3 convolutional layer that changes the number of channels in the input feature map from x to c2*2. It helps extract features and increases the number of channels to better capture target information. The first depthwise separable convolutional layer DWConv3*3 first applies a convolution operation to each input channel separately, and then combines the features between channels. This helps reduce computation and improve the efficiency of the model. The second depthwise separable convolutional layer DWConv 3*3 reduces the number of channels in the input feature map from c2 to c2. Similar to the previous step, this layer continues to reduce the number of channels and extract higher-level features. The second convolutional layer conv1*1 convolutional layer changes the number of channels in the input feature map from c2 to 4*self.reg_max and is responsible for predicting the coordinate information of the bounding box. A series of convolutional layers and depthwise separable convolutional layers are combined to extract features from the input feature map and generate the coordinate information of the bounding box. The role of each part is to progressively extract and transform features in order to accurately predict the position and size of the target. Finally, the Segmoid activation function is used to predict the class, compressing the input values to a range between 0 and 1. Its output represents the probability distribution for each class. The detection results will include the transformed bounding box coordinates and the probability values for each class. LADH reduces model parameters and computational complexity.
[0028] Those skilled in the art will understand that the target detection network is trained before detection. This embodiment uses the MS COCO 2017 dataset, which has been pre-divided into training, testing, and validation sets. The training method is well-known in the art and will not be described in detail here.
[0029] This invention also provides an image target detection system based on an improved YOLOv8, comprising:
[0030] The image acquisition unit is used to acquire the image to be detected;
[0031] The target detection network is used to extract the target to be detected from the image to be detected. The target detection network replaces all C2f modules in the YOLOv8 backbone with a novel recurrent vision module. The novel recurrent vision module includes a convolutional submodule, a local self-attention submodule, a first multilayer perceptron submodule, a global self-attention submodule, a second multilayer perceptron submodule, and a feature pyramid multi-scale fusion submodule connected in sequence. The output of each submodule is also connected to the next submodule through residuals.
[0032] In other embodiments, the target detection network may replace all C2f modules in the YOLOv8 header with CReToNeXt modules, wherein CReToNeXt is sequentially connected to CSP sub-modules and several reverse basic sub-modules, and each reverse basic sub-module is further connected to the next reverse basic sub-module through residuals.
[0033] In other embodiments, the target detection network may replace all of the YOLOv8 detection head with an asymmetric LADH detection head, which includes a first convolutional layer, a first depthwise separable convolutional layer, a second depthwise separable convolutional layer, a second convolutional layer, and a Segmoid activation function connected in sequence.
[0034] This system can be implemented using software and / or hardware and can be configured in a terminal device. The system provided in this embodiment of the invention can be used to execute the method provided in Embodiment 1 of the invention, and has the corresponding functions and beneficial effects of executing the method.
[0035] It is worth noting that in the above-described embodiments of the system, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of the present invention.
[0036] The embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art will clearly understand that each implementation can be achieved using software plus necessary general-purpose hardware platforms, or it can be implemented solely through hardware, as long as the function or purpose can be achieved.
[0037] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish different objects, not to describe a specific order. The reference to "embodiment" herein means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.
Claims
1. An image target detection method based on an improved YOLOv8, characterized in that... The method includes: The image to be detected is input into a trained object detection network to obtain the detected objects. The target detection network replaces all C2f modules in the YOLOv8 backbone with a novel recurrent vision module. The novel recurrent vision module includes a convolutional submodule, a local self-attention submodule, a first multilayer perceptron submodule, a global self-attention submodule, a second multilayer perceptron submodule, and a feature pyramid multi-scale fusion submodule connected in sequence. The output of each submodule is also connected to the next submodule through residuals. The target detection network also replaces all the detection heads in YOLOv8 with asymmetric LADH detection heads, which include a first convolutional layer, a first depthwise separable convolutional layer, a second depthwise separable convolutional layer, a second convolutional layer, and a sigmoid activation function connected in sequence. In the proposed LADH, the object detection network is divided according to various task types. It enters three LADH detection heads through three different network paths. The first convolutional layer is a 3x3 convolutional layer used to increase the number of channels in the input feature map. The first depthwise separable convolutional layer is used to apply convolution operations to each input channel and then combine the features between channels. The second depthwise separable convolutional layer is used to reduce the number of channels in the input feature map and extract higher-level features. The second convolutional layer is a 1x1 convolutional layer used to predict the coordinate information of the bounding box. The Sigmoid activation function is used to predict the class, compressing the input values to the range between 0 and 1. Its output represents the probability distribution of each class. The detection result will include the transformed bounding box coordinates and the probability values of each class. The target detection network also replaces all C2f modules in the YOLOv8 header with CReToNeXt modules. CReToNeXt includes CSP sub-modules and several reverse basic sub-modules connected in sequence. Each reverse basic sub-module is also connected to the next reverse basic sub-module through residuals.
2. The image target detection method based on improved YOLOv8 according to claim 1, characterized in that: The method also includes the step of training the object detection network.
3. The image target detection method based on improved YOLOv8 according to claim 2, characterized in that: The object detection network is trained using a dataset that includes a training set, a test set, and a validation set.
4. An image target detection system based on an improved YOLOv8, characterized in that... include: The image acquisition unit is used to acquire the image to be detected; An object detection network is used to extract the target to be detected from the image to be detected. The object detection network replaces all C2f modules in the YOLOv8 backbone with a novel recurrent vision module. The novel recurrent vision module includes a convolutional submodule, a local self-attention submodule, a first multilayer perceptron submodule, a global self-attention submodule, a second multilayer perceptron submodule, and a feature pyramid multi-scale fusion submodule connected in sequence. The output of each submodule is also connected to the next submodule through residuals. The target detection network also replaces all the detection heads in YOLOv8 with asymmetric LADH detection heads, which include a first convolutional layer, a first depthwise separable convolutional layer, a second depthwise separable convolutional layer, a second convolutional layer, and a sigmoid activation function connected in sequence. In the proposed LADH, the object detection network is divided according to various task types. It enters three LADH detection heads through three different network paths. The first convolutional layer is a 3x3 convolutional layer, which is used to increase the number of channels in the input feature map. The first depthwise separable convolutional layer is used to apply convolution operations to each input channel and then combine the features between channels. The second depthwise separable convolutional layer is used to reduce the number of channels in the input feature map and extract higher-level features. The second convolutional layer is a 1x1 convolutional layer, which is used to predict the coordinate information of the bounding box. The sigmoid activation function is used to predict the class, compressing the input values to the range between 0 and 1. Its output represents the probability distribution of each class. The detection result will include the transformed bounding box coordinates and the probability values of each class. The target detection network also replaces all C2f modules in the YOLOv8 header with CReToNeXt modules. CReToNeXt includes CSP sub-modules and several reverse basic sub-modules connected in sequence. Each reverse basic sub-module is also connected to the next reverse basic sub-module through residuals.
5. The image target detection system based on improved YOLOv8 according to claim 4, characterized in that: The system also includes a training unit for training the target detection network.
6. The image target detection system based on the improved YOLOv8 according to claim 5, characterized in that: The object detection network is trained using a dataset that includes a training set, a test set, and a validation set.