A YOLOv8-Hire target detection method and system based on a SegNeXt structure and a hierarchical rearranged MLP

Abstract

The application discloses a YOLOv8-Hire target detection method and system based on a SegNeXt structure and a hierarchical rearranged MLP, and the method comprises the following steps: selecting a data set; extracting image features from images in the data set; performing feature fusion on the extracted image features, and establishing a new data set based on the output result; constructing a target detection model; training the target detection model by using the new data set; performing target detection on an image to be detected by using the target detection model; and evaluating the performance of the target detection model by using the image to be detected. The system comprises an image feature extraction module, an image feature fusion module, a model training module, a model performance evaluation module and a target detection module. The application solves the problems that the YOLOv8 network in the prior art cannot effectively capture local information such as detailed information and spatial relationships of objects, and it is difficult to accurately identify and position objects with complex structures in the target detection process.

Description

TECHNICAL FIELD

[0001] The application relates to target detection, in particular to a YOLOv8-Hire target detection method and system based on a SegNeXt structure and a hierarchical rearrangement MLP. BACKGROUND

[0002] A target detection method for finding specific target categories and positions in images is widely used in various fields, such as autonomous driving, remote sensing images, video monitoring, and medical detection.

[0003] YOLOv8 network is a convolutional neural network commonly used for target detection, which converts the target detection problem into a regression problem and directly predicts the category and position information of the target through a single neural network. Compared with other target detection methods such as region-based convolutional neural network R-CNN, YOLOv8 network has higher detection speed. However, the existing YOLOv8 network cannot effectively capture the detailed information of objects and local information such as spatial relationship, and it is difficult to accurately identify and locate objects with complex structures. SUMMARY

[0004] The purpose of the present application is to provide a YOLOv8-Hire target detection method and system based on a SegNeXt structure and a hierarchical rearrangement MLP, which has high target detection accuracy and efficiency.

[0005] Technical scheme: In order to achieve the above purpose, the YOLOv8-Hire target detection method based on the SegNeXt structure and the hierarchical rearrangement MLP comprises the following steps:

[0006] Step S1: select MS COCO 2017 dataset;

[0007] Step S2: replace all C2f modules in the Backbone backbone network of the YOLOv8 network with Hire-Block blocks to form a YOLOv8-Hire network, and use the hierarchical rearrangement MLP in the HireBlock block to segment and recombine the images in the dataset to extract image features;

[0008] Step S3: introduce a new convolutional neural network structure SegNeXt outside the Backbone backbone network of the YOLOv8 network, and use SegNeXt to perform feature fusion on the images processed in step S2, and establish a new dataset based on the output results;

[0009] Step S4: Construct a target detection model based on the YOLOv8-Hire network, introduce a new convolutional neural network structure SegNeXt outside the Backbone backbone network of the network, and replace the CIoU loss function with the ECIoU loss function;

[0010] Step S5: Train the target detection model in step S4 using a new data set;

[0011] Step S6: Use the trained target detection model to detect targets in the image to be detected, and evaluate the performance of the model;

[0012] Wherein, the C2f module in the Backbone backbone network of the YOLOv8 network is replaced by the Hire-Block block in step S2 to form the YOLOv8-Hire network, that is, each C2f module is replaced by a Hire-Block block, wherein each Hire-Block block contains multiple MLPs that can be combined and rearranged at will, and each MLP contains a convolutional layer, a BN layer and a ReLU activation function.

[0013] Wherein, the HireBlock block in step S2 is used to extract image features by segmenting and recombining images in the data set, including the following sub-steps:

[0014] Step S201: Perform pixel padding and stride operation on the images in the data set to convert images of different sizes to a uniform size;

[0015] Step S202: Perform pixel arrangement and channel change on the images after pixel padding and stride operation through convolution operation and activation function, so as to disperse adjacent pixels into different areas and change the relationship between channels, and extract feature representations of different channels;

[0016] Step S203: Segment the image after feature transformation in height and width, rearrange the features of different areas in a certain order, and realize cross-area rearrangement operation;

[0017] Step S204: Perform weighted average on the rearranged image features through linear transformation to obtain comprehensive feature representation.

[0018] In step S3, the new convolutional neural network structure SegNeXt is introduced outside the Backbone backbone network of the YOLOv8-Hire network, specifically: the decoder part of the original convolutional neural network structure SegNeXt is removed, the DWConv deep separable convolution in the encoder, the MSCA attention mechanism, and the MSC module part are recombined into a new convolutional neural network structure SegNeXt, and the new convolutional neural network structure SegNeXt is introduced outside the Backbone backbone network of the YOLOv8-Hire network.

[0019] In step S3, the image processed in step S2 is fused by SegNeXt, including the following substeps:

[0020] Step S301: input the image processed in step S2 into the new convolutional attention network SegNeXt, which uses DWConv deep separable convolution to extract the feature representation of the input image, and then performs convolution operation on each channel of the tensor containing all feature representations of the input image, and then fuses the results of different channels through point-by-point convolution to reduce the amount of calculation and parameters;

[0021] Step S302: based on the feature map generated by the convolution operation in step S301, the new convolutional neural network structure SegNeXt uses the attention mechanism MSCA to divide the feature map into multiple subgraphs, calculate the channel attention weights of different scales in each subgraph, and then weight average all the weights to obtain feature maps of different scales, thereby improving the accuracy of feature representation;

[0022] Step S303: the new convolutional neural network structure SegNeXt uses the MSC module to perform convolution operation on each subgraph to extract context information, which is used to adjust the feature maps of different levels and fuse them to generate the final output result. The output result is a tensor with the same size as the input image, where each element represents the object category to which the pixel point at the corresponding position belongs.

[0023] In step S5, the target detection model in step S4 is trained using a new data set, which means that based on the training set, the model is trained using the gradient descent algorithm to constantly optimize the model parameters until the loss function converges. Specifically: select the learning rate of the model and define the loss function, calculate the value of the loss function according to the current model parameters and training data, calculate the gradient of the loss function with respect to each model parameter, update the current model parameters according to the formula of the gradient descent algorithm, and repeat the iteration until the loss function converges or the maximum number of iterations is reached. The formula of the gradient descent algorithm is:

[0024]

[0025] Wherein, θ(t) shows the model parameter at t moment, that is, the current parameter, θ(t+1) represents the model parameter updated at t+1 moment, that is, the parameter of next iteration round, and alpha represents learning rate, Indicates the gradient of the loss function with respect to theta.

[0026] Wherein, the target detection model trained is used for target detection in the image to be detected in step S6, that is, the target detection model trained is used for target detection on the fused feature map in the image to be detected, and the prediction result of the class and position information of the object in the image is output, and the prediction result is compared with the corresponding image respectively, and the accuracy of the model prediction is calculated.

[0027] Wherein, the performance evaluation of the trained target detection model in step S6 is to detect the performance of the target detection model according to the index average precision mean mAP, parameter quantity Parameters and floating point operation number FLOPs calculated by the model, wherein the higher the mAP, the better the model detection effect, the more the Parameters, the more complex the model, and the less the FLOPs, the faster the model calculation speed.

[0028] The application also provides a detection system applied to the YOLOv8-Hire target detection method based on the SegNeXt structure and the hierarchical rearrangement MLP, which comprises an image feature extraction module, an image feature fusion module, a model training module, a model performance evaluation module and a target detection module.

[0029] Wherein, the image feature extraction module is based on a new convolutional neural network structure SegNeXt to extract features from the image;

[0030] The image feature fusion module is a hierarchical rearrangement MLP, which fuses the features extracted from the feature maps of different scales and different levels to generate feature maps for subsequent target detection;

[0031] The model training module is a target detection model based on the introduction of a new convolutional neural network structure SegNeXt outside the Backbone main network of YOLOv8-Hire network, and the CIoU loss function is replaced by the ECIoU loss function, and the feature map generated by the image feature fusion module is used to train the model;

[0032] The model performance evaluation module is used to evaluate the performance of the trained target detection model by using the image to be detected;

[0033] The target detection module is used to find the class and position information of the target object in the image to be detected by using the trained target detection model.

[0034] Advantages: The present application has the following advantages:

[0035] 1、The target detection model constructed by the present application uses hierarchical rearrangement MLP to extract features of the image, which can better capture local information in the spatial region, thereby improving the recognition and positioning ability of the model for the target, and then a novel convolutional neural network structure SegNeXt is used to fuse the features of the image to generate a feature map for subsequent target detection, further improving the detection accuracy of the model.

[0036] 2、The ECIoU loss function used by the target detection model constructed by the present application is more accurate in positioning the bounding box of the target, which can reduce the error rate of the model, and also reduces the oscillation phenomenon of the model in the optimization process, improving the detection efficiency of the model. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 is a schematic diagram of the method of the present application. DETAILED DESCRIPTION

[0038] The technical solutions of the present application will be described in detail below in combination with embodiments and drawings.

[0039] As shown in the drawings, Figure 1 A YOLOv8-Hire target detection method based on a SegNeXt structure and a hierarchical rearrangement MLP according to the present application comprises the following steps:

[0040] Step S1: Select the MS COCO 2017 dataset;

[0041] Step S2: Replace all C2f modules in the Backbone backbone network of the YOLOv8 network with Hire-Block blocks to form a YOLOv8-Hire network, and use the hierarchical rearrangement MLP in the HireBlock block to extract image features by segmenting and recombining the images in the dataset;

[0042] Step S3: Introduce a novel convolutional neural network structure SegNeXt outside the Backbone backbone network of the YOLOv8 network, and use SegNeXt to fuse the features of the images processed in step S2, and establish a new dataset from the output results;

[0043] Step S4: Construct a target detection model based on the YOLOv8-Hire network, introduce a novel convolutional neural network structure SegNeXt outside the Backbone backbone network of the network, and replace the CIoU loss function with the ECIoU loss function;

[0044] Step S5: Train the target detection model in step S4 using the new dataset;

[0045] Step S6: performing target detection on the image to be detected using the trained target detection model, and evaluating the performance of the model;

[0046] In step S2, replacing all C2f modules in the Backbone backbone network of the YOLOv8 network with Hire-Block blocks to form a YOLOv8-Hire network means replacing each C2f module with a Hire-Block block, wherein each Hire-Block block includes multiple MLPs that can be combined and rearranged at will, and each MLP includes a convolutional layer, a BN layer, and a ReLU activation function.

[0047] In step S2, the HireBlock block is used to extract image features by segmenting and recombining images in the data set, including the following sub-steps:

[0048] Step S201: performing pixel padding and stride operation on the images in the data set to convert images of different sizes to a uniform size;

[0049] Step S202: performing pixel arrangement and channel change on the images after pixel padding and stride operation through convolution operation and activation function, so as to disperse adjacent pixels into different areas and change the relationship between channels, and extract feature representations of different channels;

[0050] Step S203: segmenting the image after feature transformation in height and width, rearranging the features of different areas in a certain order, and realizing cross-area rearrangement operation;

[0051] Step S204: performing weighted average on the rearranged image features through linear transformation to obtain comprehensive feature representation.

[0052] In step S3, the new convolutional neural network structure SegNeXt is introduced outside the Backbone backbone network of the YOLOv8-Hire network, specifically: removing the decoder part of the original convolutional neural network structure SegNeXt, recombining the DWConv depth separable convolution, MSCA attention mechanism, and MSC module part in the encoder into a new convolutional neural network structure SegNeXt, and introducing the new convolutional neural network structure SegNeXt outside the Backbone backbone network of the YOLOv8-Hire network.

[0053] In step S3, the SegNeXt is used to perform feature fusion on the images processed in step S2, including the following sub-steps:

[0054] Step S301: input the image processed in step S2 into the new convolutional attention network SegNeXt, which uses DWConv deep separable convolution to extract the feature representation of the input image, and separately convolves each channel of the tensor containing all the feature representations of the input image, and then fuses the results of different channels through pointwise convolution to reduce the amount of calculation and the number of parameters;

[0055] Step S302: based on the feature map generated by the convolution operation in step S301, the new convolutional neural network structure SegNeXt uses the attention mechanism MSCA to divide the feature map into multiple sub-maps, calculate the channel attention weights of different scales in each sub-map, and then weight average all the weights to obtain feature maps of different scales, thereby improving the accuracy of feature representation.

[0056] Step S303: the new convolutional neural network structure SegNeXt uses the MSC module to perform convolution operation on each sub-map to extract context information, which is used to adjust the feature maps of different levels and fuse them to generate the final output result. The output result is a tensor with the same size as the input image, where each element represents the object category to which the pixel at the corresponding position belongs.

[0057] In step S5, the target detection model in step S4 is trained using a new data set, which means that the model is trained based on the training set using the gradient descent algorithm to optimize the model parameters until the loss function converges. Specifically, the learning rate of the model is selected and the loss function is defined. The value of the loss function is calculated based on the current model parameters and the training data, and the gradient of the loss function with respect to each model parameter is calculated. The current model parameters are updated according to the formula of the gradient descent algorithm, and the iteration is repeated until the loss function converges or the maximum number of iterations is reached. The formula of the gradient descent algorithm is:

[0058]

[0059] where θ(t) represents the model parameters at time t, i.e. the current parameters; θ(t+1) represents the updated model parameters at time t+1, i.e. the parameters of the next iteration round; α represents the learning rate, which represents the gradient of the loss function with respect to θ.

[0060] In step S6, the trained target detection model is used to perform target detection on the image to be detected, which means that the trained target detection model is used to perform target detection on the fused feature map in the image to be detected, and the prediction result of the class and position information of the object in the image is output. The prediction result is compared with the corresponding image to calculate the accuracy of the model prediction.

[0061] The performance evaluation of the trained target detection model in step S6 is to detect the performance of the target detection model according to the index average precision mean mAP, parameter quantity Parameters and floating point operation number per second FLOPs calculated by the model, wherein the higher the mAP, the better the model detection effect, the more the Parameters, the more complex the model, and the less the FLOPs, the faster the model calculation speed.

[0062] The application further provides a detection system applied to the YOLOv8-Hire target detection method based on the SegNeXt structure and the hierarchical rearrangement MLP, comprising an image feature extraction module, an image feature fusion module, a model training module, a model performance evaluation module and a target detection module.

[0063] The image feature extraction module is based on a new convolutional neural network structure SegNeXt to extract features of the image.

[0064] The image feature fusion module is based on a hierarchical rearrangement MLP to fuse the features extracted from feature maps of different scales and different levels to generate feature maps for subsequent target detection.

[0065] The model training module is based on a target detection model constructed by introducing a new convolutional neural network structure SegNeXt outside the Backbone backbone network of the YOLOv8-Hire network and replacing a CIoU loss function with an ECIoU loss function, and the model is trained by using the feature maps generated by the image feature fusion module.

[0066] The model performance evaluation module is to evaluate the performance of the trained target detection model by using images to be detected.

[0067] The target detection module is to find the category and position information of a target object in the image to be detected by using the trained target detection model.

[0068] The target detection model constructed by the application can better capture local information in a spatial region to improve the recognition and positioning ability of the model for a target, and further improve the detection accuracy of the model by using the new convolutional neural network structure SegNeXt to fuse the features of the image and generate feature maps for subsequent target detection.

[0069] The ECIoU loss function used by the target detection model constructed by the application can more accurately position the bounding box of target detection, can reduce the error rate of the model, and further improve the accuracy of target detection, reduces the shock phenomenon of the model in the optimization process, and improves the training efficiency of the model.

Claims

1. A YOLOv8-Hire target detection method based on a SegNeXt structure and a hierarchical rearranged MLP, characterized in that, The method comprises the following steps: Step S1: selecting the MS COCO 2017 dataset; Step S2: replacing all C2f modules in the Backbone backbone network of the YOLOv8 network with Hire-Block blocks to form a YOLOv8-Hire network, and using the hierarchical rearranged MLP in the Hire-Block blocks to extract image features by segmenting and recombining the images in the dataset; Step S3: introducing a new convolutional neural network structure SegNeXt outside the Backbone backbone network of the YOLOv8 network, and using the SegNeXt to perform feature fusion on the images processed in step S2, and establishing a new dataset from the output results; The new convolutional neural network structure SegNeXt introduced outside the Backbone backbone network of the YOLOv8 network is specifically: removing the decoder part of the original convolutional neural network structure SegNeXt, and recombining a new convolutional neural network structure SegNeXt by using the DWConv depth separable convolution, MSCA attention mechanism and MSC module part in the encoder, and introducing the new convolutional neural network structure SegNeXt outside the Backbone backbone network of the YOLOv8-Hire network; The feature fusion of the SegNeXt on the images processed in step S2 comprises the following sub-steps: Step S301: inputting the images processed in step S2 into the new convolutional attention network SegNeXt, which uses the DWConv depth separable convolution to extract the feature representation of the input image, and separately performs convolution operation on each channel of the tensor containing all feature representations of the input image, and then fuses the results of different channels through point-by-point convolution to reduce the amount of calculation and parameters; Step S302: based on the feature map generated by the convolution operation in step S301, the new convolutional neural network structure SegNeXt uses the attention mechanism MSCA to divide the feature map into multiple sub-maps, calculates the channel attention weights of different scales in each sub-map, and performs weighted averaging on all weights, thereby obtaining feature maps of different scales and improving the accuracy of feature representation; Step S303: the new convolutional neural network structure SegNeXt uses the MSC module to perform convolution operation on each sub-map to extract context information, which is used to adjust the feature maps of different levels and perform fusion to generate the final output result, and the output result is a tensor with the same size as the input image, wherein each element represents the object category to which the pixel point at the corresponding position belongs; Step S4: constructing a target detection model based on the YOLOv8-Hire network, introducing the new convolutional neural network structure SegNeXt outside the Backbone backbone network of the network, and replacing the CIoU loss function with the ECIoU loss function; Step S5: training the target detection model in step S4 using the new dataset; Step S6: using the trained target detection model to perform target detection on the image to be detected, and evaluating the performance of the model.

2. The SegNeXt structure and hierarchical rearranged MLP based YOLOv8-Hire object detection method according to claim 1, characterized in that: The step S2 of replacing all C2f modules in the Backbone backbone network of the YOLOv8 network with Hire-Block blocks to form a YOLOv8-Hire network refers to replacing each C2f module with a Hire-Block block, wherein each Hire-Block block includes multiple MLPs that can be combined and rearranged at will, and each MLP includes a convolutional layer, a BN layer and a ReLU activation function.

3. The SegNeXt structure and hierarchical rearranged MLP based YOLOv8-Hire object detection method according to claim 1, characterized in that: The step S2 of extracting image features by using the hierarchical rearrangement MLP in the Hire-Block block to segment and recombine the images in the data set includes the following sub-steps: Step S201: pixel padding and stride operation are performed on the images in the data set to convert images of different sizes to a uniform size; Step S202: pixel arrangement and channel change are performed on the images after pixel padding and stride operation through convolution operation and activation function, so as to disperse adjacent pixels into different areas and change the relationship between channels, and extract feature representations of different channels; Step S203: the image after feature transformation is segmented in height and width, and the features of different areas are rearranged in a certain order to realize cross-area rearrangement operation; Step S204: the rearranged image features are weighted and averaged by linear transformation to obtain a comprehensive feature representation.

4. The SegNeXt structure and hierarchical rearranged MLP based YOLOv8-Hire object detection method according to claim 1, characterized in that: The step S5 of training the target detection model in step S4 using a new data set refers to training the model based on the training set using the gradient descent algorithm, constantly optimizing the model parameters until the loss function converges, and specifically: selecting the learning rate of the model and defining the loss function, calculating the value of the loss function according to the current model parameters and training data, calculating the gradient of the loss function with respect to each model parameter, updating the current model parameters according to the formula of the gradient descent algorithm, and repeating the iteration until the loss function converges or the maximum number of iterations is reached.

5. The SegNeXt structure and hierarchical rearranged MLP based YOLOv8-Hire object detection method according to claim 4, characterized in that: The formula of the gradient descent algorithm is: = - *∇L( )； wherein, model parameters at time t, i.e. the current parameters; updated model parameters at time t+1, i.e. the parameters of the next iteration round, denotes the learning rate, ∇L( ) denotes the gradient of the loss function with respect to the parameters.

6. The SegNeXt structure and hierarchical rearranged MLP based YOLOv8-Hire object detection method according to claim 1, characterized in that: The step S6 of using the trained target detection model to perform target detection on the image to be detected refers to using the trained target detection model to perform target detection on the fused feature map in the image to be detected, outputting the prediction results of the class and position information of the objects in the image, and comparing the prediction results with the corresponding images respectively to calculate the accuracy of the model prediction.

7. The SegNeXt structure and hierarchical rearranged MLP based YOLOv8-Hire object detection method according to claim 1, characterized in that: The step S6 of evaluating the performance of the trained target detection model refers to detecting the performance of the target detection model according to the average precision mean mAP, parameter quantity Parameters and floating point operations per second FLOPs calculated by the model, wherein the higher the mAP, the better the model detection effect, the more the Parameters, the more complex the model, and the fewer the FLOPs, the faster the model calculation speed.

8. A detection system applied to the YOLOv8-Hire target detection method based on the SegNeXt structure and hierarchical rearranged MLP of claim 1. The system includes an image feature extraction module, an image feature fusion module, a model training module, a model performance evaluation module and a target detection module. The image feature extraction module is based on a new convolutional neural network structure SegNeXt to extract features of the image; The image feature fusion module layer rearrangement MLP fuses the features extracted from the feature maps of different scales and different levels to generate feature maps for subsequent target detection; The model training module is a target detection model constructed based on the introduction of a new convolutional neural network structure SegNeXt outside the Backbone backbone network of YOLOv8-Hire, and the replacement of the CIoU loss function with the ECIoU loss function. The model is trained using the feature maps generated by the image feature fusion module; The model performance evaluation module refers to evaluating the performance of the trained target detection model using the image to be detected; The target detection module refers to finding the class and location information of the target object in the image to be detected using the trained target detection model.

Images